Lab 05 — RAG from scratch

Scaffold: 3/5. No managed vector store, no Knowledge Base. Just a model, a handful of documents, and the retrieval step, which you write.

The scenario

You have used a Knowledge Base in the reading; now build the thing it hides. A support assistant must answer questions about Greenbox (a fictional product, so the model cannot know the answers from training) using only five short documents. The point is to see retrieval with nothing in the way: embed the question, compare it to the documents, pick the closest, ground the answer in them.

The requirement

• A question the documents cover is answered from them (“when does my box arrive?” -> Tuesdays and Fridays), and the response lists which documents it used.
• A question the documents do not cover gets an honest “I do not know”, not an invented answer.

What’s provided

• template.yaml — a Lambda with permission to call Bedrock for both embeddings and generation.
• knowledge.py — the five documents.
• src/handler.py — the document embedding (on cold start), the _embed() and _cosine() helpers, and the generation call that grounds the answer in the retrieved context. The gap is retrieve().
• solution/handler.py — the reference answer.
• scripts/ — deploy, test (answerable and unanswerable questions), teardown.

Your task

Implement retrieve(query, k) in src/handler.py:

1. Embed the query with _embed(query).
2. Score it against every document vector with _cosine().
3. Return the top k documents, best first.

Four lines. Everything else is wired. This is the exact loop a vector store runs for you at scale, an approximate-nearest-neighbour search over embeddings; here you do it by brute force so the shape is clear.

Run it

# Prerequisite: Model access enabled for BOTH the generation model and the
# Titan embedding model, in your region.
./scripts/deploy.sh
./scripts/test.sh
./scripts/teardown.sh

What success looks like

Before you fill retrieve(), the function raises NotImplementedError. After, “when does my box arrive?” returns Tuesdays and Fridays with sources naming the delivery-days document, and the carbon-footprint question returns an honest refusal because no document supports an answer.

If it fails

• AccessDeniedException — enable Model access for both models; the embedding model is a separate grant from the generation model.
• ValidationException about on-demand throughput / “isn’t supported” — the generation model needs a cross-region inference profile in this region. Redeploy with the profile id, for example MODEL_ID=us.amazon.nova-lite-v1:0 ./scripts/deploy.sh.
• Wrong document retrieved — check you are sorting by cosine score descending and returning the document dicts, not the scores.
• The model answers anyway when it should not — that is the grounding system prompt at work only if you actually pass the retrieved context; make sure retrieve() returns real hits.

Reveal the solution

SRC=solution ./scripts/deploy.sh && ./scripts/test.sh

What you just learned

• Retrieval is embed, compare, rank. A vector store makes it fast at scale with an approximate-nearest-neighbour index, but the operation is the cosine search you just wrote.
• Embedding is a separate model from generation, with its own model id and its own Model-access grant.
• Grounding is two parts: retrieve the right context, and instruct the model to use only it (and to admit when it cannot). Either half alone is not RAG.
• Naming the sources used is nearly free once you retrieve, and it is what makes an answer auditable.

Next

Lab 06 — Wire a tool the model can call. You move from stuffing retrieved text into a prompt to letting the model call a tool, run it, and read the result back: function calling with a real execution loop, the mechanics a managed agent automates.