Lab: Build RAG From Scratch

This is one of the hands-on labs that run alongside these posts. The scaffolding keeps fading: earlier labs were a line or two, this one hands you the whole pipeline except the retrieval step. The full lab is in lab-05-rag-from-scratch.zip.

Before your first lab, do the one-time, once-per-account setup: run the zip’s preflight.sh to confirm your account is ready, then deploy the lab reaper, a standing backstop that auto-deletes any lab you forget to tear down after 24 hours.

The scenario

You have used a Knowledge Base in the reading. Now build the thing it hides. A support assistant has to answer questions about Greenbox, a fictional product the model cannot know from training, using only five short documents. Doing it with nothing in the way, no managed vector store, is the fastest way to see what retrieval actually is.

What you’re given

A Lambda that can call Bedrock for both embeddings and generation, the five documents in knowledge.py, the document-embedding step (done on cold start), the _embed() and _cosine() helpers, and the generation call that grounds the answer in whatever context it is handed. The one gap is retrieve().

Your task

Turn a question into the best-matching documents. retrieve(query, k) has three moves: embed the query with _embed(), score that vector against every (doc, vector) pair in _DOC_VECTORS with _cosine(), and return the k highest-scoring document dicts, best first. A sort keyed on the score, descending, and a slice is all the ranking machinery it takes; four lines cover it.

That is the whole of retrieval: embed the query, score it against every document with cosine similarityCosine similarityA measure of how closely two vectors point the same way, used as the default score for “how related is this text?”. , take the top few. A vector store does exactly this, just with an approximate-nearest-neighbourANNIndex structures (HNSW graphs, IVF partitions) that answer the k-nearest-neighbours question fast by giving up guaranteed exactness – recall becomes a tunable knob rather than a certainty. index so it stays fast at millions of documents instead of five.

Deploy and prove it

cd lab-05-rag-from-scratch
./scripts/deploy.sh
./scripts/test.sh
./scripts/teardown.sh

“When will my box arrive?” comes back with Tuesdays and Fridays and names the delivery-days document as its source. “What is Greenbox’s carbon footprint?” comes back with an honest “I do not know”, because no document supports an answer and the grounding prompt forbids inventing one.

When you want the reference answer, deploy it with SRC=solution ./scripts/deploy.sh, or unfold it here:

def retrieve(query, k=2):
    qv = _embed(query)
    scored = [(_cosine(qv, vec), doc) for doc, vec in _DOC_VECTORS]
    scored.sort(key=lambda pair: pair[0], reverse=True)
    return [doc for _, doc in scored[:k]]

The ideas the exam cares about

• Retrieval is embed, compare, rank. Every vector store, OpenSearch, pgvector, S3 Vectors, runs this same operation; the differences are speed, scale, and filtering, not the fundamental step. Building it by hand is why the vector-store choices click into place.
• Embedding is a separate model from generation, with its own model id and its own Model-access grant. Getting the embedding model wrong, or its distance metric, quietly wrecks retrieval before generation ever runs.
• Grounding is two halves. Retrieve the right context, and instruct the model to answer only from it and to admit when it cannot. Skip the second half and a good retrieval still lets the model wander; skip the first and there is nothing to ground on.
• Naming the sources is nearly free once you retrieve, and it is what turns an answer into an auditable one, the foundation of a citations-required assistant.

What’s worth remembering

1. RAG retrieval is a cosine (or other distance) search over embeddings; a vector store makes it fast, it does not change what it is.
2. Embedding and generation are different models with separate access grants; the embedding choice and its distance metric decide retrieval quality.
3. Grounding needs both the retrieved context and a system instruction to use only it and to refuse when it is missing.
4. Returning the source ids alongside the answer costs nothing extra and makes the answer auditable.
5. An honest “I do not know” when the context lacks the answer is a feature, and it comes from the grounding instruction, not the model’s goodwill.
6. Once this is clear, a managed Knowledge Base is just this loop run at scale with sync, chunkingChunkingSplitting documents into retrievable pieces before embedding them – small enough to match precisely, big enough to still make sense. , and an index bolted on.