Lab: Build RAG From Scratch

This is one of the hands-on labs that run alongside these posts. The scaffolding keeps fading: earlier labs were a line or two, this one hands you the whole pipeline except the retrieval step. The full lab is in lab-05-rag-from-scratch.zip.

Before your first lab, do the one-time, once-per-account setup: run the zip’s preflight.sh to confirm your account is ready, then deploy the lab reaper, a standing backstop that auto-deletes any lab you forget to tear down after 24 hours.

The scenario

You have used a Knowledge Base in the reading. Now write the retrieval step it runs for you. A support assistant has to answer questions about Greenbox, a fictional product that appears in no model’s training data, using only five short documents. With no managed vector store in the way, retrieval is the only moving part.

What you’re given

A Lambda that can call Bedrock for both embeddings and generation, the five documents in knowledge.py, the document-embedding step (done on cold start), the _embed() and _cosine() helpers, and the generation call that grounds the answer in whatever context it is handed. The one gap is retrieve().

Your task

Turn a question into the best-matching documents. retrieve(query, k) has three moves: embed the query with _embed(), score that vector against every (doc, vector) pair in _DOC_VECTORS with _cosine(), and return the k highest-scoring document dicts, best first. A sort keyed on the score, descending, and a slice is all the ranking machinery it takes; four lines cover it.

That is the whole of retrieval: embed the query, score it against every document with cosine similarityCosine similarityA measure of how closely two vectors point the same way, used as the default score for “how related is this text?”., take the top few. A vector store takes over the last two steps, ranking against an approximate-nearest-neighbourANNIndex structures (HNSW graphs, IVF partitions) that answer the k-nearest-neighbours question fast by giving up guaranteed exactness – recall becomes a tunable knob rather than a certainty. index instead of a Python list, so it holds up at millions of documents instead of five. The embedding stays your call either way: QueryVectors on S3 Vectors takes a query vector you have already computed.

Deploy and prove it

cd lab-05-rag-from-scratch
./scripts/deploy.sh
./scripts/test.sh
./scripts/teardown.sh

“When will my box arrive?” comes back with Thursdays and Fridays and names the delivery-days document as its source. “What is Greenbox’s carbon footprint per box?” comes back with “I do not know”, because no document supports an answer and the system instruction tells the model to answer only from the context it is handed.

When you want the reference answer, deploy it with SRC=solution ./scripts/deploy.sh, or unfold it here:

def retrieve(query, k=2):
    qv = _embed(query)
    scored = [(_cosine(qv, vec), doc) for doc, vec in _DOC_VECTORS]
    scored.sort(key=lambda pair: pair[0], reverse=True)
    return [doc for _, doc in scored[:k]]

The ideas worth keeping

• Retrieval is embed, compare, rank. Every vector store, OpenSearch, pgvector, S3 Vectors, runs the compare and the rank for you; the differences are speed, scale, and filtering, not the fundamental step. Building it by hand is why the vector-store choices click into place.
• Embedding is a separate model from generation, with its own model id, its own price and its own Region list. Titan Text Embeddings V2 (amazon.titan-embed-text-v2:0) returns 1,024 dimensions by default, and 512 or 256 on request. Pick the wrong embedding model or the wrong distance metric and retrieval degrades before generation runs at all.
• Grounding is two halves. Retrieve the right context, and instruct the model to answer only from it and to say when that context does not cover the question. Skip the second half and good retrieval still produces an answer drawn from training data; skip the first and there is nothing to ground on.
• Naming the sources is one more field once you have retrieved, and it is what makes an answer auditable, the foundation of a citations-required assistant.

What’s worth remembering

1. Retrieval is embed, compare, rank. A vector store runs the compare and rank faster at scale; the query embedding stays your own model call.
2. Embedding differs from generation. Each has its own id, price and Region list; a wrong embedding model or metric degrades retrieval before generation runs.
3. Grounding has two halves. Retrieve the right context, and instruct the model to answer only from it, saying when it falls short.
4. Instruction makes “I do not know”. Drop the grounding instruction and the model answers from training data when the context lacks the answer.
5. Source ids make answers auditable. Returning them alongside the answer is one extra field, and the basis of a citations-required assistant.
6. A Knowledge Base is this loop. A managed one wraps sync, chunkingChunkingSplitting documents into retrievable pieces before embedding them – small enough to match precisely, big enough to still make sense. and an index around the same three steps.