Cheat Sheet: RAG and Vector Stores

A dense pass over the RAG pipeline and the AWS vector stores behind it. Skim it, drill the traps, move on.

The pipeline at a glance

Decision rules

• If passages must keep surrounding context but you want tight embeddings, use hierarchical parent-document chunking.
• If documents are long and topically mixed, prefer semantic chunking over fixed-size.
• If records are short and self-contained (product rows, FAQ entries), skip chunking entirely.
• If you pick an embedding model, match the distance metric it was trained for: cosine, dot product, or L2. Mismatching the metric silently wrecks recallRecall (retrieval)The share of genuinely relevant passages a search actually returns – what you lose when you retrieve fewer chunks. .
• If storage cost matters more than top-end recall, drop Titan v2 to 512 or 256 dimensions.
• If you already run OpenSearch or PostgreSQL, reuse it (pgvector on Aurora or RDS, k-NN on OpenSearch) before adding a new store.
• If you need the lowest query latency at large scale, use OpenSearch with HNSWHNSWA graph-based vector index that walks neighbour links to find close vectors fast, at the cost of extra memory per vector. .
• If the corpus is huge and cold and latency is relaxed, use S3 Vectors to cut cost.
• If relationships between entities drive the answer, use Neptune Analytics for GraphRAG.
• If queries mix exact keywords (codes, names) with meaning, use hybrid retrieval.
• If the top result is right but buried, add a reranking step over a wider candidate set.
• If tenants share an index, enforce isolation with metadata filtering keyed to the caller’s identity.
• If you want the managed pipeline (ingest, chunk, embed, retrieve, generate), use Bedrock Knowledge Bases with RetrieveAndGenerate.
• If you need native document ACLs, use a Bedrock managed knowledge base and pass userContext on every retrieval call; it ingests source permissions and filters per user, with Web Crawler the one connector it does not cover.
• If you need broad enterprise connectors, budget for export pipelines into S3; Kendra’s catalogue is closed to new customers and the managed knowledge base covers seven sources.
• If you want a ready assistant over enterprise sources, use Amazon Quick.
• If the answer depends on volatile live facts (price, stock, balance), call a tool or API instead of retrieving stale documents.

Traps

• DynamoDB is not a vector store. It has no native similarity search; do not pick it for k-NNNearest-neighbour searchFinding the vectors closest to a query vector; at scale it’s approximated, trading a little accuracy for a lot of speed. retrieval.
• Using a different embedding model for queries than for documents breaks retrieval even if both are “embeddings”.
• A distance metric that does not match the model (L2 where cosine was expected) degrades results without any error.
• Raising top-kTop-kHow many chunks a retrieval step returns per query – the dial that trades answer coverage against token cost. indefinitely hurts: lost-in-the-middle means models weight the start and end of context and skim the middle.
• HNSW gives high recall and low latency but eats memory; IVF is lighter on memory but needs tuning and can lose recall.
• Post-filtering after retrieval can return fewer than k results; pre-filtering keeps the candidate pool full but must be indexed.
• Reranking improves precision but adds a model call of latency; do not add it if the first-stage results are already ordered well.
• Semantic-only retrieval misses exact identifiers; that is what sparse or hybrid is for.
• Bedrock Knowledge Bases, Kendra, and Amazon Quick are different layers: pipeline, retriever, and assistant. Do not treat them as interchangeable.
• Kendra has been in maintenance mode since 30 June 2026 and closed to new customers since 30 July 2026. Existing indexes keep running and keep getting security fixes, but a new build cannot reach for it.
• userContext on Retrieve is optional, so a call that omits it returns unfiltered results and a plausible answer. Document-level access control fails open, not closed.
• Stale answers usually mean the sync is not incremental; schedule ingestion, do not rebuild the whole index by hand.
• RAG over volatile numbers gives confidently wrong figures; route those to a live tool call.
• Metrics and aggregates (“how many orders last month”) are a text-to-SQL job against the database, not a vector search.

Say it in one line

1. RAG retrieves relevant text and puts it in the prompt so the model answers from grounded context.
2. Chunking strategy is a recall lever: fixed, semantic, hierarchical, or none, chosen by document shape.
3. Titan Text Embeddings v2 supports configurable dimensions (256, 512, 1024) to trade recall for cost.
4. The same embedding model embeds documents and queries, and the index metric must match that model.
5. OpenSearch (managed or Serverless) gives k-NN with HNSW or IVF index choices.
6. pgvector on Aurora or RDS PostgreSQL adds vector search to a relational store you may already run.
7. Neptune Analytics powers GraphRAG when relationships between entities drive the answer.
8. S3 Vectors is the cold, cost-optimised store for large corpora with relaxed latency.
9. Hybrid retrieval fuses dense semantic and sparse lexical matching and is the safe default.
10. Reranking reorders a wide candidate set with a cross-encoderCross-encoderA model that reads a query and a passage together and scores the pair, more accurate than comparing two independently-made vectors. to lift precision at the cost of latency.
11. Metadata filtering, scoped by identity, is how one index serves many tenants safely.
12. Bedrock Knowledge Bases is the managed pipeline with RetrieveAndGenerate and Amazon Quick is the managed assistant; Kendra was the managed retriever with connectors and ACLs, and is now in maintenance mode and closed to new customers.
13. Keep answers fresh with incremental sync, not full rebuilds; route volatile facts to a tool call.
14. Use text-to-SQL for counts and metrics; do not expect vector search to aggregate.
15. A managed knowledge base enforces document-level ACLs itself, ingesting source permissions and filtering on the userContext you pass, and re-checks against the source at query time.