Picking a Vector Store for Bedrock RAG

The situation

The retrieval assistant from earlier in the year outgrew its starter index. What began as 20,000 chunks across three sources is now 12 million vectors spanning product documentation, customer knowledge base articles, internal runbooks, historical support tickets, and a growing archive of community forum posts. Every document carries metadata, source, product line, language, published date, access level, and the queries that matter mix semantic similarity with hard filters: “articles about billing, in English, not marked internal, embedded in the last 90 days.”

The retrieval service has a 50ms budget at p99. The Bedrock generation step dominates cost at roughly AUD$0.003 per query; the vector storeVectorAn ordered list of numbers – in AI usage, almost always an embedding – and by extension the databases that index them for nearest-neighbour search. must not push that number over AUD$0.006 at peak. Peak is 200 queries per second during US business hours and 20 queries per second overnight. The index grows at roughly 500k new vectors per month, and when a document changes at source it gets re-chunked and re-embedded, the fresh vectors replacing the stale ones already in the index.

The quick-create OpenSearch Serverless collection the Knowledge Base spun up on day one has been the default. Finance is now looking at the bill.

What actually matters

A vector store does three things: stores high-dimensional vectors next to their source text and metadata, runs approximate-nearest-neighbour (ANN)ANNIndex structures (HNSW graphs, IVF partitions) that answer the k-nearest-neighbours question fast by giving up guaranteed exactness – recall becomes a tunable knob rather than a certainty. search against them quickly, and supports metadata filters alongside the vector search. That’s the job. The differentiation is in how well each of those three is done, and what it costs.

The first decision worth naming is the index algorithm. HNSW (hierarchical navigable small world) is the de-facto standard for ANN, fast, accurate, memory-hungry. IVF (inverted file) is an alternative, slower to query but cheaper at scale. Most managed stores build HNSW graphs; some expose the parameters (M, ef_construction, ef_search) so recall can be traded against speed and memory.

The second is query shape, and it has a wrinkle specific to Bedrock. Pure vector ANN is the baseline. Hybrid search, combining keyword scoring and vector scores, handles queries where the exact match matters (product codes, version numbers, error strings). Metadata filters are the other axis; they can be applied during the vector search (pre-filter, more accurate) or after it (post-filter, which can return fewer results than asked for when the filter is restrictive). The wrinkle: Bedrock Knowledge Bases only run hybrid search against Amazon RDS, OpenSearch Serverless and MongoDB Atlas stores that carry a filterable text field. Any other store falls back to semantic search on the Knowledge Base path, whatever the store itself can do when an application queries it directly.

The third is pricing shape. Dedicated vector services usually price by compute units. Adding vectors to a relational database prices by instance hours plus storage, predictable, scaling with the database. Pure-managed third-party stores price per-operation, reads, writes, and storage metered. The curves cross at different corpus sizes; the cheapest option at 100k vectors is often not the cheapest at 10M.

The fourth is operational shape. Is the store a managed service that we point at, or does it call for capacity planning, index tuning, reindexing procedures? The answer isn’t binary, some managed offerings still have compute-unit ceilings to think about; databases are managed but vector-index builds need planning.

The last one isn’t technical at all: what else we’re already running. An organisation with Aurora everywhere has ops maturity on Postgres that tips the scales toward pgvector; an organisation with OpenSearch for logs already knows the query language.

What we’ll filter on

1. Query latency at scale, p99 under 50ms at 12M vectors?
2. Hybrid search, keyword and vector scoring in one query, and does it survive the Knowledge Base route?
3. Metadata filtering, and specifically range filters on a date?
4. Cost shape, how the bill grows with corpus size and query volume?
5. Operational surface, capacity planning, index rebuilds, tuning?

The landscape

OpenSearch Serverless (vector collection). Purpose-built vector collection type, running the k-NN plugin’s HNSW implementation on the Faiss engine. Vector collections don’t support the Lucene ANN engine, IVF or IVFQ. There are two generations. NextGen collections are the current default: 32x index compression, GPU-accelerated index builds, no minimum capacity, and indexing and search that scale to zero after ten minutes of inactivity. Classic collections are billed for a minimum of 2 OCUs for the first collection in an account, one indexing and one search. Compute runs USD$0.24 per OCU-hour and managed storage USD$0.02 per GB-month, so that Classic floor is roughly USD$350 a month before a single document lands. Hybrid search works through a search pipeline with a normalization-processor, which Serverless allows (PUT _search/pipeline/<id> is a supported operation). Pre-filtering with the filter parameter inside the knn query is supported, and dimensions go to 16,000. One trap for Knowledge Base users: metadata filtering needs the index on the faiss engine, and an index created with nmslib has to be rebuilt.

Aurora PostgreSQL with pgvector. The relational option, and the one Bedrock reaches through the RDS Data API with credentials in Secrets Manager. pgvector stores vector(n) columns, builds HNSW or IVFFlat indexes, and supports operators (<-> L2, <#> negative inner product, <=> cosine). Bedrock Knowledge Bases require pgvector 0.5.0 or higher. Hybrid search comes from a GIN index over to_tsvector, and AWS suggests the english dictionary rather than simple for English content. Metadata filters are WHERE clauses, but the filter is applied after the HNSW index scan, so a selective filter returns fewer rows than asked for. The fix is HNSW iterative scans, which need pgvector 0.8.0 or later and two database settings (hnsw.iterative_scan and hnsw.max_scan_tuples). Sizing is the other sharp edge: 12M vectors at 1024 dimensions in float32 is about 49 GB of raw vector data, so a db.r7g.xlarge at 32 GiB cannot hold the index resident and a db.r7g.4xlarge at 128 GiB is the honest starting point. Index builds on 12M rows take hours and want maintenance_work_mem in the gigabytes.

Pinecone Serverless. Managed vector database, usage-metered, and one of the third-party stores Bedrock Knowledge Bases can attach through a Secrets Manager credential. Storage is separated from compute, and reads, writes and storage are metered separately. Hybrid search at Pinecone’s own API is sparse-dense: either one index holding both vectors per record, with the dense/sparse balance set client-side by scaling the query vectors before the request, or two linked indexes merged by the caller. Metadata filters are pre-filters. Through a Knowledge Base, none of the hybrid machinery applies, because hybrid is restricted to RDS, OpenSearch Serverless and MongoDB Atlas; a Pinecone-backed Knowledge Base runs semantic search.

DynamoDB vector indexes. DynamoDB indexes vectors stored on table items and searches them with SearchVectors, an ANN query returning items ranked by a similarity score under a cosine, Euclidean or dot-product distance function. Vector indexes are on-demand capacity only, at most five per table, and the table must be on-demand too. Two limits decide it here. Inline filter attributes support the equality operator only; comparison, range and set-membership operators are not available, so embedded in the last 90 days cannot be expressed. And there is no keyword scoring, so the keyword half of a hybrid query has nowhere to go.

Update, 6 August 2026. DynamoDB gained a native vector index and a SearchVectors API on 5 August 2026, after this post first went up. It replaces the DynamoDB-plus-OpenSearch plumbing this section used to describe, and its pay-per-request shape with no capacity floor makes it a real contender on cost. It still loses on the two requirements that drove the pick.

ElastiCache for Valkey. Not RediSearch on ElastiCache for Redis, which is how this option is usually described. Vector search arrived with Valkey 8.2 on node-based clusters, at no extra charge, and Valkey 9.0 added numeric, tag, full-text and aggregation search alongside it, which is the hybrid workload. Vectors go to 32,768 dimensions with FLAT or HNSW indexes over HASH and JSON keys. Everything is resident in memory, so 49 GB of vectors plus graph overhead sets the node size before any other consideration. It is also not one of the stores Bedrock Knowledge Bases can attach to, so choosing it means running retrieval in application code.

S3 Vectors. Vectors stored in S3 with a dedicated API, up to 2 billion per index, 1 to 4,096 dimensions, and up to 40 KB of metadata per vector of which 2 KB is filterable. AWS states subsecond latency for infrequent queries and as low as 100 milliseconds for frequent ones, which is an order of magnitude outside a 50ms interactive budget. It filters on metadata but does no keyword scoring; AWS’s own route to hybrid over an S3 vector index is exporting a snapshot into OpenSearch Serverless. It belongs on this list as the cheap archival tier, not as the assistant’s index.

Evaluation

Side by side

Option	Fits the 50ms budget	Hybrid search	Date-range filters	Cost shape	Ops surface
OpenSearch Serverless	✓	✓ (search pipeline)	✓ pre-filter	OCU-hours, no floor on NextGen	Managed, set an OCU ceiling
Aurora + pgvector	✓	✓ (GIN + vector)	✓ WHERE	Instance-hours	Index builds, vacuum
Pinecone Serverless	✗	✗ via Knowledge Bases	✓ pre-filter	Per-op, scales with traffic	Minimal
DynamoDB vector index	✓	✗	✗ equality only	Per-request, no floor	Minimal
ElastiCache for Valkey	✓	✓ (Valkey 9.0)	✓ numeric	Node-hours, memory-bound	Cluster management
S3 Vectors	✗	✗	✓ filterable metadata	Per-request + storage	Managed, archival fit

Reading it for this situation, 12M vectors, 50ms budget, hybrid queries, a date range in every filter, 200 qps peak, and retrieval driven through a Knowledge Base, two options clear every column: OpenSearch Serverless and Aurora with pgvector. ElastiCache for Valkey clears the query columns but not the Knowledge Base one. The choice between the first two is about what else we’re running.

How the three finalists compare

The solution

OpenSearch Serverless, when retrieval quality is what the team is protecting and the corpus is large enough to keep the collection busy. Hybrid search is one query through the search pipeline; HNSW parameters (m, ef_construction, ef_search) are set in the k-NN mapping. Pre-filtering works through the filter parameter in the knn query, applied during graph traversal rather than afterwards, which holds recall up when filters are selective. Two operational details decide the bill. On a NextGen collection nothing is charged while the collection is idle, and a maximum OCU setting caps the worst month; on a Classic collection the first collection in the account carries a 2-OCU floor, about USD$350 a month at USD$0.24 per OCU-hour. The other detail is freshness: a NextGen index becomes searchable about ten seconds after a write, a Classic one after sixty, which matters when a re-embedded document has to replace its stale chunks.

Aurora PostgreSQL with pgvector, when the team already runs Aurora, the metadata lives in relational tables, and queries can lean on SQL. A document’s row has id, content, metadata jsonb, and embedding vector(1024); the query SELECT ... WHERE metadata->>'source' = 'pricing' ORDER BY embedding <=> $1 LIMIT 10 combines filtering and vector search in one plan. Turn on HNSW iterative scans, which need pgvector 0.8.0 or later, or the post-HNSW filter returns short result sets whenever the filter bites. Size for residency: 49 GB of vectors at 12M rows puts the floor at a db.r7g.4xlarge, not the xlarge the starter index ran on. Building the HNSW index on 12M rows takes hours with maintenance_work_mem in the gigabytes, so schedule rebuilds in a quiet window.

Pinecone Serverless, when the team would rather not run a vector store at all. Upload vectors through the SDK, query them, leave the rest alone. Operational surface is close to zero; the trades are a separate vendor relationship, a separate bill, a round trip that overruns the 50ms budget, and hybrid search that only works when the application queries Pinecone directly rather than going through a Knowledge Base.

Worked example

User asks: “Why does my billing show a prorated charge on the 15th?”

The front-end embeds the query with Titan Text Embeddings V2, which here emits 1024 floats (512 and 256 are also available). It also generates a sparse keyword representation for hybrid stores. The filter is source in ('pricing', 'billing-kb', 'manual'), language equals en, and epoch_modification_time greater than the epoch second for 1 April 2026.

OpenSearch Serverless. One POST to the collection’s _search endpoint with a hybrid pipeline: { "query": { "hybrid": { "queries": [ { "match": { "content": "prorated charge 15th" } }, { "knn": { "embedding": { "vector": [...], "k": 50, "filter": { "bool": { "must": [...metadata...] } } } } } ] } } }. Response in 18ms. Top 5 chunks. Score normalisation handled by the pipeline.

Aurora + pgvector. One SQL query: WITH semantic AS (SELECT id, content, embedding <=> $1 AS dist FROM chunks WHERE metadata @> $2 ORDER BY dist LIMIT 50), keyword AS (SELECT id, ts_rank(ts, plainto_tsquery('english', 'prorated charge 15th')) AS r FROM chunks WHERE metadata @> $2 ORDER BY r DESC LIMIT 50) SELECT ... FROM semantic FULL JOIN keyword USING (id) ORDER BY (0.6 * COALESCE(semantic.dist, 1) - 0.4 * COALESCE(keyword.r, 0)) LIMIT 5;. Response in 45ms. Explicit weighting; auditable plan.

Pinecone Serverless, queried directly rather than through a Knowledge Base. One query call with the dense vector, the sparse vector, the metadata filter and top_k: 5, with the dense and sparse vectors scaled before the call to set the balance between them. Response in 80ms, most of it transport.

All three return a comparable top-5. What separates them is the 20 to 50ms left over in the latency budget, and the twenty minutes of SQL nobody has to write.

What’s worth remembering

1. The vector store bounds retrieval quality; it doesn’t create it. A better store won’t rescue a bad chunking strategy, and a worse one will cap a good retriever.
2. Hybrid search catches the queries vector search misses: product codes, version strings, error messages, proper nouns. Through a Knowledge Base it runs only on RDS, OpenSearch Serverless and MongoDB Atlas.
3. Pre-filtering beats post-filtering for selective metadata. pgvector filters after the HNSW scan unless iterative scans are on, so a filter that removes 90% of the corpus returns a short list.
4. OpenSearch Serverless is the default for AWS-native hybrid retrieval at scale. NextGen collections scale to zero; Classic collections carry a 2-OCU floor for the first collection in an account.
5. The cost curves cross. What’s cheapest at 100k is rarely cheapest at 10M, so model the bill against realistic growth rather than today’s usage.