Picking a Vector Store for Bedrock RAG

The situation

The retrieval assistant from earlier in the year outgrew its starter index. What began as 20,000 chunks across three sources is now 12 million vectors spanning product documentation, customer knowledge base articles, internal runbooks, historical support tickets, and a growing archive of community forum posts. Every document carries metadata, source, product line, language, published date, access level, and the queries that matter mix semantic similarity with hard filters: “articles about billing, in English, not marked internal, embedded in the last 90 days.”

The retrieval service has a 50ms budget at p99. The Bedrock generation step dominates cost at roughly AUD$0.003 per query; the vector storeVectorAn ordered list of numbers – in AI usage, almost always an embedding – and by extension the databases that index them for nearest-neighbour search. must not push that number over AUD$0.006 at peak. Peak is 200 queries per second during US business hours and 20 queries per second overnight. The index grows at roughly 500k new vectors per month, and when a document changes at source it gets re-chunked and re-embedded, the fresh vectors replacing the stale ones already in the index.

The quick-create OpenSearch Serverless collection the Knowledge Base spun up on day one has been the default. Finance is now looking at the bill.

What actually matters

A vector store does three things: stores high-dimensional vectors next to their source text and metadata, runs approximate-nearest-neighbour (ANN)ANNIndex structures (HNSW graphs, IVF partitions) that answer the k-nearest-neighbours question fast by giving up guaranteed exactness – recall becomes a tunable knob rather than a certainty. search against them quickly, and supports metadata filters alongside the vector search. That’s the job. The differentiation is in how well each of those three is done, and what it costs.

The first decision worth naming is the index algorithm. HNSW (hierarchical navigable small world) is the de-facto standard for ANN, fast, accurate, memory-hungry. IVF (inverted file) is an alternative, slower to query but cheaper at scale. Most managed stores build HNSW graphs; some expose the parameters (M, ef_construction, ef_search) so recall can be traded against speed and memory.

The second is query shape, and it has a wrinkle specific to Bedrock. Pure vector ANN is the baseline. Hybrid search, combining keyword scoring and vector scores, handles queries where the exact match matters (product codes, version numbers, error strings). Metadata filters are the other axis; they can be applied during the vector search (pre-filter, more accurate) or after it (post-filter, which can return fewer results than asked for when the filter is restrictive). The wrinkle: Bedrock Knowledge Bases only run hybrid search against Amazon RDS, OpenSearch Serverless and MongoDB Atlas stores that carry a filterable text field. Any other store falls back to semantic search on the Knowledge Base path, whatever the store itself can do when an application queries it directly.

The third is pricing shape. Dedicated vector services usually price by compute units. Adding vectors to a relational database prices by instance hours plus storage, predictable, scaling with the database. Pure-managed third-party stores price per-operation, reads, writes, and storage metered. The curves cross at different corpus sizes; the cheapest option at 100k vectors is often not the cheapest at 10M.

The fourth is operational shape. Is the store a managed service that we point at, or does it call for capacity planning, index tuning, reindexing procedures? The answer isn’t binary, some managed offerings still have compute-unit ceilings to think about; databases are managed but vector-index builds need planning.

The last one isn’t technical at all: what else we’re already running. An organisation with Aurora everywhere has ops maturity on Postgres that tips the scales toward pgvector; an organisation with OpenSearch for logs already knows the query language.

What we’ll filter on

1. Query latency at scale, p99 under 50ms at 12M vectors?
2. Hybrid search, keyword and vector scoring in one query, and does it survive the Knowledge Base route?
3. Metadata filtering, and specifically range filters on a date?
4. Cost shape, how the bill grows with corpus size and query volume?
5. Operational surface, capacity planning, index rebuilds, tuning?

The landscape

OpenSearch Serverless (vector collection). Purpose-built vector collection type, running the OpenSearch k-NN plugin’s HNSW implementation. There are two generations, and the limitations differ between them. NextGen collections are the current default in the console: 32x index compression, GPU-accelerated index builds, no minimum OCU requirement, and indexing and search that scale to zero after ten minutes of inactivity. NextGen index mappings don’t take the engine or mode parameters at all, because the service picks the configuration itself; radial search is the one thing it gives up at 32x compression. Classic collections default to the nmslib engine, support only HNSW on Faiss, and support neither the Apache Lucene ANN engine nor IVF or IVFQ. Classic is also billed for a minimum of 2 OCUs for the first collection in an account, one indexing and one search. Compute runs USD$0.24 per OCU-hour in US East (N. Virginia) and managed storage USD$0.024 per GB-month, so that Classic floor is roughly USD$350 a month before a single document lands. Hybrid search works through a search pipeline with a normalization-processor, which Serverless allows (PUT _search/pipeline/<id> is a supported operation). Pre-filtering with the filter parameter inside the knn query is supported, and dimensions go to 16,000. One trap for Knowledge Base users: metadata filtering needs the index on the faiss engine, and an index created with nmslib has to be rebuilt.

Aurora PostgreSQL with pgvector. The relational option, and the one Bedrock reaches through the RDS Data API with credentials in Secrets Manager. pgvector stores vector(n) columns, builds HNSW or IVFFlat indexes, and supports operators (<-> L2, <#> negative inner product, <=> cosine). Bedrock Knowledge Bases require pgvector 0.5.0 or higher. Hybrid search comes from a GIN index over to_tsvector, and AWS suggests the english dictionary rather than simple for English content. Metadata filters are WHERE clauses, but the filter is applied after the HNSW index scan, so a selective filter returns fewer rows than asked for. The fix is HNSW iterative scans, which need pgvector 0.8.0 or later and two database settings (hnsw.iterative_scan and hnsw.max_scan_tuples). Sizing is the other sharp edge: 12M vectors at 1024 dimensions in float32 is about 49 GB of raw vector data, so a db.r7g.xlarge at 32 GiB cannot hold the index resident and a db.r7g.4xlarge at 128 GiB is the honest starting point. Index builds on 12M rows take hours and want maintenance_work_mem in the gigabytes.

Pinecone Serverless. Managed vector database, usage-metered, and one of the third-party stores Bedrock Knowledge Bases can attach through a Secrets Manager credential. Storage is separated from compute, and reads, writes and storage are metered separately. Hybrid search at Pinecone’s own API is sparse-dense: either one index holding both vectors per record, with the dense/sparse balance set client-side by scaling the query vectors before the request, or two linked indexes merged by the caller. Metadata filters are pre-filters. Through a Knowledge Base, none of the hybrid machinery applies, because hybrid is restricted to RDS, OpenSearch Serverless and MongoDB Atlas; a Pinecone-backed Knowledge Base runs semantic search.

DynamoDB vector indexes. DynamoDB indexes vectors stored on table items and searches them with SearchVectors, an ANN query returning items ranked by a similarity score under a cosine, Euclidean or dot-product distance function. Vector indexes are on-demand capacity only, at most five per table, and the table must be on-demand too. Two limits decide it here. Inline filter attributes support the equality operator only; comparison, range and set-membership operators are not available, so embedded in the last 90 days cannot be expressed. And there is no keyword scoring, so the keyword half of a hybrid query has nowhere to go.

Update, 6 August 2026. DynamoDB gained a native vector index and a SearchVectors API on 5 August 2026, after this post first went up. It replaces the DynamoDB-plus-OpenSearch plumbing this section used to describe, and its pay-per-request shape with no capacity floor makes it a real contender on cost. It still loses on the two requirements that drove the pick.

ElastiCache for Valkey. Not RediSearch on ElastiCache for Redis, which is how this option is usually described. Vector search arrived with Valkey 8.2 on node-based clusters, at no extra charge, and Valkey 9.0 added numeric, tag, full-text and aggregation search alongside it, which is the hybrid workload. Vectors go to 32,768 dimensions with FLAT or HNSW indexes over HASH and JSON keys. Everything is resident in memory, so 49 GB of vectors plus graph overhead sets the node size before any other consideration. It is also not one of the stores Bedrock Knowledge Bases can attach to, so choosing it means running retrieval in application code.

S3 Vectors. Vectors stored in S3 with a dedicated API, up to 2 billion per index, 1 to 4,096 dimensions, and up to 40 KB of metadata per vector of which 2 KB is filterable. AWS states subsecond latency for infrequent queries and as low as 100 milliseconds for frequent ones, which is an order of magnitude outside a 50ms interactive budget. It filters on metadata but does no keyword scoring; AWS’s own route to hybrid over an S3 vector index is exporting a snapshot into OpenSearch Serverless. It belongs on this list as the cheap archival tier, not as the assistant’s index.

Evaluation

Side by side

Option	Fits the 50ms budget	Hybrid search	Date-range filters	Cost shape	Ops surface
OpenSearch Serverless	✓	✓ (search pipeline)	✓ pre-filter	OCU-hours, no floor on NextGen	Managed, set an OCU ceiling
Aurora + pgvector	✓	✓ (GIN + vector)	✓ WHERE	Instance-hours	Index builds, vacuum
Pinecone Serverless	✗	✗ via Knowledge Bases	✓ pre-filter	Per-op, scales with traffic	Minimal
DynamoDB vector index	✓	✗	✗ equality only	Per-request, no floor	Minimal
ElastiCache for Valkey	✓	✓ (Valkey 9.0)	✓ numeric	Node-hours, memory-bound	Cluster management
S3 Vectors	✗	✗	✓ filterable metadata	Per-request + storage	Managed, archival fit

Reading it for this situation, 12M vectors, 50ms budget, hybrid queries, a date range in every filter, 200 qps peak, and retrieval driven through a Knowledge Base, two options clear every column: OpenSearch Serverless and Aurora with pgvector. ElastiCache for Valkey clears the query columns but not the Knowledge Base one. The choice between the first two is about what else we’re running.

How the three finalists compare

The solution

OpenSearch Serverless, when retrieval quality is what the team is protecting and the corpus is large enough to keep the collection busy. Hybrid search is one query through the search pipeline; m and ef_construction are set in the k-NN method mapping when the index is created. Pre-filtering works through the filter parameter in the knn query, applied during graph traversal rather than afterwards, which holds recall up when filters are selective. Two operational details decide the bill. On a NextGen collection the indexing and search OCUs drop to zero after ten minutes without a request, so only managed storage bills through a quiet night, and a maximum OCU setting on the collection group caps the worst month; on a Classic collection the first collection in the account carries a 2-OCU floor, about USD$350 a month at USD$0.24 per OCU-hour. The other detail is freshness: a NextGen index becomes searchable about ten seconds after a write, a Classic one after sixty, which matters when a re-embedded document has to replace its stale chunks.

Aurora PostgreSQL with pgvector, when the team already runs Aurora, the metadata lives in relational tables, and queries can lean on SQL. A document’s row has id, content, metadata jsonb, and embedding vector(1024); the query SELECT ... WHERE metadata->>'source' = 'pricing' ORDER BY embedding <=> $1 LIMIT 10 combines filtering and vector search in one plan. Turn on HNSW iterative scans, which need pgvector 0.8.0 or later, or the post-HNSW filter returns short result sets whenever the filter bites. Size for residency: 49 GB of vectors at 12M rows puts the floor at a db.r7g.4xlarge, not the xlarge the starter index ran on. Building the HNSW index on 12M rows takes hours with maintenance_work_mem in the gigabytes, so schedule rebuilds in a quiet window.

Pinecone Serverless, when the team would rather not run a vector store at all. Upload vectors through the SDK, query them, leave the rest alone. Operational surface is close to zero; the trades are a separate vendor relationship, a separate bill, a round trip that overruns the 50ms budget, and hybrid search that only works when the application queries Pinecone directly rather than going through a Knowledge Base.

Worked example

User asks: “Why does my billing show a prorated charge on the 15th?”

The front-end embeds the query with Titan Text Embeddings V2, which here emits 1024 floats (512 and 256 are also available). It also generates a sparse keyword representation for hybrid stores. The filter is source in ('pricing', 'billing-kb', 'manual'), language equals en, and epoch_modification_time greater than the epoch second for 1 April 2026.

OpenSearch Serverless. One POST to the collection’s _search endpoint with a hybrid pipeline: { "query": { "hybrid": { "queries": [ { "match": { "content": "prorated charge 15th" } }, { "knn": { "embedding": { "vector": [...], "k": 50, "filter": { "bool": { "must": [...metadata...] } } } } } ] } } }. Response in 18ms. Top 5 chunks. Score normalisation handled by the pipeline.

Aurora + pgvector. One SQL query: WITH semantic AS (SELECT id, content, embedding <=> $1 AS dist FROM chunks WHERE metadata @> $2 ORDER BY dist LIMIT 50), keyword AS (SELECT id, ts_rank(ts, plainto_tsquery('english', 'prorated charge 15th')) AS r FROM chunks WHERE metadata @> $2 ORDER BY r DESC LIMIT 50) SELECT ... FROM semantic FULL JOIN keyword USING (id) ORDER BY (0.6 * COALESCE(semantic.dist, 1) - 0.4 * COALESCE(keyword.r, 0)) LIMIT 5;. Response in 45ms. Explicit weighting; auditable plan.

Pinecone Serverless, queried directly rather than through a Knowledge Base. One query call with the dense vector, the sparse vector, the metadata filter and top_k: 5, with the dense and sparse vectors scaled before the call to set the balance between them. Response in 80ms, most of it transport.

All three return a comparable top-5. What separates them is how much of the 50ms budget survives the retrieval: 32ms on OpenSearch Serverless, 5ms on Aurora, nothing at all on Pinecone. That, and the twenty minutes of SQL nobody has to write.

What’s worth remembering

1. A store caps retrieval quality. A better store will not rescue poor chunking, and a weak one limits a good retriever.
2. Knowledge Bases limit hybrid search. They run hybrid only on RDS, OpenSearch Serverless and MongoDB Atlas; other stores fall back to semantic search.
3. pgvector filters after the scan. Without HNSW iterative scans (pgvector 0.8.0 or later), a selective filter returns fewer rows than asked for.
4. OpenSearch Serverless suits hybrid retrieval. NextGen collections scale to zero after ten minutes idle; Classic carries a 2-OCU floor for the first collection.
5. Cost curves cross. The cheapest store at 100k vectors is often not the cheapest at 10M, so model the bill against growth.
6. S3 Vectors suits archive, not interactive. Latency is subsecond, or as low as 100 milliseconds for frequent queries, outside a 50ms budget; no keyword scoring.