Picking an Embedding Model for Retrieval

The situation

A global content platform runs a retrieval layer over its customer knowledge base. The corpus is 20 million chunks across four locales: English 60%, Spanish 20%, Portuguese 10%, Japanese 10%. Chunks average about 300 tokens. Queries arrive in the reader’s locale, and the retriever has to reach content in any locale. A Spanish-speaking customer asking about a product feature should get the English documentation when the Spanish page is missing or stale.

The index was built with Titan Embeddings G1 - Text (amazon.titan-embed-text-v1) when the platform launched 18 months ago. Retrieval is acceptable in English, mediocre in Spanish and Portuguese, visibly bad in Japanese.

The bill is about USD$1,600 a month, and the OpenSearch Serverless collection is nearly all of it. Serverless bills compute at USD$0.24 per OCU-hour and managed storage at USD$0.02 per GB-month, so a handful of indexing and search OCUs running continuously dominates everything else. The embedding calls come to roughly USD$25 a month: about 1.5 million chunks are added or edited monthly, and the 20 million queries average around 30 tokens each. Product wants better non-English retrieval without doubling the retrieval bill, and without another re-index next year.

What actually matters

An embedding model turns text into a fixed-length numeric vectorVectorAn ordered list of numbers – in AI usage, almost always an embedding – and by extension the databases that index them for nearest-neighbour search.. Two pieces of text with similar meaning should produce vectors that sit close together under cosine or Euclidean distance, and two with different meanings should sit far apart. Models differ in how well they do that on different text, and in what it costs to run them.

Language coverage is the first thing to settle, and it is the one most often read off a marketing page rather than the documentation. AWS describes Titan Text Embeddings V2 as optimised for English, with support for a long list of other languages still in preview, and states directly that cross-language queries return sub-optimal results. Cohere’s Embed Multilingual model card describes 100-plus languages for cross-lingual search. Those are different claims about different capabilities, and this corpus needs the second one.

Chunk size is the second, and it is a hard limit rather than a quality gradient. Cohere Embed v3 accepts 512 tokensTokenThe unit of text an LLM actually sees – usually a short character sequence, not a whole word. per text and truncates or errors beyond that, depending on the truncate setting. Titan Text Embeddings V2 accepts 8,192 tokens or 50,000 characters. Cohere Embed v4 accepts around 128,000. A chunker tuned for one ceiling has to be retuned for another.

Dimension size sets index cost. More dimensions allow finer distinctions and take more room. Titan V2 emits 1,024 floats by default and accepts 512 or 256 through the dimensions parameter. Embed v4 takes an output_dimension of 256, 512, 1,024 or 1,536, defaulting to 1,536, and can return int8 or binary vectors instead of floats. OpenSearch Serverless has its own dial here: NextGen vector collections apply 32x compression by default and accept compression_level values of 1x, 2x, 8x, 16x and 32x, so stored size is not a straight function of dimension count.

How the backfill runs matters more than the per-token rate. Embedding models on Bedrock are throttled by requests per minute rather than tokens per minute, so a 20-million-chunk job is governed by an RPM quota. Titan Text Embeddings V2 supports Bedrock batch inference in a dozen Regions. The Cohere Embed models do not appear in the batch inference support table at all, which puts their backfill on InvokeModel against that RPM quota.

Switching models means re-embedding every chunk, so the choice is sticky. That argues for reading the model card before committing: each one carries an “EOL no sooner than” date and a Legacy period, normally at least six months, during which a replacement has to land.

What we’ll filter on

1. Whether the documentation states cross-lingual retrieval, rather than only listing languages.
2. Maximum input tokens per chunk.
3. Output dimensions available, and whether they are configurable.
4. Published price per million input tokens.
5. Whether the 20-million-chunk backfill can run as a Bedrock batch inference job.

The landscape

1. Amazon Titan Text Embeddings V2 (amazon.titan-embed-text-v2:0). Active, 8K context, 1,024 dimensions by default with 512 and 256 available, USD$0.02 per million input tokens, on-demand and Provisioned Throughput, batch inference supported. Optimised for English; the wider language list is in preview, and the documentation states that cross-language queries return sub-optimal results.

2. Titan Embeddings G1 - Text (amazon.titan-embed-text-v1). The incumbent here. Still Active on Bedrock rather than withdrawn, but superseded by V2: 1,536 fixed dimensions, 8K context, more than 25 languages. Not a candidate for a rebuild.

3. Cohere Embed Multilingual v3 (cohere.embed-multilingual-v3). Active, 100-plus languages for cross-lingual search, 1,024 dimensions, 512-token context, USD$0.10 per million input tokens, billed through AWS Marketplace. No batch inference and no streaming.

4. Cohere Embed English v3 (cohere.embed-english-v3). Same family and same rate, English only. Right for a monolingual corpus, wrong for this one.

5. Cohere Embed v4 (cohere.embed-v4:0). Active since April 2025, roughly 128K context, output_dimension of 256 to 1,536, float / int8 / uint8 / binary / ubinary embedding types, text and image input, USD$0.12 per million input tokens. It carries Geo and Global cross-Region inference IDs (us.cohere.embed-v4:0, eu.cohere.embed-v4:0, global.cohere.embed-v4:0), which neither Titan embedding model does. Also no batch inference.

6. A self-hosted multilingual model on SageMaker AI. An open-weights encoder on a real-time endpoint, billed by instance-hours rather than tokens. Worth pricing from the SageMaker AI page for the instance in question; the crossover against per-token billing sits wherever that instance rate meets your monthly token volume. Endpoint operations, patching and scaling come with it.

7. Locale-specific models, one per language. Maximum quality per locale, and it breaks the requirement: vectors from different models are not comparable, so a Spanish query cannot reach a Japanese chunk.

Evaluation

Side by side

Model	Documented cross-lingual retrieval	Max input tokens	Output dimensions	Per 1M input tokens	Bedrock batch
Titan Text Embeddings V2	✗ (English-optimised)	8,192	1024 / 512 / 256	USD$0.02	✓
Cohere Embed Multilingual v3	✓	512	1024	USD$0.10	✗
Cohere Embed English v3	✗ (English only)	512	1024	USD$0.10	✗
Cohere Embed v4	✓	~128,000	1536 / 1024 / 512 / 256	USD$0.12	✗
Self-hosted on SageMaker AI	Model-dependent	Model-dependent	Model-dependent	Instance-hours	N/A

Run the token arithmetic before letting the rate column decide anything. The backfill is 20 million chunks at about 300 tokens, near enough 6 billion tokens: USD$120 on Titan V2, USD$600 on Embed Multilingual v3, USD$720 on Embed v4, one time. Steady state is 450 million ingest tokens and 600 million query tokens a month, so about USD$21 a month on Titan V2 against USD$105 on Embed Multilingual v3 and USD$126 on Embed v4. The spread is under USD$1,300 a year, against a retrieval bill near USD$19,000 a year. A five-fold difference on a rounding error is still a rounding error.

Choosing between them

The solution

Cohere Embed Multilingual v3 is the model to rebuild this index on, and Embed v4 is the one to rebuild it on if the chunker is going to change anyway. Both are documented for cross-lingual search over 100-plus languages, which is the requirement the current index fails. Titan Text Embeddings V2 is the cheaper and more AWS-native option, and on AWS’s own description it is the wrong shape here: English-optimised, wider language support in preview, cross-language queries called out as sub-optimal. Paying USD$0.02 per million tokens for retrieval that does not span locales solves nothing.

Between the two Cohere models, the chunker decides. Embed v3 caps a text at 512 tokens, and 300-token chunks fit with room to spare, so an index rebuilt on v3 keeps the existing pipeline. Embed v4 accepts roughly 128,000 tokens, returns 256 to 1,536 dimensions, and can return int8 or binary vectors instead of floats, which cuts index size independently of dimension count. It also has Geo and Global cross-Region inference IDs, so a single application can route across a geography. Pick v4 when larger chunks, compact vectors or image input are on the roadmap; pick v3 when nothing else is changing.

One model covers the whole corpus. Splitting by locale, Cohere for Japanese and Titan for the rest, sounds like a saving and does not work: vectors from different models are not comparable, so a Spanish query cannot reach a Japanese chunk. Two indices and a merged result list add plumbing without restoring cross-locale reach.

Dimension and storage. At 1,024 dimensions, 20 million float32 vectors come to roughly 82 GB before compression. OpenSearch Serverless NextGen collections compress at 32x by default and take a compression_level of 1x through 32x, so reach for that dial before dropping dimensions, and measure recall at each level against a held-out set. NextGen collections also scale indexing and search to zero when idle, which matters more to the monthly bill than vector width does.

Running the backfill. This is where the Cohere choice costs something real. Titan Text Embeddings V2 supports Bedrock batch inference; the Cohere Embed models do not appear in the batch inference support table, so 20 million chunks go through InvokeModel. Embedding models on Bedrock are throttled on requests per minute rather than tokens per minute, so size the job against the RPM quota for the model and Region, batch up to 96 texts per Embed call, and request an increase before starting rather than after the first throttle. Embed v4 running under a Global inference ID spreads that load further than an in-Region call does.

Lifecycle. Check the model card rather than assuming an AWS-badged model outlives a partner one. Every card carries an “EOL no sooner than” date and a Legacy period, normally at least six months and sometimes 45 days, and the EOL date appears on the card once the Legacy period starts. Titan Embeddings G1 - Text and Cohere Embed Multilingual are both Active today, nearly three years after launch. Put the card on a review schedule instead of relying on which company built the model.

Worked example

Week 1. Stand up a second OpenSearch Serverless vector collection, kb-v2, as a NextGen collection so compression and scale-to-zero apply by default. Request the Embed Multilingual RPM increase. Confirm the chunker caps every text at 512 tokens and set truncate to END so an over-length chunk loses its tail instead of failing the call.

Week 2. Run the backfill through Step Functions Map over chunks in S3, 96 texts per InvokeModel call, input_type of search_document. Roughly 6 billion tokens, about USD$600. Throughput is bounded by the RPM quota, not the token spend.

Week 3. Dual-write new and edited content to both collections, and query both. Compare recall on a held-out evaluation set of 500 queries, broken out per locale so the Japanese result is visible rather than averaged away, and embed those queries with input_type of search_query.

Week 4. Canary 10% of traffic to kb-v2, watch click-through and reformulation rate per locale, then ramp. Keep the old collection for a week as a rollback path and delete it after.

Steady state: about USD$105 a month in embedding calls, up from roughly USD$25, on a retrieval bill near USD$1,600 a month, with cross-locale retrieval the documentation actually states.

What’s worth remembering

1. A long language list is not a claim of cross-lingual retrieval. AWS describes Titan Text Embeddings V2 as English-optimised, with 100-plus languages in preview and cross-language queries called out as sub-optimal; Cohere Embed Multilingual is documented for cross-lingual search.
2. Per-token rates rarely decide an embedding choice. At 20 million chunks the gap between USD$0.02 and USD$0.12 per million tokens is hundreds of dollars a year, against an OpenSearch Serverless bill in the thousands.
3. Context limits are hard edges. Cohere Embed v3 takes 512 tokens, Titan V2 takes 8,192, Embed v4 takes around 128,000, and the chunker has to match whichever is chosen.
4. Titan Text Embeddings V2 supports Bedrock batch inference and the Cohere Embed models do not, so a large backfill on Cohere runs against a requests-per-minute quota instead.
5. Index size has two dials. Output dimension is one; OpenSearch Serverless NextGen compression, 32x by default and adjustable from 1x, is the other.
6. One model for the whole corpus. Vectors from different models cannot be compared, so mixing them removes cross-locale retrieval entirely.