A forecast built on questions per day settles nothing until somebody says how many tokens a question runs to. At AWS’s published ratio of 4.7 characters a token for English, a two-page brief and a one-line query are not the same request, and the choice between a better instruction, the document at question time, and training on company material does not resolve until the unit is on the table.
A contract summariser handles the short agreements well. On a forty-page lease it returns a summary that reads perfectly and leaves out the indemnity clause, because the application trimmed the input to fit the window before the model saw it. No error appears anywhere, and the reviewer signs.
The same two terms decide which pricing shape suits. Steady, high-volume traffic can make an hourly commitment worth signing; a few hundred internal users asking occasional questions will not. Neither call can be made without a token estimate per request.
Flash card
The unit a foundation model is metered in, and the cap on what one exchange holds.
- A token is a run of characters read as one unit of meaning: a whole word, a word part such as -ed, or a punctuation mark. AWS publishes the ratio in characters, not words, and it varies by model: 4.7 characters a token for English, nearer six on Amazon Nova. A page holds more tokens than words.
- On-demand inference meters input and output separately, output usually the dearer. Batch inference is a separate per-token rate for work that can wait. Provisioned Throughput is the third shape on Amazon Bedrock: an hourly charge per model unit, cheaper on a longer commitment, that runs whether or not a request arrives.
- A model carries nothing between calls. The standing instruction, the conversation so far and every retrieved passage go back in, billed again on every request, so cost follows the traffic, not the size of the library behind it. Prompt caching lowers the rate on the repeated part, though cache writes can cost more than standard input.
- The context window is set by the model, not the account: more room means a different model, or a larger-context variant of the same one. It covers prompt and reply together, so a long input leaves less room for the answer.
- Too much text rarely reaches the business as an error. A request over the window is rejected or flagged as exceeding it; a reply that hits the output limit comes back marked as cut short. An application that trims a document to fit returns a complete-looking answer with no sign of the loss.
Pick it when
Reach for these when a forecast has no unit underneath it, or a feature works on small inputs and degrades on the largest. Ask what happens to a document that does not fit: rejected, trimmed or split, and who decided.
It's the wrong answer when
Wrong explanation for a problem that does not track length: an assistant quoting a policy replaced last April has a source problem. A larger window is not a cost control: it raises the ceiling without lowering any rate.