These are set per request, not per model, so two calls to the same model an hour apart can behave nothing like each other. The selection walkthrough works them through against three real features; this card is the four of them on one page. The symptom is what a scenario gives you, and the slider is what you have to name.
Two of them get confused for each other often enough to be worth separating. Temperature and top-p both narrow the same next-token distribution, and max tokens only bounds length, so an over-long answer is never a temperature problem. Input/output length is the pricing half of the same story: tokens in and tokens out are billed separately, and both share the room in the context window.
Flash card
The inference parameters: settings sent with each request that change the answer without changing the model or the prompt. Temperature, top-p, top-k and max tokens.
- Temperature scales how much randomness is allowed into the choice of the next token. Low values give focused, repeatable output, which suits extraction, classification and structured summarising. High values give variety, which suits drafting and brainstorming.
- Top-p, also called nucleus sampling, restricts the choice to the smallest set of tokens whose cumulative probability reaches p. It narrows the same distribution that temperature reshapes, so the Bedrock guidance is to adjust one or the other and not both at once. Claude Sonnet 4.5 and Claude Haiku 4.5 on Bedrock accept only one of the two in a request.
- Top-k caps the candidate list at the k most likely tokens regardless of how their probabilities are spread, which is the same narrowing counted a different way. It sits outside the Converse base inference configuration (maxTokens, stopSequences, temperature, topP), so it travels as an extra model-specific field, on the models that support it.
- Max tokens bounds the length of the answer, and so bounds latency and output cost with it. It truncates when the answer runs long rather than summarising, so ask for the length wanted in the prompt and set the cap just behind it. Bedrock also deducts the whole max tokens value from the per-minute token quota at the start of a request and returns the unused part at the end, so a generous cap brings throttling on sooner.
- Input and output length are metered separately for pricing, at different rates, and both are constrained by the model’’s context window. A cap on the answer does nothing about a long prompt.
Pick it when
Reach for temperature and top-p when the complaint is that answers vary between runs or ramble off the brief, and for max tokens when the complaint is length, latency or cost.
It's the wrong answer when
No setting guarantees identical output on every run: a low temperature narrows the variation without removing it. And none of them fixes a factual gap, which is a retrieval problem or a model problem rather than a decoding one.