Exam-style
An extraction job pulls fixed fields out of invoices into a JSON schema. Run the same invoice twice and the field wording differs, and occasionally the model appends a sentence of commentary after the JSON. Which inference settings should change, and in which direction?
Reveal the answer
C. Lower temperature toward 0, leave top-p alone, and set max tokens and a stop sequence to bound the response
Temperature shapes the probability distribution the next token is drawn from, and a value near 0 steepens it so higher-probability tokens are selected. That is what a fixed-schema extraction needs. Bedrock documents adjusting either temperature or top-p, not both at once, and Claude Sonnet 4.5 and Haiku 4.5 accept only one of the two, so moving both is the wrong habit even on older models that allow it. Max tokens bounds the response length, and a stop sequence is the documented way to halt generation at the close of the JSON rather than after trailing commentary. Raising temperature, top-p or top-k all widen the candidate pool and produce more of the variation being reported. Cutting max tokens to 50 truncates the JSON mid-object, and a larger model at default settings still samples.
Q. Outputs are too random for a structured extraction task. Which inference parameters, and which way?
A. Lower temperature toward 0 so higher-probability tokens are selected. Adjust temperature or top-p, not both: Bedrock documents changing one of the two, and Claude Sonnet 4.5 and Haiku 4.5 accept only one. Set max tokens to bound the response length, and a stop sequence to end generation at the close of the JSON.
Why? Both parameters control how wide the pool is that the next token is drawn from. Extraction calls for a narrow pool, brainstorming for a wide one. For schema conformance itself, structured outputs constrains the response to a supplied JSON schema, which parameter tuning only approximates.