Exam-style
An audit team is testing a control on an assistant that answers policy questions. Part of the test is a repeatability check: they send one fixed prompt to the same Amazon Bedrock model three times over an afternoon and record what comes back. They get three answers that are worded differently, ordered differently, and one of which cites an extra clause the other two leave out. A subject-matter expert reads all three and confirms every one of them is factually correct and properly grounded in the policy document. The auditors want to know which named limitation of generative AI they are looking at. Which is it?
Reveal the answer
D. Nondeterminism, because the same prompt can produce a different answer on different runs even when every answer is correct
Four of these are real limitations of generative AI, and the expert’s verdict rules three of them out before the argument starts. Hallucinations are fluent, confident statements that are not supported by fact or by the source the model was given; here every statement checked out against the policy document, so nothing was invented. Inaccuracy is the answer being wrong, and the wrongness has to be in the content rather than in the process that produced it. An inconvenient test is not an inaccurate answer. A knowledge-cutoff gap is the model having no training data past a certain date, so it does not know about something that happened after that point. The extra clause came out of the policy document supplied in the prompt, which the model can read regardless of when it was trained. Interpretability is about whether anyone can explain how a model arrived at an output, and while it is genuinely hard here, it describes a property of the whole system rather than the thing that changed between the three runs. What is left is nondeterminism: a model samples its next token from a distribution rather than always taking the most probable one, so repeated runs of one prompt wander down slightly different paths and arrive at differently worded answers. The variation is a property of how the model generates text, not a fault in it. Lowering the temperature narrows the spread and makes runs look much more alike, and it still does not promise two identical strings, because model versions, batching and inference infrastructure all move underneath a request. The workable audit control is a recorded output, kept as evidence of what the system actually produced on the day, plus a fixed evaluation set scored on whether each answer is correct and grounded. Both survive the wording changing; a control that asserts byte-identical responses does not.
Q. One prompt, one model, three runs, three differently worded answers, and an expert who confirms all three are correct. Which named limitation is that?
A. Nondeterminism: the same prompt can answer differently on different runs. It is how the model generates text, not an error in what it generated.
Why? The expert’s verdict does most of the sorting. Hallucinations are confident claims unsupported by fact or by the supplied source, and these answers were grounded. Inaccuracy means the content is wrong, and it was not. A knowledge-cutoff gap is the model having never seen something, which cannot explain a clause it read straight out of the prompt. Interpretability, the difficulty of explaining how an output was reached, is real here but describes the system rather than the variation between runs. That leaves sampling, and lowering the temperature narrows the variance without guaranteeing identical output. For an audit control, record the output the system produced and score a fixed evaluation set on correctness and grounding rather than expecting the same string back three times.