Exam-style
A food business prints an allergen declaration on every box it despatches. Which warnings appear is set by labelling regulations applied to the ingredient list of the items packed that week. A team proposes training a classifier to read the ingredient list and predict the warnings, and reports 99.4% accuracy against four years of past labels as evidence it is ready to print. What is wrong with the approach?
Reveal the answer
D. The warnings are fixed by regulation applied to a known ingredient list, so this is a calculation for deterministic code and no accuracy figure makes a model appropriate
A trained model returns its best estimate and is wrong on individual cases with nothing attached to say which ones. Here a named ingredient list has one lawful set of warnings, today and at an audit two years from now, which is a situation where a specific outcome is needed instead of a prediction. Read the accuracy figure as what it describes: 0.6% of labels wrong is 0.6% of boxes mislabelled, and a mislabelled allergen is a recall and a regulatory breach rather than an acceptable error rate. Nothing on the other four options changes that shape. More training data moves the figure and never reaches certainty. A higher confidence threshold sorts cases by how sure the model is, not by whether it is right. A foundation model is still producing an estimate. Human review is a mitigation for a model you had a reason to build, and a reviewer who has to apply the regulation to the ingredient list in order to know whether the prediction was right has just done the whole job by hand. The regulation is already written down as criteria, so writing it as code is a fortnight of work that gives the same answer every time and can be shown to an auditor. The cost-benefit analyses land the same way: the rule costs almost nothing to run and the model costs a build, a labelling exercise and a permanent review process to catch mistakes that the rule does not make.
Q. A classifier predicts which allergen warnings to print, at 99.4% accuracy on four years of labels. Ship it?
A. No. The declaration is regulation applied to a known ingredient list, so it is a calculation, not a prediction, and it belongs in ordinary code.
Why? Accuracy describes a population of past cases and promises nothing about the next one. At 0.6% wrong, the model mislabels boxes and gives nobody a way to tell which ones. Adding a reviewer means applying the regulation by hand to check the model, which is the job you were trying to skip.