Exam-style
A food business prints an allergen declaration on every box it despatches. Which warnings appear is set by labelling regulations applied to the ingredient list of the items packed that week. A team proposes training a classifier to read the ingredient list and predict the warnings, and reports 99.4% accuracy against four years of past labels as evidence it is ready to print. What is wrong with the approach?
Reveal the answer
D. The warnings are fixed by regulation applied to a known ingredient list, so this is a calculation for deterministic code and no accuracy figure makes a model appropriate
A trained model returns its best estimate, and is wrong on individual cases with nothing marking which ones. Here a named ingredient list has one lawful set of warnings, today and at an audit two years from now: a specific outcome is needed instead of a prediction. 0.6% of labels wrong is 0.6% of boxes mislabelled, and a mislabelled allergen is a recall and a regulatory breach rather than an acceptable error rate. More training data moves the figure and never reaches certainty. A higher confidence threshold sorts cases by the score attached to the output, not by whether that output is right. A foundation model still returns an estimate. Human review is a mitigation for a model you had a reason to build. A reviewer who applies the regulation to check the prediction has done the whole job by hand. The regulation is already written as criteria, so coding it is a fortnight of work, returns the same answer every time, and can be shown to an auditor. The cost-benefit analysis lands there too: no labelling exercise, no standing review process.
Q. A classifier predicts which allergen warnings to print, at 99.4% accuracy on four years of labels. Ship it?
A. No. The declaration is regulation applied to a known ingredient list, so it is a calculation, not a prediction, and it belongs in ordinary code.
Why? Accuracy describes a population of past cases and says nothing about the next one. At 0.6% wrong, the model mislabels boxes, and nothing in the output marks which ones. Adding a reviewer means applying the regulation by hand to check the model, which is the job you were trying to skip.