Exam-style
A lender's approval model scores 96 per cent on its training data and 94 per cent on held-out validation data. Sliced by customer cohort, one group of several thousand applicants sits at 71 per cent, and is 2 per cent of the training set. What has gone wrong, and what fixes it?
Reveal the answer
D. The cohort is under-represented in the training data. Collect more examples of it, or reweight the ones there are, and re-evaluate per cohort.
Training is 96, validation is 94, and a gap that small says the model generalises about as well as it fits. Overfitting shows the opposite shape, strong on rows the model saw and much worse on rows it did not, so cutting capacity here removes fit it is already getting right. Underfitting shows poor accuracy on training and validation at once, and 96 on training is not that, so more epochs change nothing. The failure shows up only in the per-cohort comparison. Two per cent of the rows carry 2 per cent of the weight in the training loss, so the majority pattern dominates the parameters and the model applies it to everybody. The 71 per cent is a data problem surfacing as inaccuracy for one set of people, and the repairs sit on the data side: gather more examples of that cohort, reweight the ones already there, and re-evaluate cohort by cohort. Dropping the records is the worst move available: it removes the evidence of the harm and leaves the harm in production. Calling it noise fails too: 23 points below the overall figure is a pattern, worth confirming on a fresh sample rather than dismissing. Record the finding in the model card next to the headline accuracy.
Q. 96 per cent on training, 94 per cent on validation, 71 per cent for a cohort that is 2 per cent of the training data. Overfitting, underfitting, or something else?
A. Something else. The cohort is under-represented, so collect more examples of it or reweight what you have, then re-measure cohort by cohort.
Why? The diagnostic is one comparison read twice. The gap between training and validation accuracy tells you variance: a wide gap is overfitting, and the repair is less capacity or regularisation. Low accuracy on both is underfitting, and the repair is more capacity, better features, or longer training. Here the gap is two points and both figures high, so the fit is right and neither repair applies. The same comparison per cohort exposes the third cause. A group holding 2 per cent of the rows carries 2 per cent of the weight in the training loss, so the majority pattern dominates and 71 per cent follows. Bias and variance show up here as effects on demographic groups: inaccuracy concentrated on one set of people is a fairness finding rather than a tuning defect. Record it as one, in the model card alongside the headline accuracy, and keep the per-cohort slice in the monitoring after release, because representation drifts as the customer base changes.