The situation
A mechanical and electrical maintenance contractor holds planned and reactive contracts on about 4,000 commercial sites: offices, schools, distribution centres, two hospitals. Reactive work arrives as a callout, roughly 900 on a working day, by phone, by email and through a customer portal. Thirty schedulers triage every one. They read the description, set a priority band, pick a trade, and put the job in a window. There are four bands, from a four-hour attendance obligation on P1 down to next planned visit on P4, and eleven trades. Triage runs about eight minutes a callout. That is 120 hours against the 225 the desk has, so a little over half its day goes on triage and the rest on rescheduling and chasing.
A cold snap pushes callouts to around 1,400 a day for a fortnight at a time, mostly heating failures. The four-hour clock on P1 does not move for the queue.
The operations director’s board paper hands first-pass triage to a model, with schedulers reviewing every suggestion before a job reaches an engineer.
What actually matters
Data readiness fails here on the one test no amount of cleaning repairs: whether the thing being predicted was ever written down. Seven years of closed jobs, about 1.6 million, carry a description, a final priority, a trade, an engineer and a completion note. None records the band the scheduler chose at intake. A site manager escalates by phone, or an engineer downgrades on arrival, and the one priority column takes the new value. A model fitted to that column reproduces the end of a negotiation, not the judgement made at nine in the morning. The training labels, the past answers a model copies, are absent rather than thin, which puts a duration and a cost in the plan rather than stopping it.
Continuity is what the process does on the two kinds of day the model does not answer: the days it is unavailable, which everyone plans for, and the days it answers without confidence, which nobody does. A callout logged as “unit making a noise, site says urgent” has no confident answer in it. Both routes end with a human triaging, and a desk cut to twenty on the strength of an average day cannot run a 1,400-callout peak unaided.
Performance impact is how much human capacity the design consumes on the first day it runs. The model’s accuracy is a number on the way to that one. Reviewing a suggestion is quicker than triaging from scratch only when the suggestion is usually right. When it is wrong, the scheduler reads it, distrusts it, and does the original job anyway with the reading time added. Below some agreement rate the review desk is slower than the desk it replaced.
Cost is thinnest in the proposal. Inference, the charge for running one callout through a model, is the small number and the only one anybody quotes. Five other lines decide it. The recording change on the intake screen, the labelling programme, the parallel run, the review capacity staffed from day one, and the retraining when the world moves. Thirty salaries are a fixed monthly number. Consumption pricing moves with callout volume, which rises by more than half in a cold snap.
What we’ll filter on
- Data readiness is evidenced rather than assumed. Does the option rest on a recorded decision that exists today? If not, does it name what has to be produced first, and how many weeks that takes?
- The corrected history accrues to us. As the thing runs and people fix its mistakes, does the corrected decision end up in systems the business controls, or in a vendor’s?
- Continuity at peak. Can the process still absorb 1,400 callouts a day with the model switched off entirely?
- Performance impact is a number before go-live. Is the daily human review load computed in advance, or discovered in week one?
- Reversible against a stated floor. Is there a written accuracy floor, and a way back that somebody has actually rehearsed?
The landscape
Train on the closed jobs as they stand. Take the 1.6 million records, treat the final priority as the answer to be learned, and train a model to put each new callout in one of the four bands. It needs no new data collection, which is why it appears first in every version of this paper. The final priority stands in for the intake decision, and a dataset can be large, clean and still unfit for the thing you want to learn. Roughly one job in six changes band after intake. Those are the jobs where a customer pushed, which correlates with the customer rather than the fault. The model reproduces which clients complain loudest, and nobody notices, because the accuracy figure against the same column looks excellent.
Put the written priority matrix in the prompt. The four bands are defined in a contract schedule, in writing, with worked conditions. A foundation model on Amazon Bedrock, one of the general-purpose models a business rents rather than trains, reads the matrix and the callout text and names a band. No training run, no labelling exercise, no history, and rules the business can point at when a customer asks why. Amazon Bedrock bills on-demand use by the token, input and output metered separately at different rates, which is a small number at 900 callouts a day. It runs as suggest-and-review: the scheduler accepts or overrides every answer. Each override is the recorded intake decision the business lacks, made on today’s work, so this is also the cheapest way to produce the labelled set the trained-model option needs.
Label a set first, then train. Pull a sample of historical callouts weighted to cover every band and trade. Have experienced schedulers triage each one blind, settle the disagreements, and end with fifty or sixty thousand decisions that mean what they say. It is the textbook answer and it is not wrong. It costs six months of senior scheduler time before anything runs, and the review load is still unknown at the end, because nothing has been in front of a user.
Buy triage as a product. Field-service triage is close enough to commodity that products exist, sold as SaaS contracts and negotiable through an AWS Marketplace private offer, and buying closes the delivery gap fast. The accuracy figure in the deck was measured on the vendor’s data and says nothing about a portfolio with two hospitals in it, so a parity test on this firm’s own callouts is still owed. Every correction a scheduler makes over the next two years accrues in the vendor’s platform rather than in the firm’s own records.
Change nothing except what gets recorded. Add one field to the intake screen so the band chosen at intake is stored separately from the band the job finishes on, and leave the process alone. It is two weeks of work on the field-service system. It builds nothing, saves nothing and demonstrates nothing, and in a year it turns the trained-model option from impossible into ordinary.
Evaluation
Side by side
| Option | Readiness evidenced | Corrections accrue to us | Peak survivable unaided | Review load known first | Floor and a way back |
|---|---|---|---|---|---|
| Train on the closed jobs as they stand | ✗ | ✓ | ✓ | ✗ | ✗ |
| Matrix in the prompt, suggest-and-review | ✓ | ✓ | ✓ | ✓ | ✓ |
| Label a set first, then train | ✓ | ✓ | ✓ | ✗ | ✓ |
| Buy triage as a product | ✗ | ✗ | ✓ | ✗ | ✗ |
| Record the intake decision, change nothing else | ✓ | ✓ | ✓ | ✓ | ✓ |
Two rows clear every column and they are not competing. The last row clears the table because it does nothing. That makes it stage zero rather than an answer: without it, no recorded decision exists to measure the others against. The row above it is the answer, and the two run in sequence. Training on the closed jobs fails the first column for a reason no later stage repairs. Buying fails four of the five, and most of those are one problem: the evidence and the accumulated corrections sit with somebody else.
The solution
The recording change goes in front of everything else, and the operations director owns it whether or not the board funds the rest.
Record the decision before you try to reproduce it. Two weeks of work on the intake screen stores the band, trade and window chosen at intake, separately and immutably, alongside the ones the job finishes on. The business then accumulates about 900 rows a working day whether or not the board approves anything else. Six weeks of that is 27,000 recorded decisions.
Run silent before you run in front of anybody. Stage one is the Bedrock model with the contract priority matrix in the prompt, reading every callout as it arrives. Its answer goes to a table nobody looks at. That consumes no scheduler minute and produces nine hundred pairs a day, model answer against scheduler choice. Six weeks gives the agreement rate and the bands the model is worst on.
State the floor as bands, not as one number. Filing a genuine P1 as a P3 misses a four-hour attendance obligation; raising a P3 to P1 sends an unnecessary van and breaches no contract. So the floor is two conditions, measured on the silent run. At least ninety-nine in a hundred genuine P1s caught. And band by band, at least eight in ten of the model’s calls agreed by the scheduler. Write both down before the silent run ends, or the numbers get chosen to match the result.
Size the review desk with arithmetic, then staff it. Accepting a suggestion takes three minutes. Reading it, rejecting it and redoing the triage takes eleven. Against the old eight minutes the desk breaks even at about thirty-eight per cent agreement, which the silent run either clears or does not.
Keep the manual path warm through one peak. Triaging 1,400 callouts by hand is 187 hours of work against the 225 hours the desk has. It can be done, and nothing else can. A desk cut to twenty has 150 hours and cannot do it at all. Redeploy the hours the model frees into the chasing backlog rather than removing roles in the first year. The design also needs a no-answer route. Low confidence, a hospital site, or any callout mentioning gas, water ingress or a lift goes to a human with no suggestion attached.
Put every line in the cost paper, with inference the smallest. The intake recording change, one-off. The parallel run, where both paths do the same work for a fixed window. The review desk, staffed at full strength through the first winter. Retraining and re-evaluation, recurring, because a model stops matching the world without anything breaking and the symptom here would be a drift in band mix that looks like weather. And the Bedrock consumption, modelled in the AWS Pricing Calculator before build and watched in AWS Cost Explorer after it, tagged so this workload is its own line. Cost Explorer holds thirteen months of history and forecasts eighteen months out. That is short of the paper’s three-year horizon, so the later years stay a modelled number.
Worked example
Six weeks of silent running produces 27,000 pairs. Agreement comes out at seventy-eight per cent overall, and unevenly: ninety-one per cent on P4, eighty-four on P3, sixty-two on P2, fifty-five on P1. P1 is the worst because the description carries least and the scheduler is working from site knowledge.
At seventy-eight per cent the average handling time under review is 0.78 times three minutes plus 0.22 times eleven, which is 4.8 minutes against eight. Across 900 callouts that is 71 hours a day against 120. Roughly 49 hours come back to the desk, a little over six roles, well clear of the thirty-eight per cent break-even.
P3 and P4 are two-thirds of volume, split about evenly, and both clear the eight-in-ten floor. Those are the bands the model triages. P1 and P2 fall short of it and are not offered to the model in the first year, which leaves the four-hour obligation with a person. That is 600 callouts a day at an average agreement of eighty-seven and a half per cent. The saving falls to 40 hours a day rather than 49, a little over five roles, and that is the number the paper should carry.
Over three years that is about AUD$1.3 million of desk capacity against AUD$305,000 of spend. A fully loaded scheduler costs the firm about AUD$82,000 a year, so 40 hours a day is about AUD$440,000 of capacity. The one-off lines come to about AUD$95,000. AUD$20,000 for the intake recording change, AUD$30,000 for the silent and parallel runs, and AUD$45,000 for retraining the desk and reworking the process. The recurring lines are about AUD$8,000 a year of Bedrock consumption and AUD$62,000 of quarterly re-evaluation. The first year redeploys the capacity into the chasing backlog rather than taking it off the payroll.
What’s worth remembering
- An outcome is not a decision. A closed job records where the priority ended up, not what intake chose, and cleaning cannot create it.
- Record before you reproduce. Two weeks on the intake screen produces 900 decisions a working day, and makes next year’s trained model possible.
- Continuity sizes the desk at peak. A desk staffed for an ordinary day cannot absorb 1,400 callouts, so the peak sets the headcount floor.
- Review has a break-even rate. Day-one impact is the review load, and below thirty-eight per cent agreement the desk is slower than triaging fresh.
- Inference is the smallest line. Transition cost is the recording change, labelling, the parallel run, the review desk staffed from day one, and retraining.
- State the floor per band. A missed P1 breaches a four-hour obligation; an over-called P3 sends a van, so one accuracy number hides both.