Forty Ideas and Nowhere to Start

The situation

A commercial and personal lines insurance broker, 900 people across eleven offices, ran an internal call for AI ideas over six weeks. The submission form asked for a title and a paragraph. Forty came back: twelve from claims handling, eleven from sales and marketing, seven from the actuarial team, six from the in-house development group, and four from finance and compliance.

No two of them describe the same kind of object. One asks for a chatbot on the claims line. One asks for a model that prices commercial motor. One asks for a coding assistant for the twelve developers. One asks for the marine cargo survey reports to be summarised. Several name a vendor rather than a problem. Several name a problem with no idea attached to it at all.

The chief operating officer sponsors the exercise, has build budget for two or three of these in the coming year, and has a steering group meeting in three weeks. She needs a shortlist she can defend and a way to tell the other thirty-odd submitters something better than “not this time”.

What actually matters

The forty are not comparable because they are not the same category of thing. A chatbot is a delivery surface, a pricing model is a decision, a coding assistant is a licence with a per-seat bill and no build at all. Ranking those three on impact and effort ranks how confidently each paragraph was written, which tracks the seniority of the writer and nothing else. Two properties they all share make them comparable: the business number each would move, and the class of capability that could move it. Written that way, the claims chatbot and the coding assistant both move a number by generating text, so they are the same shape and can be argued about.

Anchoring to a number forces the baseline into the open before anything is built. Cost per claim is measured today and sits in a monthly pack; developer throughput is not measured here at all, so the first action on any developer-productivity idea is six weeks of measuring cycle time, not procurement. A benefit that cannot be checked against a number that already exists cannot be checked later either, which is how an AI business case ends up funded on an estimate and reviewed against nothing.

Twelve submissions from claims and none from underwriting operations does not mean claims holds the opportunity. It means the claims director forwarded the email and the underwriting operations manager did not. An assessment that ranks only what arrived inherits that sampling bias and locks it into a year of budget, so the exercise needs a second source: a walk through the functions of the business, asking what each one spends its hours on.

Most of the forty will not make the shortlist, and each of those needs a named reason rather than a low score. A score gets the paragraph rewritten with more confidence and resubmitted next quarter. A reason gets fixed, or gets the idea dropped, and some reasons route somewhere useful: “the data does not exist” is an instruction to a data owner, not a dead end.

What we’ll filter on

1. Comparability. Does the method put a chatbot, a pricing model and a code assistant on one axis, or does it compare them on how well the paragraph was written?
2. Coverage beyond the inbox. Does it find work that nobody submitted, or does it only rank what arrived?
3. Outcome anchoring. Does every survivor name a business number that is measured today, with the baseline recorded before anything is built?
4. Rejections that carry a reason. Does a “no” tell the submitter what would have to change, so it does not return unchanged next quarter?
5. Repeatable in-house. Can the broker run this again in six months on its own, without a fresh fee and without another month of senior time?

The landscape

Score what arrived

Build a sheet with columns for business impact, feasibility, cost and strategic fit, score all forty out of five, weight, sum, sort. The rows are not comparable: the claims chatbot scores a four on impact because its author asserted a four, the marine cargo summariser a two because its author was honest about a small book. It holds only what was submitted, so it says nothing about coverage, and a ranking is not a reason, so the ones that miss out come back unchanged.

Scan the value chain function by function

Walk the business one function at a time, from new business quoting through underwriting support, policy servicing, claims handling, finance and compliance, asking each what its people spend hours on. It is the coverage answer, and it surfaces the policy-servicing work nobody submitted. It also takes a month of senior time across five function heads, so it misses the steering group by a week. It produces its own long list rather than converging, and it discards the forty ideas the business already spent six weeks collecting.

Import an industry list of proven use cases

Take a published list of broker and insurer AI use cases and adopt the ones that apply. Cheap, quick, and a useful coverage checklist of what worked at other companies with other data. First notification of loss extraction sits on every such list and is a good idea here; churn prediction sits on every such list and is impossible here, because nothing in this broker’s systems records a relationship touchpoint. The list says nothing about which side of that line an idea falls on.

Buy a vendor-run opportunity assessment

Bring in a consultancy or an AWS partner for four to six weeks of workshops and a prioritised roadmap formatted for a board. Three quotes came back between AUD$140,000 and AUD$220,000. It supplies outside pattern knowledge and the senior time to run it. The list reflects what the vendor can deliver, so a partner whose practice is generative AI returns a roadmap of generative AI, and nobody inside the broker learns to run it, so the fee recurs next year. Offerings of this kind sit under Professional Services in AWS Marketplace, in its Assessment category. Every professional services offer there is a private offer: the buyer requests one from the listing, settles scope and price with the seller, and the engagement then lands on the same AWS bill as the software.

Two passes over the forty: map, then reject

Take the forty as they arrived. The first pass maps each to the business outcome it would move and the capability class that could move it, returning anything that cannot be written that way for one more sentence. The second pass rejects the ones where the outcome is real but AI is the wrong instrument, and a light value-chain scan runs alongside so the shortlist is not only what the mailbox caught. Slower than a scoring sheet, much faster than a vendor assessment, and the only one of the five producing a written reason per rejection.

Evaluation

Side by side

Approach	Comparable rows	Coverage beyond inbox	Outcome anchored to a baseline	Rejections carry a reason	Repeatable in-house
Score what arrived	✗	✗	✗	✗	✓
Value-chain scan	✓	✓	✓	✗	✗
Imported industry list	✗	✓	✗	✗	✓
Vendor opportunity assessment	✓	✓	✓	✗	✗
Two-pass assessment	✓	✓	✓	✓	✓

The scoring sheet is the cheapest and fails four of the five columns: rows that cannot be compared, no coverage, no baseline, no reason. Its one tick is that the broker can run it again, which counts for nothing if the output was not worth having. The value-chain scan and the vendor assessment both produce comparable, outcome-anchored lists. Neither tells the thirty-odd submitters who miss out anything, the scan costs a month of senior time the three-week deadline does not allow, and the vendor fee recurs next year. The two-pass assessment clears all five. It runs on about ten days of one analyst’s time plus an afternoon from each of five function heads, under AUD$10,000 of internal cost against a vendor quote starting at AUD$140,000. It absorbs the value-chain scan as its coverage source rather than treating it as a competing method. The vendor fee is worth paying where the broker has no analyst to spare, or where the board wants an outside name on the roadmap.

Two passes over forty submissions

The solution

Pass one, map the outcome

Four numbers are candidates for AI to move: cost per claim, quote conversion rate, renewal retention rate, and developer throughput. Every submission attaches to exactly one. Three of the four already sit in a monthly pack, so today’s value and its source go next to the submission. Developer throughput is measured nowhere, so a submission attaching to it starts with six weeks of cycle-time measurement before it can be costed. A submission naming none of the four goes back for a sentence rather than being rejected, which about a third of the forty needed.

Pass one, map the capability class

Five classes are worth distinguishing at portfolio level; turning one of them into an actual problem statement comes after the funding decision, and belongs to whoever builds it.

Prediction estimates a number or a likelihood from history, such as which open claims will exceed reserve. Classification sorts an input into a fixed set of buckets, such as the occupancy code for an exposure record. Both learn from the broker’s own labelled history on Amazon SageMaker AI, so somebody assembles and labels that history before anything can be trained, and that runs to months rather than weeks. Extraction pulls structured fields out of unstructured documents: the loss date, the vehicle and the third party from a first notification of loss pack. Generation produces text. Both run on Amazon Bedrock against a foundation model, a general-purpose model that arrives already trained, so neither waits on the broker to label anything. Agentic execution takes several steps and acts in other systems rather than answering a question, which is what makes it the hardest of the five to govern and to cost before it is built. Where the ask is questions over the broker’s own documents, Amazon Quick is a finished product with nothing to build.

Pass two, the four rejections

The decision has to be deterministic and auditable. Nine ideas asked a model for a call that has to come out the same way twice and be justified to the customer it went against. A declined commercial motor risk comes back as a request for reasons, the broker has to say what the decision turned on, and a score below a threshold is not a reason a person can be given. Where written terms applied to recorded facts fix the answer, the thing being asked for is a rule and not a prediction.

The underlying process is broken. Eight ideas would automate a process that does not work. The renewal-letter drafter takes its list from a CRM whose contact records are around a third stale, so personalising the letter reaches a third of the names wrongly, faster and at scale. The reason routes to the CRM owner and the idea queues behind a fix.

The data does not exist. Ten ideas assume a record nobody keeps. Predicting which SME clients will lapse needs a history of relationship contact, and this broker logs a phone call only when it opens a case; claims notes from before the 2024 platform migration were migrated as flat images, not text. The largest rejection group and the most productive: it becomes a data-readiness backlog with named owners.

The volume cannot repay the build. Seven ideas are correct, feasible and too small: forty marine cargo survey reports a year at an hour of reading each, so perfect automation saves about five working days against a build measured in months. The test comes after the mapping, because forty hours a year is enough to justify a class with nothing to build, and never enough for a trained model on its own serving instance.

What the eight look like

Six of the forty cleared both passes and two more came out of the value-chain scan. Four are generation or extraction work on Bedrock, two are prediction or classification on SageMaker AI, and two are bought by the seat with nothing built. The two nobody submitted came from policy servicing, where the mailbox goes unread and a quarter of the department’s hours go on re-keying mid-term adjustments.

Worked example

Submission	Outcome it moves	Capability class	Verdict
Chatbot on the claims line	Cost per claim	Extraction, then generation	Survives, reshaped: the value is structured intake, not a conversational front end
Price commercial motor with a model	Quote conversion	Prediction	Rejected: the decision has to be reproducible and explainable to a declined customer
Auto-draft and send renewal letters	Renewal retention	Generation	Rejected: the renewal list is a third stale, so this scales a bad list
Coding assistant for the developers	Developer throughput	Generation, bought by the seat	Survives, with a six-week baseline measurement first
Predict which SME clients will lapse	Renewal retention	Prediction	Rejected: no record of relationship contact exists to learn from
Summarise marine cargo survey reports	Cost per claim	Generation	Rejected: forty reports a year cannot repay a build

As submitted, the claims chatbot is a delivery surface with no outcome attached. Mapped, it becomes extraction of the loss date, vehicle, third party and policy reference from an inbound pack, which is where the handling minutes go, and generation of the acknowledgement, which is where almost none of them go. Splitting it cut the estimate by an order of magnitude and changed who sponsors it.

What’s worth remembering

1. Map every idea twice. To the business number it would move, and to the capability class that could move it; unmapped, you rank the writing.
2. Send it back before rejecting it. A submission naming no business number gets one more sentence, not a low score; about a third needed that.
3. A chatbot is not an outcome. Split a delivery surface into the extraction that saves handling minutes and the generation that saves almost none.
4. Measure the baseline before you build. Anchor every survivor to a number already measured, because once the tool is in nobody can reconstruct the before.
5. Rejections name one of four reasons. Decisions needing determinism and audit, a broken process, absent data, or volume that cannot repay the build.
6. A call for ideas samples enthusiasm. Pair it with a function-by-function pass over the value chain, or the shortlist maps who forwarded the email.