“Data-driven” sounds like a compliment. It sounds rigorous. It sounds like the opposite of guessing. But the most useful decision-makers do something data-driven organisations often forget: they add context. “The data suggests X” is an observation. “The data says we should do X” is an abdication of judgement. The teams that get hard calls right are the ones that remember numbers don’t make decisions; people do.
The distinction
Data-driven means the data decides. You look at the numbers, the numbers point in a direction, and you go that direction. It sounds scientific. It feels safe. If the decision turns out badly, you can point at the spreadsheet and say “we followed the data.”
Data-informed means data is one input. Alongside it: judgement, context, experience, domain knowledge, and the messy human understanding that numbers can’t capture. The data informs the decision; it doesn’t make the decision.
The difference isn’t academic. It changes what you do when the data points one way and your gut points another. A data-driven organisation ignores the gut. A data-informed organisation investigates the tension, because sometimes the gut is picking up on something the data hasn’t measured yet.
Take a number that sounds catastrophic in isolation: a $3 margin per box on a produce subscription. The data-driven response is “your business is doomed.” The data-informed response is “your margin is $3 per box. What do we think that means at 1,000 subscribers? At 10,000? Against what competition? With what runway?” The number is identical. The question is different. The data-driven version closes the conversation. The data-informed version opens it.
This isn’t just a communication style; it’s a position on what data actually is. Evidence needs interpretation. Truth doesn’t. Treating a number as truth is how you end up making confident decisions based on figures that don’t mean what you think they mean.
The same number, opposite meanings
The clearest example is the cohort-age trap. A multi-city subscription business tracks monthly churn by region:
| City | Monthly churn |
|---|---|
| Perth | 2.8% |
| Melbourne | 3.4% |
| Brisbane | 3.9% |
| Hobart | 5.3% |
The data-driven conclusion is obvious. Hobart has a churn problem. Investigate. Intervene. Maybe pull back and focus resources on the cities that are working.
The data-informed question is different: how old is each market? Perth has been running for three years. Melbourne for eighteen months. Brisbane for nine months. Hobart for three months.
Churn in a new market is always higher than churn in an established one. Early subscribers include people who are trying something new but haven’t committed. The first six months of any market’s subscriber base include a natural shake-out: the curious leave, the committed stay. Perth’s 2.8% is a mature-market number. Hobart’s 5.3% is a launch number.
Pull Perth’s churn from its first three months: 6.1%. Higher than Hobart’s.
Hobart isn’t failing; Hobart is doing better at three months than Perth did at three months. Treat 5.3% as a crisis and you’ll over-invest in retention for a market that’s performing above the historical baseline. The data-driven response diverts resources to Hobart. The data-informed response recognises that the same number means different things depending on context. 5.3% churn for a three-month-old market is healthy. 5.3% churn for a three-year-old market would be catastrophic. The number is identical. The meaning is opposite.
This pattern (numbers that look alarming in isolation but make sense with context) shows up constantly in subscription businesses. Churn, revenue per subscriber, acquisition cost, NPS: all of these vary by cohort, by season, by region, by how long you’ve been running. Stripping out the context and reacting to the raw number is how teams make expensive mistakes that feel rigorous.
Trajectory matters more than the snapshot
Net Revenue Retention (the percentage of revenue you keep from existing customers after accounting for churn, downgrades, and upgrades) is another number that gets misread when stripped of context.
A consumer subscription business with 92% NRR looks alarming on a board slide. In SaaS, anything below 100% means you’re shrinking your existing base. Even allowing for the lack of upsell in a produce-box business compared to software, 92% reads as a problem.
Refuse to present it without the trend line. Three quarters ago, NRR was 86%. Two quarters ago, 89%. Last quarter, 92%. The trajectory matters more than the snapshot. Ninety-two percent in isolation is a problem. Ninety-two percent as the latest point in a six-point improvement trend is a story about a team that’s learning. Show both.
Investors who see only the snapshot react to the snapshot. Investors who see the trajectory respond to the trajectory: “the improvement rate suggests you’ll cross 100% within two quarters if you maintain the current interventions.”
A data-driven team reading the 92% snapshot might panic, cut costs, or pivot away from something that’s working. The data-informed approach is to read the trajectory and ask: are the things we’re doing causing the improvement, and can we sustain them?
Output metrics that hide outcome blindness
The most seductive version of the data-driven trap looks like productivity. Twelve features shipped in a quarter. Four per squad. All on time. The output metrics are stellar. If you measure the team by throughput, they’re having their best quarter ever. (For a worked example of this exact failure mode, see The Feature Factory.)
The data-informed question at quarterly review: which of these twelve features is responsible for your subscriber growth this quarter?
If nobody can answer, nobody has connected features to outcomes. The team has optimised for output (features shipped) and forgotten to measure impact (what those features actually changed).
The dashboard is green. Velocity is up. Deployment frequency is high. Every metric that measures activity looks excellent. The metrics that measure value (retention, revenue, subscriber satisfaction, NPS) aren’t connected to the features that supposedly drive them.
The fix isn’t more metrics; it’s changing which metrics matter. Before building any feature: what number will move if this works? After shipping: did it move? If you can’t answer the first question, don’t build it. If the answer to the second is “no,” learn from it.
This is the difference between data-driven and data-informed taken to its logical end. A data-driven team optimises the numbers on the dashboard. A data-informed team asks whether the dashboard is measuring the right things.
What you can’t measure
Not everything that matters can be measured.
A team is building a supplier-scoring system: consistency, variety, yield reliability, delivery timeliness. Four metrics, weighted, producing a score from 0 to 100. The suppliers with the highest scores get priority in the matching algorithm.
A long-time supplier listens to the design and says: “You can measure soil pH but you can’t measure whether a tomato tastes like home.”
It’s the kind of statement that sounds like a folksy platitude. It isn’t. The best tomato from his farm (the heirloom variety he grows because his father grew it) isn’t the most consistent, the highest-yielding, or the most punctually delivered. It’s the one that tastes right. Subscribers write in about it by name. It’s irreplaceable, and it would score poorly on every metric in the system.
Qualitative judgement matters. Not instead of data, alongside it. The supplier-scoring system is useful for flagging reliability problems. But the final decision about which suppliers to feature, which produce to prioritise, and what goes into the curated experience involves taste, relationship, and story: things that exist outside any dashboard.
The fix is a qualitative field next to the quantitative ones: a “curator’s note” free-text column where the subjective assessment lives alongside the score. The score filters. The note decides.
The dashboard discipline
A mature board pack carries a small set of metrics. Five is plenty for a single-product company: subscriber count, monthly churn by region, NRR, gross margin, customer acquisition cost. Five numbers, one page, clean and scannable.
What separates a useful pack from a misleading one is a section the numbers can’t convey: “What Changed.” Three to five bullet points explaining the context behind the numbers. Not just what the numbers are; what happened in the world that made them move.
“Churn increased 0.4% in Brisbane. Contributing factor: a competitor launched a $15 meal kit there in October. We’re monitoring but not reacting yet; their product quality is significantly lower and we expect their churn to be higher than ours within two quarters.”
“NRR improved from 89% to 92%. We attribute this primarily to the pause feature (19 subscribers paused and resumed this quarter who would otherwise have cancelled) and the recipe-card quality improvements that shipped in Q3.”
“Customer acquisition cost dropped from $48 to $39. This is mostly seasonal; Q4 always sees higher organic signups as people plan New Year health goals. We don’t expect this to sustain into Q2.”
The numbers without the narrative are misleading. The narrative without the numbers is vague. Together, they’re a decision-making tool.
This is what data-informed looks like in practice. Not “ignore the data.” Not “trust your gut instead.” It’s “present the data with enough context that intelligent people can make good decisions.” Board members aren’t data scientists; they’re investors, operators, and advisors with experience and judgement. Giving them raw numbers insults that judgement. Giving them numbers plus context respects it.
The vanity metric trap
There’s a related failure mode worth catching early: vanity metrics. Numbers that go up and to the right but don’t measure anything that matters.
Subscriber count is the classic example. Twelve thousand subscribers sounds impressive. But subscriber count doesn’t tell you whether those subscribers are profitable, whether they’re staying, or whether the business can serve them sustainably. A company with 12,000 subscribers at negative margin is worse off than a company with 6,000 subscribers at positive margin. The bigger number feels better. The smaller number works better.
A simple test for distinguishing vanity metrics from real ones: does this metric change your behaviour? If subscriber count drops by 500, what do you do differently? If you don’t know, it’s a vanity metric.
Teams push back on this. “Subscriber count isn’t vanity. It’s our primary growth metric.”
It’s your primary reporting metric. Your primary growth metric is NRR, because NRR tells you whether growth is sustainable. You can add 500 subscribers this month and lose 600. Subscriber count goes up by 500. NRR tells you the truth.
This reframing (from the number the board asks about to the number that changes behaviour) is the same distinction as data-driven versus data-informed, applied to which metrics to track. A data-driven organisation tracks the numbers that look good. A data-informed organisation tracks the numbers that tell the truth.
Survivorship bias and the data you don’t have
One of the subtlest traps is the data you don’t have.
A subscription business that surveys only active subscribers gets consistently positive results: high satisfaction, strong NPS, nice things about the product. Survey day feels affirming. The numbers are good.
The data-informed question: what about the people who cancelled?
The survey only goes to active subscribers. The people who left (the ones with complaints, the ones who found a competitor cheaper, the ones whose dietary or accessibility needs weren’t met) aren’t in the data. The survey is measuring the satisfied survivors, not the dissatisfied departed. This is survivorship bias, and it’s rampant in subscription businesses. Active customers like your product; that’s why they’re still customers. The interesting data is why people left, and the people who left aren’t answering your surveys.
The fix is to make the conversation, not the survey, the artefact. Call every cancelling subscriber. Not a form, a conversation. “We’re sorry to see you go. Could you tell me why, so we can improve?” Half won’t respond. Of the half who do, the reasons are illuminating. In one real example: price (32%), didn’t cook enough to justify a weekly box (28%), moved (15%), dietary restrictions not met (12%), quality concerns (8%), other (5%).
The 28% who “didn’t cook enough” was invisible in the active-subscriber data. Nobody who stayed had that problem, by definition. But it represented the largest addressable churn reason after price. The product response (adding a fortnightly box option) came from data the team would never have found in a satisfaction survey.
The data you collect is always incomplete. The data-informed discipline includes asking: what are we not measuring, and could the missing data change our decision?
When to trust the data
None of this means data doesn’t matter. There are situations where the numbers should override your instincts.
When the sample size is large enough. A useful rule of thumb: don’t draw conclusions from fewer than thirty data points. A/B tests with twelve participants are noise, not signal. But when you have 12,000 subscribers and a clear measurement, the data is reliable.
When the trend is consistent. A single month of high churn could be noise. Six months of rising churn is a signal. Trends that persist across seasons, cohorts, and regions are almost always real.
When your gut has a known bias. A founder whose identity is tied to a particular product belief will resist data that challenges that belief, even when the data is solid. (One worked example: a founder convinced that local sourcing was the brand’s primary draw discovered, in JTBD interviews, that 60% of subscribers didn’t care about local sourcing. Her gut said the data was wrong, but the data was right.) Knowing your biases is what lets you override them when the data is strong.
When to trust your gut
And there are situations where the data should take a back seat.
When the data can’t measure what matters. The tomato that tastes like home. The brand feeling. Whether the recipe cards spark joy or just convey instructions. Customer experience has dimensions that quantitative data can’t capture, and qualitative judgement is the only tool that reaches them.
When the context has changed. Historical data reflects historical conditions. If a new competitor enters the market, last quarter’s churn data is about the old competitive landscape. Your gut, informed by awareness of the change, may be more current than your dashboard.
When you’re making a values decision. A commitment to paying suppliers fairly is a values decision, not a data decision. The data might show that squeezing supplier margins would improve the bottom line. The values say that’s not who we are. Data can tell you what’s efficient; it can’t tell you what’s right.
When to run an experiment
The best response to a conflict between data and gut is often: don’t decide. Experiment.
(For a worked example: the two-tier pricing decision was effectively this. Survey data said 60% of customers were convenience seekers who might accept a cheaper mixed-sourcing box. The founder’s gut said abandoning 100% local sourcing would betray the brand. Neither view was provably correct. The fix: launch both tiers. Track which one new subscribers choose. Track whether existing Local Box subscribers switch. Let the market tell you what the theory can’t.)
This is the third option data-driven organisations often miss. When the numbers say one thing and your judgement says another, the answer isn’t always to pick one. Sometimes the answer is to design a cheap experiment that produces better data. The experiment costs more than just deciding, but it’s cheaper than deciding wrong.
There’s a discipline to this: define the experiment before you run it. What are you testing? What will you measure? How long will you run it? What result would make you change direction, and what result would confirm your current path?
Without these guardrails, experiments become open-ended. “Let’s try it and see” is not an experiment; it’s hope with a plan to measure something later. Every experiment needs a hypothesis, a metric, a duration, and a decision rule. “If churn drops below 4% in the new-tier segment within eight weeks, we’ll expand to all regions. If it doesn’t, we’ll investigate why before expanding.”
The discipline matters because experiments without decision criteria never end. They become “well, the results were mixed, let’s run it longer,” which is just deferring the decision. The data-informed approach is to decide what data would change your mind, then collect that data, then actually change your mind if the data says to.
The danger of dashboard culture
There’s a pattern that develops in organisations that take metrics seriously: dashboard culture. Screens on walls showing real-time numbers. Daily check-ins about the graphs. Colour-coded alerts when a metric crosses a threshold.
Dashboard culture feels like being data-informed. It looks rigorous. Everyone can see the numbers. Decisions happen fast because the data is visible.
But dashboard culture often produces the opposite of data-informed decision making. When a number on a wall turns red, the instinct is to react. Churn spiked this week, do something! Acquisition cost jumped, investigate! The dashboard creates urgency where there might not be any, because a red number feels like a fire even when it’s just noise.
A useful antidote is a principle worth naming: signal over alarm. A single data point is noise. Two consistent data points might be a signal. Three consistent data points is a trend. Don’t react to noise. Investigate signals. Act on trends.
The corresponding dashboard discipline: rolling averages and trend lines rather than point-in-time values. Show “churn 4-week average: 3.6%, trending flat” instead of “churn this week: 4.2%.” The first presentation invites observation. The second invites panic. The same data, presented differently, produces different decisions.
This is a small design choice with large behavioural consequences. A data-informed organisation pays attention to how it presents data, not just what data it presents.
The practice
The consistent practice (data suggests, not data says) isn’t a communication trick; it’s a discipline of thought.
Never present a number without context. Never recommend an action based solely on a metric. Always invite the team to interpret, challenge, and add their own knowledge. Treat data as a starting point for conversation, not an ending point for debate.
Hold yourself to the same standard. When your own models suggest something you don’t believe, say so. “The model says we should cut the recipe-card budget. I don’t believe it. The model doesn’t capture the brand value of the cards, and I think cutting them would increase churn in a way the model can’t predict. Let’s keep the budget and track churn for two quarters. If I’m wrong, the data will show it.”
That’s data-informed decision-making. The data has a voice. So does experience. So does judgement. The decision lives at the intersection of all three.
The human in the loop
There’s a philosophical underpinning to data-informed decision making that’s worth naming: it assumes humans are valuable.
A purely data-driven approach implies that the ideal decision maker is an algorithm. Given enough data, the algorithm decides correctly. Human judgement is noise (bias, emotion, politics) and the goal is to minimise it.
A data-informed approach implies the opposite. Human judgement isn’t noise; it’s signal. Experience, intuition, empathy, and domain knowledge are forms of information that data can’t capture. The goal isn’t to minimise human input; it’s to combine human judgement with quantitative evidence so that each compensates for the other’s weaknesses.
Data is good at scale, consistency, and pattern detection. Humans are good at context, novelty, and values. A churn number tells you how many people are leaving. A conversation with a leaving customer tells you why, and “why” is almost always more actionable than “how many.”
There’s a quote often attributed (probably wrongly) to W. Edwards Deming: “In God we trust. All others must bring data.” The data-informed addendum: And context.
The principle
Data-informed means the data is one input alongside judgement, context, and experience. Not above them. Not below them. Alongside.
The teams that get this right are the ones whose data lead never hides behind the numbers and never dismisses them either. They present evidence with context, and trust the team to make good decisions.
The three failure modes earlier in this post (over-reacting to a younger market’s churn, panicking at an NRR snapshot, celebrating a feature factory’s output) are all cases where the data read in isolation pointed the wrong way. Context fixes each one.
If your organisation says “we’re data-driven” like it’s a badge of honour, ask one question: what happens when the data conflicts with judgement? If the answer is “we follow the data,” you’re not rigorous; you’re abdicating the hardest part of decision-making to a spreadsheet.
The spreadsheet doesn’t know about the tomato that tastes like home. And the tomato matters.
If you take one thing from this post, make it the habit. Next time you present a number to your team (a churn rate, a conversion rate, a velocity metric, a customer score) add one sentence of context. “Churn is 5.3%. That’s in a three-month-old market, and our mature markets were at 6.1% at the same age.” That one sentence changes the number from an alarm into information. It changes the team’s response from reaction to reasoning.
Data-informed isn’t harder than data-driven. It’s one extra step: the step where you ask “what does this number actually mean, given what we know?” Ten seconds. It prevents decisions that take months to undo.
Related references
- Retrospectives at Every Scale: the feedback loop that lets the data change your mind
- The Planning Onion: where each metric belongs in the planning rhythm
- The Feature Factory: worked example of output metrics hiding outcome blindness
- Jobs to Be Done: worked example of data that overrode a founding assumption
- What Changes First: worked example of the experiment that resolves a data/gut conflict
- The Greenbox Story: the full narrative behind the worked examples