CANON: The Reproducible Result (software as the experiment)

COMMISSIONED. The continuity bible for The Reproducible Result, promoted to the same status as CANON.md: when a post and this file disagree, fix the post, or fix this file in the same PR if the post is right.

This story is fully standalone: new cast, new institution, no shared characters with any other series. It lives on the same blog, so all the house chronology rules apply.

Everything institutional is fictionalised, per the Permit Office rules. The university is never named beyond “the university”. The reef system is the Culver Shoals, which does not exist. The monitoring programme, the state department that funds it and the number it publishes are all invented. No real reef, no real programme, no real agency, no real paper. The shape follows how long-running ecological monitoring and its reporting actually work, because the realism is what makes the stakes legible, but nothing maps onto a real place or a real result.

Premise

A marine ecology group in Perth has eleven years of survey data from the Culver Shoals, a model that turns it into a single published figure, and a pipeline that one postdoc can run. When Casper Ngo leaves for an industry job, the group discovers that the number it has been publishing for four years cannot be reproduced by anyone, including him.

Professor Marike Achterberg has run the group for nine years and has never once thought of herself as managing a software project. She is formidable in her own discipline and a beginner in this one, and the story is her getting a decade of science onto a footing where the number can be defended.

Why this inverts the blog’s usual economics

Everywhere else on this blog, software has to keep running. Here it has to be re-runnable once, years later, by a stranger, and produce the identical number. Nothing about Culver’s pipeline needs uptime, scale, latency or a deployment story. It needs to be an experiment somebody else can repeat, which is a different engineering problem with different answers, and that inversion is why familiar techniques land differently in this story than they do in the others.

Correctness and provenance are the product here, not quality attributes of it.

The clock

Publication	Story time	Beat
Feb 2032 (posts 1-2)	Feb - Apr 2032	Casper resigns; the first re-run fails; the four formats
Mar 2032 (posts 3-5)	Apr - Aug 2032	What the number carries; pinning the environment; the seed
Mar - Apr 2032 (posts 6-7)	Aug 2032 - Mar 2033	The off-by-one in the 2019 season and what to say publicly; the grant deadline
Apr 2032 (posts 8-9)	Apr - Nov 2033	The re-run that matches; a stranger reproduces it

Story time runs close to publication time here, which is unusual for this blog and harmless: nothing in this story touches another series’ clock.

Cast

Names collision-checked against all CANON files and the post corpus at commissioning.

Name	Role	Notes
Marike Achterberg	Professor, group leader, protagonist	Fifty-four. Nine years running the group, twenty-two in the field. Internationally known for the survey method the group uses, which she designed and which is genuinely good. Reads a figure in a paper faster than anyone in the building and has never run a command she did not copy from an email. Her arc is acquiring judgement about software without pretending to acquire skill.
Casper Ngo	Postdoc, leaving	Six years with the group, four of them the only person who could run the pipeline. Leaves for a Perth industry job at roughly twice the money. Not careless and not a villain: every shortcut in the pipeline was the right call for a person on a twelve-month contract, and the story is scrupulous about that. Stays reachable, answers every email, and genuinely cannot reproduce his own result either.
Sorrel Nilsen	Research assistant	Eleven years with the group, which makes her the longest-serving person in it including Marike. Keeps the field logs, in paper and in her head, and is the only reason the 2019 problem is ever found. The institutional memory of the whole operation sits with the most junior contract in it.
Jamila Berhane	PhD student, third year	Writes better code than anybody else in the group and is not supposed to be spending time on this, because her thesis is not about a pipeline. The story is honest that the rebuild costs her six months she does not get back and that she chooses it.
Merran Treadgold	Policy officer, the state department	Uses the Culver figure in a statutory report. Not an antagonist: her questions are the ones Marike should have been asking herself, and when the error surfaces she is the person who makes disclosure survivable.
Gunnar Steffensen	Researcher, another institution	Appears in post 9 only. Reproduces the result from the archive without contacting anybody, and emails afterwards to say so. His email is the story’s resolution and it is four sentences long.

The science, the number, and the stakes

The Culver Shoals: an invented reef system north of Perth. The group has surveyed it every spring since 2021 using Marike’s transect method: fixed sites, repeat photo quadrats, counts scored by trained observers.
The number: the group publishes an annual cover index for the Shoals, a single percentage with an interval on it. The 2031 figure is 34.1%, down from 41.6% in 2021.
Why it matters: the state department quotes the index in a statutory environmental report, and a management plan for the Shoals has a trigger point written into it at 30%. Below the trigger, a review of commercial access is mandatory. The index has been inside five points of the trigger for two years.
So the stakes are not academic and not commercial. Somebody is making a decision on this number, the decision has a legal consequence for other people’s businesses, and in February 2032 the group cannot say whether the number is right.

The pipeline, as found

Twelve years of accretion, every layer of it a reasonable decision at the time:

Eleven survey seasons in four formats: hand-entered spreadsheets (2021-2023), an Access database somebody built in 2024 and abandoned, CSV exports from a scoring tool (2025-2029), and the current structured capture app (2030-2031).
An R model, roughly 2,000 lines across nine files, no package structure, run by sourcing them in an order recorded in Casper’s email to himself.
A dependency on a spatial package that was archived from the public repository in 2029. Casper’s laptop has the last version anyone has.
Three manual steps between loading and the figure, none of them written down, one of them a visual check on a plot where Casper excluded sites by eye.
No seed set anywhere, in a model with a bootstrap in it.
The published figures live in the papers and in the department’s report; the code that produced them lives in a folder called analysis_final_v3.

Canonical numbers (printed in posts; update here when a post moves them)

Cover index: 41.6% (2021), 34.1% (2031). Management trigger 30%.
First re-run, post 1: Jamila gets 35.8% against the published 34.1%. The gap is the whole story’s engine.
Post 5: with the seed set, the bootstrap interval moves in the third decimal place, which is not the 1.7-point gap and proves the seed is not the cause. This is a deliberate false lead and the story says so.
Post 6: the 2019 scoring tool exported quadrat IDs off by one for part of one season, 2019, affecting 140 of 1,728 quadrats. Not a code defect in the model: a defect in a tool nobody in the group wrote. Corrected 2031 index: 33.8%, within the published interval.
Post 7: the rebuild costs Jamila six months. The grant resubmission is due 14 March 2033.
Post 8: the re-run on the rebuilt pipeline reproduces 33.8% exactly, from the archive, on a machine that has never seen the old code.
Post 9: Gunnar Steffensen reproduces 33.8% in eleven days, from the archive, without contacting the group.

Domain language

“season” only in its literal sense: a survey season, the spring fieldwork. Never a unit of publication. This story has more opportunity to break that rule than any other on the blog, so watch it in every draft.
“the index” for the published cover figure
“quadrat” for a photo sample, “transect” for a surveyed line, “site” for a fixed survey location
“scoring” for the trained-observer classification of a quadrat
“the department” for the state agency, never named further
“the archive” for what the group builds: the data, the code, the environment and the provenance, together

Story rules

All CLAUDE.md chronology rules apply: story time forward with publication order; techniques and vocabulary only after their introducing post; series-link backward only; never break the fourth wall; no publishing-unit words in prose. The word “season” is a particular hazard here and is always literal.
This is not a story about tidying up. The engine is the number’s public weight. Every scene that risks reading as housekeeping gets anchored back to the trigger point at 30% and to Merran Treadgold, who is going to publish whatever the group gives her.
Casper is not the problem. Every shortcut was correct for a person on a rolling twelve-month contract judged on papers. The story never has anyone blame him, and he is the one who says the hardest sentence in post 1. Key-person risk here is the default state of the discipline rather than anybody’s failure.
Marike is formidable and a beginner, at the same time, all the way through. She never becomes a programmer and never pretends to. What she acquires is the judgement to ask the right question of people who are, which is a real and separate skill.
The false lead is load-bearing. Posts 4 and 5 chase the environment and the seed, which are real problems that must be fixed and are not the cause of the discrepancy. The story does not apologise for that and does not signpost it; the reader finds out with the group.
Testing a model whose correct answer is unknown is the discipline’s hardest problem and the story treats it as hard. The answers it reaches are partial: known-input tests on synthetic quadrats, invariants (reordering inputs cannot move the index), and reproducing last year’s published figure as the regression test.
The disclosure in post 6 is handled like adults. Nobody covers anything up and nobody is destroyed. The correction is within the published interval, which is both a relief and the thing that makes it tempting to say nothing, and the story sits in that temptation for a scene before Marike does the right thing.
New techniques this story introduces (reproducibility as an engineering requirement, environment pinning, seeds and determinism, data versioning and provenance, the distinction between a result and the pipeline that produced it, code review where nobody is a software engineer, testing against unknown truth, the archive as deliverable) follow the introduction rules. Techniques the blog already teaches may be linked backward on natural phrases.
Posts are Tuesday mornings, standalone series (no collection), series: "The Reproducible Result", story banner on every post.

The posts (9, pub 2032-02-24 → 2032-04-20)

#	Pub date	Title	Beat	Teaches
1	2032-02-24	The Postdoc Leaves on Friday	Casper’s last week. Marike asks him to show Jamila how to run the model, which is the first time anyone has asked. It does not run. By Thursday Casper cannot run it on his own laptop either, and says the sentence the whole story hangs on.	Key-person risk as the default state; the difference between a result and the pipeline that produced it.
2	2032-03-02	Eleven Springs, Four Formats	The data. Spreadsheets, an abandoned Access database, CSV exports, the capture app. Sorrel’s paper field logs turn out to be the only record of which sites were excluded in 2023 and why.	Data provenance and lineage; why the oldest record is paper.
3	2032-03-09	What the Number Carries	Merran Treadgold visits, and the conversation is the first time Marike hears her own figure described as an input to somebody else’s decision. The trigger point at 30%. What it would mean for 34.1% to be wrong in either direction.	Stakes; what it means for a number to be load-bearing.
4	2032-03-16	A Package That No Longer Exists	Pinning an environment nobody documented, with a spatial dependency archived from the public repository in 2029 and surviving only on a laptop that now belongs to a company. Jamila gets the pipeline running. It produces 35.8%.	Environment pinning; reproducibility as an engineering requirement.
5	2032-03-23	The Seed Nobody Set	The bootstrap had no seed, which is a real defect and is fixed. Setting it moves the result in the third decimal place and not the 1.7 points, and the group has to let go of the explanation it wanted.	Determinism and seeds; the discipline of not accepting a cause that does not fit the size of the effect.
6	2032-03-30	One Hundred and Forty Quadrats	Sorrel, reading a 2019 field log for an unrelated reason, finds that the scoring tool exported quadrat IDs off by one for part of that season. The corrected index is 33.8%, inside the published interval. The scene where they decide what to tell the department.	Code review with no software engineers in the room; disclosure when the error is small.
7	2032-04-06	The Grant and the Rebuild	The resubmission is due 14 March and the rebuild is not done. Jamila is spending her thesis time on it. Marike has to decide what the group is for, and makes a decision that costs her something real.	Research-engineering time against publication pressure; what a group chooses to be.
8	2032-04-13	The Re-Run	A clean machine, the archive, one command, and 33.8%. The scene is deliberately undramatic and everyone in it is wrecked. Then the harder part: writing down what the archive does not cover.	The archive as deliverable; testing against unknown truth, and saying what the evidence does not reach.
9	2032-04-20	A Stranger in Trondheim	Coda, story Nov 2033: Gunnar Steffensen reproduces the index in eleven days from the archive and emails to say so, having needed nobody. Sorrel’s contract is made permanent. Jamila submits late and fine.	What reproducibility is actually worth, measured in a stranger’s eleven days.

Every post ends with a next-chapter-notice.html include pointing at the next post (estimated date = its publication date), except post 9.