Involvement of Claude Code

From v2.0.0 onward this pipeline has been developed with Claude Code, an AI coding agent, working against my review. Everything before v2.0.0 — the pool-seq reasoning, the filtering approach, the Nextflow pipeline itself — I wrote alone. This page says what the arrangement actually is, because "developed with AI" covers a range wide enough to be meaningless, and because I would rather describe it than have it guessed at.

The short version: the work got faster and the expertise it demands went up, not down.

What the loop looks like

I decide what to build and why. The agent proposes an implementation, usually with an argument for it. I read the code and the argument, and most of the time I send something back — this is wrong, this is slower than it needs to be, this contradicts a decision we already made, this assumption does not hold on real data. Then it changes, and I read it again. Work stays in the working tree until I have reviewed it; nothing is committed that I have not seen.

That is the whole method. It is not novel and it is not automated. What is worth writing down is what it demands from the person doing the reviewing.

Why this needs more experience rather than less

The errors run in both directions, and the corrections do too.

The agent has also come back with the opposite of a plausible wrong answer: a confident alarm, graded FATAL, about a result being scientifically incorrect — and been wrong. One review concluded that the pipeline never removes PCR duplicates, which would make every allele frequency it produces suspect. It does: duplicates are marked and removed during cleaning, several steps before anything is called. Another claimed a distance measure "produces negative eigenvalues", stated flatly; measuring it across realistic population structures showed that it usually does not, and only one particular shape produced a single small one.

Both were the same mistake in a different direction. Code written to answer a biological question carries context that is not local to the lines being read — the duplicate removal is real but it happens in a different file at a different stage, and whether an eigenvalue is negative depends on data the code does not contain. A model reading the code cannot see either. On those occasions the scientist has to know, and hold the position — a FATAL verdict about your own analysis is exactly the moment you are least inclined to argue, and sometimes arguing is correct.

None of which means I am the one who is always right. It means the two of us are wrong about different things, and it is worth being precise about which.

The science is the part I can stand on. Ten years of working on these questions and six (on and off) on this pipeline is what lets me say a proposal is wrong before I can say why, and that judgment has held up. Where I have been wrong is the engineering — repeatedly, and about shape rather than correctness:

• I had a whole "per-window callable-sites" track built into the reporting step, and took it back out the same day. The pipeline was producing an artifact on my guess that somebody would eventually want it, and it was derivable from data already being kept. The measurements behind that are still in the development notes, because they turned out to size a different part of the system. And more importantly it was expensive on the storage.
• I gave the analysis layer a wrapper of its own, so that the main command would not have to take an argument. Then I merged it back and amended the contract instead, because once the shared surface was visible the second executable was plainly the more expensive of the two.
• I added flag-style subcommands for about an hour before reversing them to bare words, for consistency with the rest of the CLI.

Those are architecture calls, and I make them with much less certainty than I make the scientific ones. Each cost time to build and throw away — and this is one of the places the agent genuinely earns its keep, because building the wrong shape fast enough to discover it is wrong is far better than reasoning about the right shape indefinitely. A feature built and reverted in a day is cheap (kind of). The same mistake argued about for a week and then built is not.

So: neither my intuition nor the agent's is the authority. What settles it is the data — build the case that would distinguish the two claims, run it, and read what comes back. Nearly every argument in this project that took more than an hour ended that way rather than in agreement.

A wrong answer that looks wrong costs nothing. You read it, you reject it, you move on. That is not the failure mode here.

What actually arrives is a locally coherent proposal, often with evidence behind it, that violates something the project settled months earlier. The clearest example: for the phenotype-association module the agent proposed a correction for multiple testing across the alleles at a site, argued it carefully, and backed it with a simulation over two hundred thousand replicates showing the type-I error rate was controlled exactly as claimed. The simulation was correct. The proposal was still wrong, because it worked by discarding the reference allele — and the reference allele in a pool-seq cohort is not a privileged one, it is whichever allele happened to be most common when the call set was polarized. Dropping it privileges "not the most common allele" and throws away the row that carries the signal when two alternates rise together.

Nothing in the code or the simulation showed that. It was only visible if you already held the principle. I caught it in three words — this sounds wrong — and it took another day's work to establish what the right answer was.

That pattern repeated. A claim that a particular distance measure "produces negative eigenvalues" turned out to be too strong once measured: it usually does not, and only one realistic population structure produced one. A performance decision was sized against one to five million sites, when the honest planning figure for the genomes people actually use is a hundred million — the decision inverted once that was said out loud. A citation for an effective-sample-size correction was attributed to the wrong source twice before it was right. A statistic was implemented in a form that did not answer the question it was named for, and both versions were being carried at once.

Every one of those is a plausible wrong answer. Someone without the experience to hold the principles firmly ships all of them, and the results look completely ordinary.

That is the thesis, and it is the opposite of the usual claim made for these tools: generating candidate solutions got cheap, so the entire bottleneck moved onto evaluating them — and judgment does not compress. I spend more of my time thinking about this project than I did when I was writing every line myself.

A second, different failure mode

The errors above are about analysis. There is another kind that has nothing to do with judgment and everything to do with reliability, and it needs a different defense.

Over the course of this work the agent has: run the full test suite after I explicitly asked it not to, costing the better part of an hour; committed work I had not reviewed, against a standing instruction not to; and reported a test run as passing when twelve cases had failed, because it piped the output through tail and the tail of a failure summary is a list of case names that looks exactly like a list of passes.

None of those is a wrong analysis. They are an unreliable process, and no amount of domain expertise on my side prevents them — I can only catch them afterwards. Anything that has to hold has to be enforced by a check rather than by an instruction. That is why the rules below are split the way they are: some are for judgment, and those live in prose because judgment cannot be automated; the rest are tests, because prose does not bind.

The rules this produced

Every one of these would improve a project with no AI anywhere near it. What the agent did was make them load-bearing rather than advisable, by supplying a steady stream of the exact mistakes each one prevents.

A comment says what the code does; a decision goes somewhere else. A design decision left in a comment reads to whoever comes next as a current constraint, and gets argued from long after it stopped being true — with an agent, that means yesterday's abandoned choice comes back as an objection to what I am asking for today. The opposite mistake is on record too: a file sat in the output directory of every run from 1.0 onward because the reason a deletion had been placed where it was never got written down, and the deletion later moved. So the test is not "why versus what", it is whether the code becomes inexplicable without the line.

Two things that must agree are checked, never kept in step by hand. The wrapper's file list against what the release archive actually ships; the column definitions in the Python parser against the Groovy that renders them; paths computed in two places from the same values. Each pair has a test that re-derives both sides. A pair kept in step by care drifts silently and produces a wrong result rather than a failure — and where two lists are supposed to differ, the test says so and says why, so the difference is a decision rather than a discovery.

A test's expectations must not come from the code it tests. The obvious way to write a corpus down is to run the thing and record what it said, which produces a test that cannot fail — a changelog with assertions, locking in whatever the code did that day. The depth-detector corpus derives every bound from how each case was constructed instead, and only the generator is committed, so there is no expectation file anyone can quietly edit to make a failing test pass.

Fail loudly, or document — never automate away the decision. Where the right answer depends on something the pipeline cannot know, it reports and stops. Two depth profiles are deliberately left uncapped because they are indistinguishable from a library that simply ran deep; two runs that disagree about how to build a shared reference are refused by name rather than quietly split. And a computed default never removes the knob: everything derived can be set by hand, and a hand-set value still feeds what is computed from it.

Recompute rather than record. Three times, in unrelated parts of the system, the question was whether to write something down for a later stage to read back. Three times the answer was to recompute it, because a record can be edited after the fact and a derivation keyed on the same identity cannot disagree with the run it describes.

Commit only what has been reviewed. Work stays in the working tree until I have read it. The tree is the review surface — one diff, one view — and committing something I have not seen removes my chance to reject it before it becomes history.

What this does not transfer

The output is bounded by the reviewer. Everything above works because I can tell when the answer is wrong, and I can tell because I built the thing it is wrong about. Hand the same tool to someone who could not have written this pipeline themselves and the plausible-but-wrong proposals go straight in, with a simulation attached and nothing to stop them.

So I am not claiming this is a way to build something you do not understand. It is a way to build something you do understand, faster, if you are willing to spend the time saved on reading. If you are not going to read it, do not do this.

I also cannot tell you how much of the code here the agent wrote, and I am suspicious of anyone who gives that number for a project like this. Almost nothing arrived and stayed unchanged. What is here is the result of a long argument, and attributing lines at the end of one is not a meaningful exercise.

The raw material, if you want it

Both of these are in the repository rather than summarized here, because a description of a working method is worth much less than the thing itself.

CLAUDE.md, at the repository root, is the file the agent reads at the start of every session. It is the rules on this page in the form they are actually given — 108 lines, written as instructions rather than as prose about instructions. It is worth reading against this page: the difference between how a rule is explained and how it has to be worded to hold is most of what I learned.

.claude/development-notes/ is the record of how the project got here: 21 notes, about 1,900 lines, covering design churn, alternatives tried and dropped, what things used to be, the measurements behind particular choices, and who decided what and when. A whole feature that was built and then reverted is kept there as a patch rather than deleted, because the sizing work behind the decision to revert it is what shaped the analysis layer.

Each note carries the date it was written and the commit it was written against, and is not updated to follow the code. They are a record of how the project got here, not a second manual — where a note and this manual disagree, the manual is right and the note is history. The one exception says so at the top: the file of platform traps is appended to as they are found, because a trap does not expire.

Both are development material and are kept out of release downloads, so they live in the repository and on the project's page rather than in a tarball.