How PoolSeqFlow Was Built¶
PoolSeqFlow was written by one person over six years, and an AI coding agent has been involved for the most recent few months of that. Both halves of that sentence matter, and this page gives the dates so neither has to be taken on trust.
The three phases¶
| When | What | Where it lives |
|---|---|---|
| 2020–2022 | The original pipeline and the false-positive filtering approach, written in bash during my PhD studies. The pool-seq reasoning — what a pool's size does to its detection limit, why a fixed reference is not a privileged one, which filters a real dataset needs — was settled here. | Not in this repository |
| 2025–2026 | The rewrite into Nextflow: the step model, the resume logic, the parameter system. Published as v1.0.0 and v1.0.1. The rewrite was needed because I was asked to look at some analysis for some of my collaborators and I decided to redo their variant calling. Without an access to a personal cluster, I needed proper parallelization, for which Nextflow was the perfect vehicle. | This repository, from v1.0.0 |
| From v2.0.0, August 2026 | The v2.x series and v3.0.0, developed with Claude Code: multi-run sharing, storage tiering, the metadata file, the analysis layer and its modules. | This repository |
The repository is younger than the tool¶
This is worth stating plainly, because the obvious way to judge a project's history is to read its commit log, and here that gives the wrong answer.
I have done little work after the first work was completed around June 2025. In around March 2026, I decided to package it and put it on my GitHub. That is the reason that the first commit is dated 2026-03-13. v1.0.0 was tagged thirteen days later, on 2026-03-26, with 26 files in the tree. That is pretty much the "end" of the development of the main pipeline. A working pipeline existed before this repository did; what happened in March was that it was put under version control in order to release it. Everything in the first two phases above therefore has no commits behind it at all.
The distribution compounds the impression. Eight commits reach v1.0.1; every other commit in the repository comes after it, and the great majority of those fall in a single month, August 2026. The fast development in August was also a necessity rather than just AI enabled development. During August 2026 I spent around 200 hours on the development to help a colleague use this tool and analyze data in September 2026. That is where I made all engineering decisions rather than scientific ones to make the tool accessible by a larger base of scientists. So the log reads as a project built almost entirely in one month — and that is the month the agent arrived. What it actually shows is when the work started being recorded at this granularity, not when it was done. The tree tells the same story from the other side: 26 files at v1.0.1 and many times that today, almost all of the growth being the analysis layer, the test suite and this manual rather than the pipeline itself.
What the rewrite changed, and what it kept¶
My original pipeline, written in bash between 2020 and 2022, had the main frame of the pipeline as it is today. It also included clipping from the ends during trimming (though not autodetected by the script), the basics of parallelization, basic automated RG tag handling, false-positive read filtering, and the logic of tool-by-tool core assignment according to tool benchmarks.
When I decided to analyze new data in 2025 I had more experience in automation, so I rewrote the parts that mattered most.
- Writing the pipeline in Nextflow, to improve its reproducibility.
- Auto-detection of how much to clip from each end of each sample, from the FastQC output. The common practice is to look at a few FastQC reports after trimming, decide on one number, and clip that much from every sample. I wanted a more robust and automated version of that, so that most of the valuable sequencing data is retained, with confidence.
- Better multi-allele handling, and major-allele reassignment moved inside the pipeline. I had been doing that step during downstream analysis.
- A much more stringent cleaning step: everything but properly paired reads is eliminated. It costs a little depth, and I think that is a reasonable trade where allele frequencies matter.
I kept a lot of it as it was.
- The clipping logic that keeps the A/T and G/C ratios near 1 after trimming, since those can drift once adapters are removed.
- The alignment and cleaning protocol that prepares BAM files for variant calling — though I replaced all the GATK tools with samtools.
- The variant calling protocol.
- The protocol for obtaining allele frequencies from VCF files.
The releases¶
| Version | Tagged | What it was |
|---|---|---|
| v1.0.0 | 2026-03-26 | The first published release of the Nextflow pipeline |
| v1.0.1 | 2026-06-02 | Fixes |
| v2.0.0, v2.0.1 | 2026-08-12 | Nextflow 26 and Trim Galore 2.x; parameters.config becomes yours rather than tracked |
| v2.1.0, v2.1.1 | 2026-08-15 | migrate_config; real CPU requests per process; the first change guards |
| v2.2.0 | 2026-08-16 | vcffilter.minDP begins filtering; the documentation site |
| v3.0.0 | 2026-09-10 | The current release. The analysis layer, multi-run sharing, storage tiering |
v1.0.1 is the last release written without an agent; v2.0.0 is the first written with one. That line is drawn at a tag rather than a date so it can be checked: git diff v1.0.1..v2.0.0 is where the working method described in this section begins.
Each release is archived on Zenodo and has a DOI of its own; see Citation & License for which one to cite.
Why an agent was brought in¶
The decision was about accessibility, not about writing code faster. A tool that only its author can install, configure and interpret is not a tool anyone else can use, and most of what v2.x and v3.0 added — the verification step that explains what it is checking, the migration path between releases, the preview, the manual this page is part of — is that problem rather than the science.
It was also a considered decision rather than an experiment. At this point on a completely different side project I was using Claude Code on a larger codebase, which is where I realized that I could implement guardrails to keep pipeline's scientific reasoning and integrity intact.
During the time I was working on this side project, I realized that using an agent I was able to speed up some mundane tasks like updating package bases, and making sure that a tool was compatible in multiple enviorenments. This was the other reason that contributed to my decision. Because two major tools in this pipeline, Nextflow and TrimGalore made major version releases and changes to how they worked between March 2026 and July 2026. I wanted to keep the tool up to date, but the maintenance cost of the tool was becoming higher for a single developer alone.
What the commit record does and does not show¶
It does not mark where the assisted work begins. Two commits carry a Co-Authored-By trailer, both from September 2026, because the convention was adopted late rather than at the start. The trailer marks two commits; it does not draw the boundary. The boundary is v2.0.0, stated above and in the release table, and Involvement of Claude Code says what the work on either side of it actually looked like.
What the trailer marks is the commit, not the code — which is the opposite of how it reads. Work is left uncommitted until I have reviewed it in the working tree, and the commits are mine: since v2.0.0 the median commit changes three files. Those two commits cover a change of 47 and 22 files, with about 3,300 insertions each and messages of 39 and 41 lines where one line is normal. The changes in them had been reviewed as they landed, like every other change. What was done with the agent was the packaging — deciding to separate the additions into two commits rather than one, and writing them out in that much detail. So the trailer records help composing a commit, and a reader who takes it as marking AI-written code has it backwards.
AI generated text and responsibility¶
This documentation contains AI generated text and a lot of it (This section is not one of them). Every piece of text that is in this project came to life as a result of hours of discussion with the agent and after a lot of corrections by me. What this project does not contain is unreviewed text. I have reviewed every sentence heavily, edited them and made sure each statement was true and accurate. So the responsibility of all text is solely on me. I believe this is an important point in the AI use, we cannot abdicate responsibility of our words and actions to the tools we use.
I discuss the academic publishing system, and why this project is not being published in academic journals, in Why a Living Repository. Another important point is that the same system is also stigmatizing the AI use today. However, as a person who grew up with a completely different language, I can see how AI can remove language barriers for researchers from all backgrounds and democratize scientific research. To be clear, by any means, I am not saying AI business model is something I support with no reservations. I keep my reservations on the developments and I want to remain clear eyed about it. But as a tool, generative AI has potentials to unlock doors to so many skilled people who have been denied opportunity because of language barriers, and I believe this includes neurodivergent people too.