Back to blog

Engineering

I asked an AI reviewer to stop when the code was clean. It ran for 46 hours.

A 179-line change became 37,451 lines because reviewer silence was an impossible stopping condition.

·7 min read

Forty-six hours later

A 179-line change had become 37,451 added lines.

On July 13, 2026, I opened a small change to my engineering standards repository. It added two controls to a command-line tool: choose which version of the standards to use, and opt out of scheduled updates. The first draft touched seven files.

I handed that change to an experimental review-loop workflow. Five reviewers inspected the full pull request. Workers fixed every accepted finding. Then five fresh reviewers inspected the enlarged pull request. The workflow would stop after two consecutive passes found nothing.

01

Review the entire current diff

02

Repair every accepted finding

03

Add the repairs to the diff

↳ NEXT PASS

Send the enlarged diff back to step 01. A clean pass counted as 1 / 2, then the whole diff went around once more.

Review-loop repeatedly reviewed its own repairs. Only two consecutive clean passes could end the run.

Original change

179 added lines · 7 files

Force-stopped branch

37,451 added lines · 350 files

39full review passes

~46 hbefore I stopped it

0 / 2clean-pass counter

The process multiplied the added lines by about 209 and still could not satisfy its stopping rule.

The clean-pass counter never moved. I stopped the run during pass 39. The branch now touched 350 files across 70 commits. It had spent roughly 46 hours trying to prove that no reviewer could find one more thing.

The complete record

Pull request #28 contains every review pass and the closing report. The final branch and its post-mortem remain archived as the failed experiment.

The workflow began reviewing itself

Roughly 260 findings concerned code written during the run.

The reviewers confirmed 289 findings. The workers fixed 286. The workflow discarded zero. My later audit estimated that the original change contained about nine to eleven defects, all found in the first pass. Almost everything after that came from the repairs.

Many findings were real. Their fixes still had no place in this pull request. The workflow treated reproducible as mandatory and never asked whether a repair matched the feature's actual risk. Every fix gave the next reviewers more code to inspect.

Pass 17 is where the scale became indefensible. One finding described a rare failure while cleaning up copied files. The repair added about 9,575 lines across 74 files. A tool that copies standards files now had crash recovery machinery closer to a database than a file copy.

The later passes became stranger. The workflow built release automation, repository administration, filesystem identity checks, and custom code scanners. By pass 39, it added a parser whose job was policing scanners written one pass earlier. The process had become its own customer.

It escaped the pull request

The branch changed live repository settings and broke unrelated CI.

One repair applied the branch's rewritten GitHub settings to the live repository. It removed the declared protection for the main branch, added an undeclared release-tag rule, and created a deployment environment. The repository's configuration check then failed on every branch, including pull requests unrelated to the experiment.

I restored the settings after stopping the run. The pull request stayed closed and unmerged. This was the clearest cost of the experiment. The workflow had moved beyond producing too much code and changed the system it was supposed to protect.

Silence was the wrong finish line

A capable reviewer can always descend one level deeper.

Earlier versions of the workflow had finished in a few passes. Their repairs became smaller each time until reviewers ran out of material findings. I tightened the stopping rule after a stronger model found 17 valid issues in a change that a weaker model had declared clean.

Requiring two clean full reviews sounded safer. On this change, each repair round added code faster than the reviewers could clear it. The stronger reviewers kept finding defects because the workflow kept writing defects. Silence moved farther away after every pass.

The workflow also lacked a pass limit, a time limit, and an alarm for explosive diff growth. I had given it blanket approval to continue, which disabled the last human interruption point. Those decisions made a 46-hour run possible.

Seven files survived

The useful change fit into 217 added lines.

I reopened the original feature and its first-pass hardening as pull request #32. It touched seven files, added 217 lines, passed its checks, and merged. The other 37,000 lines stayed in the museum branch.

I retired review-loop in pull request #33 and replaced it with review-fix. The replacement reviews once, repairs material findings, checks those repairs, and stops. A person receives the remaining risk and decides whether to merge.

That bounded workflow later found fifteen material problems across two real changes. CodeRabbit found zero valid problems on the same original and repaired snapshots. The public comparison shows what happened after the failed experiment became a usable skill.


See the bounded workflow at work

The follow-up experiment compared review-fix with CodeRabbit on two real changes.

Read the comparison