Cheaper AI reviews missed the bugs that mattered

I wanted to spend less on AI-assisted development. Code review seemed like a reasonable place to save. Let a cheaper model check the work and bring in a stronger one only when needed.

The cheaper reviews missed real bugs. One repair even brought a failure back. I ended up choosing GPT-6 Astra for every part of the process.

My review-fix workflow asks AI to check a proposed code change, fix important bugs, and verify the repairs. A lead agent coordinates other agents, which are separate AI sessions assigned to reviewing or fixing the code. I wanted to compare the finished work, not just how convincing the review sounded.

From September 5 to 7, 2026, I tested several setups across nine code changes in two projects. The most revealing round used two changes, each starting from the same code and instructions across four setups:

Astra found and repaired three important bugs that the cheaper setups did not independently catch. These affected normal, intended behavior, not just unlikely edge cases or coding preferences.

Two bugs affected an unpublished internal tool. The third affected Rota, my clothing-tracking app. After an error, it could get stuck repeatedly trying to reload data instead of stopping.

A browser test recorded 2,494 calls to its data-loading function in five seconds. That demonstrates a runaway loop, not 2,494 paid AI requests or a measured production bill.

Luna introduced a different problem while making a repair. Its change allowed server errors to look like successful responses again, bringing back the kind of failure the original change was meant to remove. I checked this separately against the actual framework. The repair stayed local and was never shipped.

Earlier rounds also produced unnecessary code and tests that looked more reassuring than they were. One test still passed after the behavior it was supposed to protect had been deliberately broken. More findings, more code, and more passing tests did not reliably mean a better result.

The obvious compromise was to put Astra in charge while cheaper Luna agents did the reviewing and fixing. Astra was better at rejecting weak findings and avoiding unnecessary work. But this mixed setup still missed both bugs in the internal tool.

Better judgment about reported problems did not recover the problems nobody reported.

The mixed setup did contribute a useful Rota fix that Astra missed. Spark also contributed a worthwhile test improvement. I kept both. The result was not that Astra found everything or the cheaper models did nothing useful.

One apparent mixed-model success could not count as independent evidence. Its reviewer had read Astra's completed fix before reporting the same loop. That exposed a weakness in my experiment setup, not another independent discovery.

I estimated what the two latest tasks would cost at the model prices recorded for the study. These are estimates from recorded usage, not actual bills or measurements of my subscription allowance.

The mixed total is a minimum because some usage could not be counted. Even the recorded portion cost more than using Astra throughout. Mixing models did not deliver the saving I had hoped for in these runs.

Spark and Luna were much cheaper, but their results were not equivalent. Luna's internal-tool review was incomplete, and its Rota repair contained the reproduced failure. Spark missed the main bugs.

The cheapest review was not the cheapest successful result. My supervision time and the cost of missed bugs are not included in these figures. I cannot put an exact dollar value on avoiding them, but they are part of what I am choosing to pay for.

I will use Astra at medium effort for the lead agent and every reviewer and repair agent. Medium is my chosen balance, not a proven optimum. I did not compare Astra's effort settings against each other.

I also do not want to ask before every review whether this change is complicated enough for Astra. That is another decision between me and getting work done.

“Use the cheap model and switch when it fails” sounds sensible. But a review that misses a bug can still return a confident report. There may be no warning that I need a stronger review. If I need Astra to check whether I can trust the cheap review, I have not replaced Astra.

This was a small study of real work, not a universal model ranking. The tools and conditions varied, and Astra missed a useful fix too. Those limits matter. They do not erase the demonstrated bugs, the broken repair, or the mixed setup's failure to save money.

I wanted to save money on reviews. I decided I would rather pay for the setup that caught the problems I needed it to catch, without choosing a model all over again each time.

The optional methods and cost notes include the accounting and limitations. Unpublished project details are omitted, so this is not a fully public replication package.