GPT-5.6 Luna vs GPT-6 Astra. A $1.20 model catches 75% of bugs.

Twenty-three bugs. That's the gap between GPT-5.6 Luna and GPT-6 Astra in a benchmark posted to r/artificial this week, run against 50 real pull requests pulled from actual repos. Astra confirmed 92 bugs. Luna confirmed 69. The assumption going into a test like this is usually that a cheaper model will lose a little accuracy and save a lot of money, and that the trade is obviously worth it. Then you look at the list of what Luna missed, and the calculus stops feeling so clean.

The poster ran a full eval breakdown, not just a top-line score: 50 PRs, two models, confirmed bugs counted by hand rather than inferred from a rubric. Luna caught 75 percent of what Astra caught, at roughly 3.6 percent of the cost, according to the numbers in the thread. If you stop reading there, the story is "cheap model is good enough." Most of the interesting part of this benchmark is in what happens when you don't stop reading there.

The 23 bugs that don't average out

A 75 percent catch rate sounds like a passing grade. It isn't, because bugs in a code review context are not evenly distributed across difficulty. In the thread's breakdown, the misses cluster: Luna was fine on obvious stuff, null checks, off-by-one errors, missing await calls, the kind of thing a linter half catches anyway. Where it fell off was on bugs that required tracing state across two or three files, or reasoning about a race condition that only shows up under specific timing. Those are exactly the bugs a code reviewer exists to catch, because they are the ones a human skimming a diff on a Friday afternoon is also going to miss.

This is the part a single aggregate percentage hides. A 75 percent catch rate on easy bugs and a 75 percent catch rate spread evenly across easy and hard bugs are two very different products, even if the top-line number is identical. The Reddit breakdown suggests it's closer to the first case: Luna's misses were disproportionately the expensive kind. That matters because the cost of a missed easy bug is a five-minute fix in code review. The cost of a missed hard bug is a production incident three weeks later that nobody can trace back to the PR that introduced it.

None of this means Luna is a bad model. GPT-5.6, the model family it's part of, launched specifically as a lower-cost option in OpenAI's lineup, and being 25 percent behind a model priced many times higher on a task this specific is a respectable showing. What it means is that "good enough" isn't a property of the model. It's a property of what you're using it for and how expensive your worst-case miss is.

The case for shipping the cheaper model anyway

Here's the argument against everything the section above just implied, and it deserves a full hearing rather than a caveat. Code review at most engineering shops is not a single high-stakes gate. It's a volume problem. You have hundreds of PRs a week, a review queue that's always backed up, and a human reviewer who is already skimming half of what comes through. In that world, a model that catches 75 percent of bugs at 3.6 percent of the cost isn't a compromise. It's a multiplier. Run Luna as the first pass on every single PR, and it becomes practical to run Astra, or a human, or both, as a second pass only on the subset of PRs that Luna flags as risky or that touch sensitive code paths. You get more total coverage for less total spend, because you stopped treating "one model reviews everything" as the only architecture.

There's also a version of this argument that's less about architecture and more about honesty regarding the counterfactual. The comparison in the thread is Luna versus Astra. It is not Luna versus nothing. Most PRs, at most companies, get a human review and nothing else. A human reviewer under deadline pressure, reviewing their fifth PR of the day, is not catching 92 out of 92 bugs either. Nobody ran that benchmark, because it's harder to construct, but it's the comparison that actually matters if the alternative to Luna isn't Astra, it's a tired engineer approving a diff at 6pm on a Thursday.

And the cost gap is not a rounding difference. If Astra runs at something like $10 in and $50 out per million tokens, per OpenAI's published rate, a 3.6 percent cost ratio for Luna means the difference between reviewing every PR your org generates in a month and reviewing a fraction of them. For a team that's currently doing zero automated review because the budget line for "run the expensive model on everything" never cleared finance, Luna at this price is not a downgrade. It's the thing that gets automated code review approved at all.

A team lead and a cost-conscious CTO argue about the eval

CTO: 3.6 percent of the cost for 75 percent of the bugs. That's the easiest infrastructure decision I've made all quarter.

Team lead: Which 25 percent, though? The eval says Luna's misses skew toward the harder bugs. Those are the ones that cost us a Saturday on-call page, not a code review comment.

CTO: So we run Luna on everything and Astra on anything touching payments, auth, or the scheduler.

Team lead: That's a reasonable policy. But somebody has to write the rule for what counts as "sensitive," and somebody has to enforce it when a PR touching the scheduler gets mislabeled as a config change.

CTO: Fair. Still cheaper than running Astra on all 50 PRs a week we're currently reviewing for free with tired engineers.

Team lead: Agreed. I just don't want "good enough" to quietly become "the only line of defense" six months from now when nobody remembers we picked Luna for cost reasons.

Run the same 50-PR test before you pick a default

The number that should travel with you out of this thread isn't 75 percent or 3.6 percent. It's the fact that the gap between two models on the same task can be measured on your own repos, with your own bug history, in an afternoon. The r/artificial post is a useful data point precisely because it's specific: 50 real PRs, confirmed bugs rather than estimated ones, a transparent breakdown rather than a marketing chart. That's a template, not a verdict, because bug distribution in your codebase is not the same as bug distribution in whatever repos the original poster pulled from.

If you're deciding between a cheap review model and an expensive one for your own pipeline, the earliest you could act on this productively is your next sprint planning cycle, because that's when you'll have a natural batch of 30 to 50 merged PRs with known outcomes to test against. Pull the ones where a bug slipped through review in the last quarter, run both models against them, and see whether the misses cluster the same way they did in this thread. If they do, tier your review process instead of picking one model for everything. If they don't, you've learned something the aggregate percentage never would have told you.

For more on how coding agents perform once the benchmark numbers meet an actual repo, see our look at code search agent claims and our breakdown of LLM benchmark hype versus practice. If you're comparing review tooling directly, Cursor vs GitHub Copilot covers a related tradeoff between speed and thoroughness. For the models themselves, ChatGPT and Claude Code both remain common baselines teams test against before standardizing. The original thread with the full eval breakdown is worth reading in full on r/artificial, and for a sense of how the field measures coding tasks more broadly, SWE-bench is the closest thing to a standard reference.