AI Coding Models Got Faster. Your Review Queue May Not.

On Monday, the engineering lead celebrates a record number of AI-assisted pull requests. On Thursday, the release is late. The review queue is longer, the largest changes are harder to understand, and two test failures need a senior engineer. The model worked faster. The system did not. That is the measurement problem behind the current wave of stronger coding models.

What is new this month

OpenAI’s September GPT-6 Astra release reports stronger performance on professional and computer-use tasks. Anthropic’s September 22 Opus 5.5 announcement reports stronger coding and lower cost per task than Opus 5 in its tests. A quoted Deloitte evaluation in Anthropic’s release says Opus 5.5 caught 72% of known bugs at its lowest effort setting versus 56% for Opus 5 at high effort; that is a customer-reported result inside a vendor announcement, not an independent result for every repository. Separately, GitHub says its Copilot code review default will move from Lite to Balanced no earlier than September 28, with organizations able to set their own default.

These changes make it reasonable to retest a coding workflow. They also make it easier to increase output without increasing review capacity. The procurement question is not “Which model writes more code?” It is “Which configuration moves a safe change from request to production with less total effort?”

Use the merge-to-recovery ledger

Track one cohort of comparable work from issue accepted to production and, when necessary, recovery. Record developer time, model expense, reviewer time, number of review cycles, tests added or repaired, time waiting for approval, change lead time, and post-deployment incidents. Put the same ledger on the previous workflow. If the model saves an hour of drafting but adds two hours of review and a rollback, the economic result is negative even when the generated code looks good.

DORA’s software-delivery measures provide an outcome frame: change lead time and deployment frequency for throughput, plus change fail rate and deployment rework rate for instability; failed-deployment recovery time is also tracked. These measures should be read for a single application or service, not averaged into a company-wide trophy. AI-specific reviewer load and cost per merged change belong beside them.

• Accepted change: Did the pull request meet the task and merge without hidden scope expansion?
• Reviewer burden: How many minutes and rounds did a human spend understanding and correcting it?
• Test quality: Did the change add relevant tests and pass the existing regression suite?
• Production result: Did the deployment create a failure, rollback, or unplanned repair?
• Unit cost: What did model use, developer time, review, compute, and recovery cost per accepted change?

Run three tests, not one coding benchmark

First, replay a known bug with a reliable test and known root cause. Second, ask for a small feature across several files with an explicit acceptance test. Third, include an ambiguous request where the right behavior is to ask a question before editing. Use the same repository snapshot, instructions, permissions, effort settings, and reviewer for every candidate. Judge the complete change and explanation, not the first patch.

Code review assistance deserves its own test. GitHub’s documentation says Copilot reviews are comments rather than required approvals by default, although configuration can change that. If an organization chooses a different approval role, that is a governance decision, not a proof of correctness. Before GitHub’s announced default-effort change, record review volume, false alarms, missed bugs, and credits or costs so that an altered setting is not mistaken for a sudden change in engineer quality.

Where the system faceplants

A model can fix the visible failing test while changing an unrelated interface. It can add a broad test that passes but does not guard the edge case. It can produce a long, polished explanation that hides the one assumption a reviewer needed to challenge. Higher output volume then saturates the approval queue. GitHub’s own engineering discussion warns that short individual tool outputs are not necessarily efficient if they force more calls later. Cost should be measured at the task, not at the message.

For a four-week pilot, cap the number and size of AI-authored changes, keep existing human review, and sample merged changes for a second independent check. Stop if the change-fail rate, review backlog, or severity of escaped defects rises beyond a threshold agreed before the trial. If accepted changes move faster with stable quality and lower total cost, expand. If only draft volume rises, the model has found your next bottleneck.

Primary sources

• OpenAI: GPT-6 Astra — September model release and provider-reported evaluations.
• Anthropic: Claude Opus 5.5 — September coding and efficiency claims, including the labeled Deloitte evaluation.
• GitHub: upcoming Copilot policy and billing changes — September 28 or later default code-review effort change.
• GitHub: Copilot code review documentation — How reviews and approvals work.
• DORA: software delivery metrics — Throughput and instability definitions.
• GitHub: coding-task efficiency — Why task outcomes matter more than individual token or tool counts.

Put this to work

• AI in the software development lifecycle — Put model assistance inside a controlled delivery path.
• AI ROI decision lab — Price accepted work and extra reviewer time.