Most teams can show that developers are using AI, but far fewer can show that it reduces the total effort required to deliver working software.

A change may be drafted in half the usual time while review, correction and production support consume the saving, which means coding speed alone says little about actual delivery capacity.

A real productivity gain exists when more accepted work reaches production for the same total engineering effort, without a corresponding increase in failures, rework or maintenance.

Measuring it requires a combined view of delivery speed, human effort and production results rather than a count of prompts, generated lines or pull requests.

Why AI usage does not prove productivity

AI development dashboards usually report active users, suggestion acceptance, generated code and pull-request activity. These figures indicate whether a tool is being used and whether developers find some of its output relevant, but they do not show how much work survived review or what happened after release.

GitHub’s Copilot usage metrics make this distinction explicit by separating adoption, engagement and code generation from pull-request lifecycle measures. Even merge counts and median time to merge remain incomplete because they do not reveal how much reviewer attention was required or whether the release later needed corrective work.

Research shows why the distinction matters. DORA found that a 25% increase in AI adoption was associated with a 1.5% reduction in delivery throughput and a 7.2% reduction in delivery stability. The researchers connected the result partly to larger batches of generated code, which can take longer to review and carry more release risk.

METR found a similar gap between perception and observed results. In an early-2025 randomized study, experienced open-source developers took 19% longer when they were allowed to use AI, despite expecting to be faster. A 2026 follow-up found some evidence of gains from newer tools, but the researchers concluded that selection effects made the size of those gains difficult to estimate reliably.

The effect therefore depends on the task, developer, codebase and delivery process. Companies need evidence from their own work rather than a general percentage supplied by a vendor or benchmark.

Use accepted production work as the unit

Generated code has no independent business value. A more useful unit is an accepted production change, which might be a deployed pull request, a completed customer requirement, a resolved production defect or a migration step that has passed the team’s normal engineering and release requirements.

The input should include all engineering work needed to deliver that result. Development time, review, corrections, failed checks and attributable production repairs consume capacity even when they are performed by different people.

Productivity = accepted production output divided by total engineering effort.

A 1.5x productivity gain means 50% more accepted output for the same effort, or the same output with approximately 33% less effort. A 3x gain means three times the accepted output, or the same output with approximately 67% less effort.

Consider a change that previously required ten hours of development, two hours of review and one hour of correction, giving a total of thirteen hours. If AI reduces development to five hours but review rises to four hours and correction to two, the new total is eleven hours. The draft appeared twice as fast, while the actual productivity gain was approximately 18%.

The six numbers worth tracking

A useful management view can be built around six measures that connect AI activity with accepted delivery.

1. Accepted output per total engineering hour
   Count completed production changes against all development, review and correction time required to deliver them. This is the closest direct measure of whether capacity has increased.
   
2. Total lead time to production
   Measure the time between starting a change and deploying it, including waiting, review, failed checks and release. If drafting becomes faster while total lead time remains unchanged, the main constraint lies elsewhere.
   
3. Reviewer effort per accepted change
   Record human review time, not only the elapsed period between review request and approval. A pull request approved within an hour can still be expensive if a senior developer spends most of that hour reconstructing its logic.
   
4. Substantial rework rate
   Track the proportion of AI-assisted changes that require major rewriting, are abandoned before merge or are changed again shortly after release. Suggestion acceptance matters far less than how much of the accepted work survives.
   
5. Delivery stability
   Monitor change failures, rollbacks, hotfixes and unplanned releases needed to repair earlier deployments. Additional output cannot be treated as a gain if it creates proportionate production work.
   
6. Cost per accepted production change
   Combine engineering time, reviewer time, AI costs and attributable repair work, then divide the total by accepted changes delivered. Token spending becomes meaningful only when connected to this result.

Review and rework determine whether the gain survives

Review is often the largest hidden cost because repository data records when a review starts and ends, but not how much attention it requires. Generated code can compile and follow a visible pattern while still misunderstanding permissions, business rules or operational exceptions.

Teams should estimate reviewer effort for a representative sample of changes and record whether corrections concern minor implementation details or substantial problems with logic and architecture. The purpose is not to monitor individuals, but to determine whether faster drafting is moving more work onto scarce senior reviewers.

This is particularly important in mature systems, where a plausible implementation may still conflict with rules that are not obvious from the surrounding code. The issues covered in what code review must catch when AI helped write a change belong inside the productivity calculation because every missed assumption consumes reviewer time or reaches production.

Rework should also include abandoned attempts and code changed again within the following 30, 60 or 90 days. A change that passes automated checks but is difficult to maintain has not delivered the same value as one that another developer can safely understand and modify.

The quality of the original task affects these results. As discussed in why AI coding gets expensive when context is badly prepared, weak instructions create extra generation, review and correction loops. Measuring rework shows whether better context is reducing those loops.

Compare equivalent work over time

A useful baseline can be established with four to six weeks of data from one or two repositories, followed by eight to twelve weeks of AI-assisted work measured in the same way. Tasks should be grouped into broad categories because repetitive mappings, test preparation and complex business-logic changes are unlikely to show the same result.

Medians and ranges are more informative than averages alone because a single incident or large migration can distort the figures. Changes in staffing, project complexity and release schedules should also be noted before attributing the result to AI.

The comparison should remain at repository or team level. Once developers are judged by generated lines, pull-request volume or suggestion acceptance, those figures become targets that can be increased without delivering more value.

How to interpret the result

When accepted output rises while total effort and production stability remain unchanged, AI has added real capacity.

When drafting becomes faster but total lead time does not move, the team has accelerated a stage that was not limiting delivery.

Higher output accompanied by more review, rework or failed releases usually means that part of the cost has moved downstream.

If delivery speed remains unchanged but developers spend less time on repetitive work or produce better documentation, AI may still provide value, although that result should not be presented as increased delivery capacity.

At Blocshop, our senior developers typically consume around $300 per month in AI tokens while the productivity gains observed across suitable tasks and codebases range from approximately 1.5x to 3x.

These figures come from a senior-led process in which AI-assisted work remains subject to normal architecture, review, testing and release requirements, with the gain judged by accepted delivery rather than generated volume.

For teams already using AI but unable to determine whether the improvement survives review and production, Blocshop can assess the current workflow, establish a baseline and identify where AI is adding capacity and where it is only moving the cost.