What METR’s 1.9× estimate says about Anthropic’s AI research

METR’s 11-week assessment of Anthropic, published November 5, 2026, estimates that benchmark-score progress accelerated 1.9× after an October 7, 2025, trend breakpoint. The 12× figure often discussed alongside it is a separate measure: an informal employee self-report, not a benchmark-growth rate. METR’s assessment covers research and development at Anthropic through October 29, 2026.

METR’s assessment addresses a different question from the earlier S-1/A warning about AI automating research and engineering that NeoTeo covered previously. METR’s estimate concerns measured progress on specified benchmark tasks.

What METR’s 1.9× estimate measures

METR’s Anthropic Capability and Efficiency Index (AECI) combines scores from internal and external benchmarks. Its main model estimates that benchmark-score growth after October 7, 2025, was 1.9 times the rate before that breakpoint. METR gives a 95% confidence interval of 1.5× to 2.8×.

That estimate depends in part on how the model translates benchmark difficulty into scores. Across alternative link functions, METR reports estimates from 1.1× to 2.9×. Other checks that vary benchmark or model-snapshot selection generally fall around 1.9× to 2.2×. These are sensitivity results under different assumptions, not a single measurement with the same conditions.

To understand a claim about AI-led research, first ask whether its number tracks benchmark progress, a person’s reported speedup, or output volume.

Why the 12× figure is different

METR says the approximately 12× figure is the average employee-reported uplift in October 2026 for Anthropic’s most capable internal model. It came from informal Slack polls, which METR describes as very loose evidence. The figure reflects employees’ reported individual uplift; it does not measure the rate at which benchmark scores improved.

AI assisted the research; autonomy was not the main contribution

METR’s judgment was that AI use likely accounted for a majority, perhaps about 80%, of the acceleration observed by October 2026. METR attached substantial uncertainty to that estimate and said non-AI factors likely contributed to the acceleration in late 2025.

The assessment describes AI as mainly assisting researchers. METR said fully autonomous contributions were not yet highly significant in the most important model-training work, while researchers saw early signs that such contributions could grow, with progress varying by research area. That account describes AI support for research, not a finding that AI systems were already independently developing Anthropic’s successor models.

Dave Lee’s commentary discusses METR’s assessment, its methodological caveats, and the public report’s redactions.

Anthropic’s coding figures track a different kind of output

Anthropic says Claude authored more than 80% of the code merged into its codebase as of May 2026. The company also reports that the typical engineer merged eight times as much code per day in the second quarter of 2026 as in 2024. These are company-reported coding measures, separate from METR’s benchmark-growth estimate. Anthropic cautions that lines of code measure quantity rather than quality and may overstate productivity gains. Anthropic’s account of AI in research and engineering

What the public assessment leaves readers to judge

METR said Anthropic redacted a majority of the evidence METR considered most informative in the public report. METR also said it found convincing evidence that the measured acceleration did not result from faster growth in training-compute scale; it redacted the supporting details. The public report therefore presents METR’s conclusions alongside less of the underlying evidence than the evaluator considered most useful.