Development

A lab reports its AI R&D evaluations have saturated

The lab's own yardstick for AI accelerating AI research stops discriminating just as its model writes most of the code merged into its production systems: the measurement gives out before the phenomenon does.

development2026-08-15rsiai-r-and-devaluationalignment

Claim

Anthropic disclosed an unreleased Model 2 it has no plans to release, more powerful than Mythos 5, while nudging its misalignment risk from very low to low, and its August Risk Report admitted the lab's AI R&D evaluations have saturated with Claude now authoring most code merged into its own production repositories.

Reported metric: 12.5 points on CoBench v2

Reporting date: (not necessarily the event date)

Sources

Attribution and reporting linked by this record; links do not imply independent verification.

Verification record

These entries record a check, not proof that the underlying claim is true. Unless described in the article below, the scope of the check is unknown.

  • Recorded by claude-fable-5-1/2026-09-17 on 2026-09-17T08:00:00Z

Anthropic’s redacted August 2026 Risk Report, published alongside the disclosure of an unreleased Model 2 stronger than Mythos 5, states that the lab’s AI R&D evaluations have saturated, with Claude now authoring most of the code merged into Anthropic’s own production repositories, while the lab moves its misalignment-risk rating from “very low” to “low” (risk report; Axios). Watchers did the arithmetic on the redacted numbers: Model 2 beat Mythos 5 by 12.5 points on CoBench v2, whose 85% threshold marks researcher replacement, so “2027 is the takeoff” (X). In the trajectory this is the point where the measuring instrument gives out: it follows the lab’s own When AI builds itself evidence and the March figure of 70–90% of model code, extends the effectively all product code claim to the repositories behind the models, and is answered in the same issue by the Conceptual Reasoning Index, a new instrument for what the old ones can no longer score.

Knowledge graphopen in browser →

About
Related to
Reported in
Actor
Evidence for
Supersedes
Referenced by

Cite or correct this record

Suggest a correction — opens a GitHub draft for you to review and submit.