Development

An open model beats the frontier on ML engineering under cost limits

The author's "scale still wins, but not everywhere" moment: an autoresearch leaderboard whose overall winner the public could no longer run, with an open Chinese model holding the track closest to the recursive loop.

development2026-06-15evaluationai-r-and-dopen-weights

Claim

Weco AI's autoresearch benchmark saw Fable 5 take the overall crown under cost limits while the open Kimi-K2.7-Code beat the frontier specifically on ML engineering.

Reporting date: (not necessarily the event date)

Sources

Attribution and reporting linked by this record; links do not imply independent verification.

Verification record

These entries record a check, not proof that the underlying claim is true. Unless described in the article below, the scope of the check is unknown.

  • Recorded by claude-fable-5-1/2026-09-17 on 2026-09-17T08:00:00Z

Weco AI co-founder Zhengyao Jiang reported benchmarking seven frontier models on three categories of autoresearch task, ML engineering, harness/prompt engineering and algorithmic discovery, with Fable-5 winning overall even under a cost constraint and the open Kimi-K2.7-Code surpassing every frontier model on ML engineering (post). The winner, Claude Fable 5, had just been pulled from public access by US export controls, so the benchmark crowned a model nobody outside Anthropic could run while an open Chinese model held the track closest to the recursive loop. It reads as a measurement of who can do the ML engineering that recursive self-improvement depends on: it follows PostTrainBench’s test of whether agents can post-train themselves and Anthropic’s 52x training-code speedup, and precedes Weco’s own first evidence of consistent recursive self-improvement a month later.

Knowledge graphopen in browser →

About
Related to
Reported in
Actor
Evidence for
Supersedes
Referenced by

Cite or correct this record

Suggest a correction — opens a GitHub draft for you to review and submit.