A counterweight to the 35-hour autonomous runs: duration is not the same as coherence.
Development
Frontier agents resolve under a fifth of a multi-hour task marathon
Claim
The new SWE-Marathon benchmark strings together twenty multi-hour engineering tasks, and even frontier models resolve under 19%, losing the thread over hours.
Reported metric: <19%
Reporting date: (not necessarily the event date)
Sources
Attribution and reporting linked by this record; links do not imply independent verification.
- Welcome to June 7, 2026 (theinnermostloop.substack.com)
Cite or correct this record
Suggest a correction — opens a GitHub draft for you to review and submit.