The chain stops being a leaderboard and becomes evidence that the optimizations were general. The NanoGPT speedrun’s main track asks for 3.28 FineWeb validation loss on 8xH100, and six months of work had pushed that under two minutes, most recently to 115.1 seconds; applied to the GPT-2-medium track, which targets 2.92 loss, the same advances cut that record by a quarter (X). The newsletter links the same record-holder’s post as the 115.1-second result, whose top-level text describes the 3.28-track gain; the 2.92-track transfer is the newsletter’s report of it. The result reframes the whole chain from 127.7 seconds onward as accumulating transferable training knowledge, which is what later makes handing the speedrun to agents a test of automated research rather than of tuning; the main-track record falls again to 113.7 seconds three days later.
Development
Speedrun gains transfer to the harder track and break it by 25%
The newsletter's 'free compute in the geometry of the model': a leaderboard chase turns into evidence that its accumulated tricks were general improvements to training rather than overfitting to one target.
Claim
Six months of NanoGPT speedrun optimizations found on the 3.28 loss track were shown to generalize to the harder 2.92 track, breaking that world record by 25%.
Reported metric: -25% on the 2.92 track
Reporting date: (not necessarily the event date)
Sources
Attribution and reporting linked by this record; links do not imply independent verification.
- Welcome to January 2, 2026 (theinnermostloop.substack.com)
- New NanoGPT Speedrun WR at 115.1 (-1.3s) (x.com)
Verification record
These entries record a check, not proof that the underlying claim is true. Unless described in the article below, the scope of the check is unknown.
- Recorded by claude-fable-5-1/2026-09-17 on 2026-09-17T08:00:00Z
Cite or correct this record
Suggest a correction — opens a GitHub draft for you to review and submit.