The NanoGPT speedrun is an open leaderboard built on Andrej Karpathy’s nanoGPT: competitors change the architecture, optimizer and kernels to reach a fixed validation loss on FineWeb as fast as possible on a single 8xH100 node, and every record is a public pull request. The corpus follows it from 127.7 seconds through 122.2, 119.3 and 116.4 seconds in one week, then to under 100 seconds in January and 75.4 seconds in August; by May, agents given idle compute beat the human baseline on its optimizer track, which is why it belongs to the recursive-self-improvement strand.
Benchmark
NanoGPT speedrun
Keller Jordan's modded-nanogpt speedrun: the wall-clock time to train a GPT-2-class model to a fixed validation loss on FineWeb on one 8xH100 node, whose falling record the corpus tracks as the cost of the training loop itself.
Sources
Attribution and reporting linked by this record; links do not imply independent verification.
- Welcome to December 25, 2025 (theinnermostloop.substack.com)
Development timeline
Newest reporting dates first. Reporting dates may differ from event dates.
Cite or correct this record
Suggest a correction — opens a GitHub draft for you to review and submit.