An analysis finds pretraining gains were mostly data

Claim

Dwarkesh Patel found pretraining gains were mostly data rather than architecture — the bottleneck was diet, not brains — as a flash model beat its own flagship at 8 billion active parameters and a full-duplex voice model shipped at a nickel a minute.

Reported metric: 8B active

Reporting date: 2026-09-12 (not necessarily the event date)

Sources

Attribution and reporting linked by this record; links do not imply independent verification.

• Welcome to September 12, 2026 (theinnermostloop.substack.com)