Sakana compiles documents directly into model weights

Claim

Sakana demonstrated compiling documents directly into model weights via hypernetworks, giving models durable memory without bloating context, while researchers self-distilled foundation models into multi-token predictors decoding three times faster at under 5% accuracy loss.

Reported metric: 3x faster

Reporting date: 2026-02-27 (not necessarily the event date)

Sources

Attribution and reporting linked by this record; links do not imply independent verification.

• Welcome to February 27, 2026 (theinnermostloop.substack.com)