The paper (Tandon, Dalal and twelve co-authors including Hashimoto, Guestrin, Choi and Yu Sun; arXiv 2512.23675, posted 29 December 2025) formulates long-context modeling as continual learning: a standard sliding-window Transformer keeps training at test time by next-token prediction on the context it is reading, compressing that context into its weights, with meta-learning at training time to initialize the learner. TTT-E2E scales with context length the way full attention does while Mamba 2 and Gated DeltaNet do not, yet keeps RNN-like constant latency, 2.7 times faster than full attention at 128K context (arXiv). In the trajectory it turns the prediction that continual learning is solved in 2026 into a published mechanism within a week, opens the weight-level route to long context that ByteDance’s in-place test-time training extends in April, and sits beside the scaffold-level route of the Recursive Language Model three days later.
Development
Test-time training gives continual learning constant latency
Long context is reframed as a continual-learning problem the model solves by compressing what it reads into its own weights, six days after an Anthropic researcher predicted continual learning would be solved in 2026.
Claim
Stanford researchers achieved continual learning via test-time training, letting models learn from next-token prediction at constant latency regardless of context length.
Reporting date: (not necessarily the event date)
Sources
Attribution and reporting linked by this record; links do not imply independent verification.
- Welcome to December 30, 2025 (theinnermostloop.substack.com)
- End-to-End Test-Time Training for Long Context (test-time-training.github.io)
- End-to-End Test-Time Training for Long Context (arxiv.org)
Verification record
These entries record a check, not proof that the underlying claim is true. Unless described in the article below, the scope of the check is unknown.
- Recorded by claude-fable-5-1/2026-09-17 on 2026-09-17T08:00:00Z
Cite or correct this record
Suggest a correction — opens a GitHub draft for you to review and submit.