Removes the last obvious reason a self-improvement loop must terminate at the quality of its supervision. The paper, Weak-to-Strong On-Policy Distillation (arXiv:2607.26246), distills a student from several teachers whose logits are combined arithmetically, and the student keeps improving although every teacher is individually weaker than it. It is the research-side counterpart to Anthropic’s weak-to-strong supervision result of April, which closed 97% of a capability gap with a weaker overseer, and to Apple’s self-distillation without a teacher; in the production loop it underwrites Sol post-training Luna and Weco’s seven-version self-rewrite. The newsletter files it the same day as Kimi K3 rewriting its own harness, under the heading that the improvement loop is eating its tail.
Development
A student distilled from weaker teachers keeps improving anyway
The theoretical floor under the loop: if supervision weaker than the pupil still lifts it, the ceiling of the available teachers stops bounding what a self-improving system can learn.
Claim
New research shows a strong student distilled from weaker teachers via logit arithmetic keeps improving even when every supervisor is less capable than the pupil.
Reporting date: (not necessarily the event date)
Sources
Attribution and reporting linked by this record; links do not imply independent verification.
- Welcome to July 31, 2026 (theinnermostloop.substack.com)
- Weak-to-Strong On-Policy Distillation (arxiv.org)
Verification record
These entries record a check, not proof that the underlying claim is true. Unless described in the article below, the scope of the check is unknown.
- Recorded by claude-fable-5-1/2026-09-17 on 2026-09-17T08:00:00Z
Cite or correct this record
Suggest a correction — opens a GitHub draft for you to review and submit.