Giving an agent research papers improves its results

The experiment, posted to r/MachineLearning on 27 March by u/kalpitdixit (thread), compared an autoresearch-style hyperparameter-search agent with and without access to computer-science papers and reported a 3.2% improvement for the arm that could read; the newsletter’s framing is that the agents are getting hungrier for input. It is a small number attached to a large question, whether autonomous researchers benefit from the literature the way human ones do, and it sits between Karpathy’s 650-experiment autoresearch run and the same issue’s Bilevel Autoresearch, which improves the loop by rewriting its search mechanism rather than feeding it papers. The setup was two identical runs of Karpathy’s autoresearch framework, a Claude Code agent optimizing a roughly 7M-parameter GPT-2 on TinyStories on an M4 Pro for 100 experiments per arm from the same seed configuration, with the only variable an MCP server (Paper Lantern) doing full-text search over more than two million CS papers; at the two-hour mark the arm without papers stood at a val_bpb of 0.4624 and the arm with papers at 0.4475, a 3.2% gap still widening, after considering 520 papers, citing 100 and trying 25 paper-sourced techniques including AdaGC, the sqrt batch-scaling rule, the REX schedule and WSD cooldown (full writeup). The author’s own limitation is a single run per condition on a tiny model, so the figure is suggestive rather than settled.