AlphaEvolve discovers an activation function that triples ReLU

In Mining Generalizable Activation Functions (arXiv 2602.05688, submitted 5 February), DeepMind researchers ran AlphaEvolve, with a frontier LLM as the mutation operator, over the space of Python functions within a FLOP budget, using out-of-distribution performance as the fitness signal; the newsletter singles out “Turbulent”, which it reports as outperforming ReLU threefold. The same issue notes xAI’s Grok-Imagine-Image pushing the image-generation Pareto frontier on the Arena leaderboards (post). AlphaEvolve entered the corpus assisting Terence Tao on Erdős #1026 and returns in March with new Ramsey bounds; here it is turned on a component of the networks it runs on, which in the recursive-self-improvement trajectory is the step from AI doing mathematics to AI redesigning AI, extended in August when a co-scientist invents an architecture that beats six frontier models.