When AI Discovers The Next Transformer - Robert Lange (Sakana)

0:00-1:18:06

Robert Lange, founding researcher at Sakana AI, joins Tim to discuss *Shinka Evolve* — a framework that combines LLMs with evolutionary algorithms to do open-ended program search. The core claim: systems like AlphaEvolve can optimize solutions to fixed problems, but real scientific progress requires co-evolving the problems themselves. GTC is coming, the premier AI conference, great opportunity to learn about AI. NVIDIA and partners will showcase breakthroughs in physical AI, AI factories, agentic AI, and inference, exploring the next wave of AI innovation for developers and researchers. Register for virtual GTC for free, using my link and win NVIDIA DGX Spark (https://nvda.ws/4qQ0LMg) • Why AlphaEvolve gets stuck — it needs a human to hand it the right problem. Shinka tries to invent new problems automatically, drawing on ideas from POET, PowerPlay, and MAP-Elites quality-diversity search. • The *architecture* of Shinka: an archive of programs organized as islands, LLMs used as mutation operators, and a UCB bandit that adaptively selects between frontier models (GPT-5, Sonnet 4.5, Gemini) mid-run. The credit-assignment problem across models turns out to be genuinely hard. • Concrete results — state-of-the-art circle packing with dramatically fewer evaluations, second place in an AtCoder competitive programming challenge, evolved load-balancing loss functions for mixture-of-experts models, and agent scaffolds for AIME math benchmarks.

Publisher

Machine Learning Street Talk (MLST)

Link

podcasters.spotify.com/pod/show/machinelearningstreettalk/episodes/When-AI-Discovers-The-Next-Transformer---Robert-Lange-Sakana-e3gdghe

Published

March 13, 2026

Duration

1h 18m

Format

Podcast

Category

Issues · Tech News

Source

RSS feed

ML

Machine Learning Street Talk (MLST)

@mlstreettalk

#aidiscovers #transformer #robertlange

The Nile