AI Is Learning at the Wrong Level of Abstraction — Matthieu Wyart

0:00-1:18:56

This episode is sponsored by Notion. Learn more about Notion's Developer Platform today at https://notion.com/mlstWhy can deep networks discover abstractions that shallow models miss? Statistical physicist Matthieu Wyart joins Tim Scarfe to argue that the answer lies in the hidden hierarchy of data. Language and images are built from parts within parts; depth lets a network recover those coarse-grained variables and escape the curse of dimensionality. The conversation moves from jamming transitions and rough loss surfaces to Chomsky, context-free grammars and machine creativity. Wyart explains why next-token prediction can still recover compositional structure, where current systems fall short of genuine scientific invention, and why predicting latent representations rather than raw tokens could make learning far more sample-efficient. They also examine diffusion models, neural scaling laws and the limits of physics-inspired theory.

Publisher

Machine Learning Street Talk (MLST)

Link

podcasters.spotify.com/pod/show/machinelearningstreettalk/episodes/AI-Is-Learning-at-the-Wrong-Level-of-Abstraction--Matthieu-Wyart-e3n81u2

Published

August 10, 2026

Duration

1h 19m

Format

Podcast

Category

Issues · Tech News

Source

RSS feed

ML

Machine Learning Street Talk (MLST)

@mlstreettalk

#level #abstraction #matthieuwyart

The Nile