Wyart connects statistical physics with deep learningDeep learning uses neural networks with many layers to learn representations from data. by treating intelligence as the discovery of useful large-scale variables hidden inside high-dimensional data. When observations have a hierarchical structure, deeper networks can build successive abstractions more efficiently than shallow models that try to represent every interaction at once.
The discussion links this hierarchy to the curse of dimensionalityThe curse of dimensionality describes problems that become harder as the number of independent variables increases., language and creativity. Wyart argues that useful generalizationGeneralization is an AI system's ability to apply what it learned to new examples and conditions that were not part of its training experience. depends on finding lower-dimensional structure and recombining learned patterns, rather than memorizing every possible configuration of the world.
He proposes predicting learned latent representationsA latent representation is an internal numerical encoding an AI model uses to represent and process information. instead of raw sensory detail as a route to greater sample efficiencySample efficiency measures how much useful learning an AI system achieves from a given number of training examples or experiences.. The theory is still being tested empirically, and reconstructing rich observations from those abstract representations remains an open generative problem.
Watch on YouTube




