We Finally Know What Ilya Is Working On...
Ilya Sutskever argues that AI needs a more sample-efficient path to generalization because larger pretraining runs alone do not close the gap with human learning.
AI Scaling
Videos about how model capability changes with more compute, data, parameters, training, and deployment resources. 1 video.