Jeremy Berman Explains How Language Models Are Trained

In conversation with Matthew Berman, researcher Jeremy Berman uses an apartment-price example to introduce parameters, loss and gradient-based training. He then explains how language models learn to predict tokens and illustrates the difference between a base model and an assistant tuned on examplesFine-tuning adapts a pretrained model by updating its parameters on examples chosen for a particular task or subject..

Berman describes reinforcement learning with verifiable outcomesReinforcement learning trains an AI system to choose actions using feedback about the results of its earlier choices. and group-relative scoring as a way to improve reasoning, distinguishing training from the thinking performed at inference time. His discussion is an intuitive overview; simplified examples and predictions should not be read as a complete description of every model.

Watch on YouTube↗