Loss functions

A loss function scores how wrong a model's predictions are as a single number: lower is better. Mean squared error averages the squared gaps between predictions and true answers. Language models use cross-entropy, which is small when the model gave the right next word a high probability. Training is the search for weights that make the loss small.

Why it matters: Gradient descent needs a loss to go downhill on.

Builds on: Averages and spread, Functions and graphs, Training versus using a model

Learn it, then prove it

Sign in (free) to take this step's mastery check and save your journey.