Embeddings

Each token is turned into an embedding: a vector of hundreds or thousands of numbers learned in training. Tokens used in similar ways end up with similar vectors, so "cat" sits near "kitten" and far from "carburettor". Directions can carry meaning too: in a classic example, king − man + woman lands near queen. Similarity is measured with the dot product.

Why it matters: Embeddings are how meaning becomes geometry the network can compute with.

Builds on: Tokens, Vectors, The dot product and similarity

Learn it, then prove it

Sign in (free) to take this step's mastery check and save your journey.