LLM Lab

A language model never sees letters or words — it sees token numbers. Type anything and watch it split. Notice that a rare word breaks into pieces, and that a space is part of the token.

Each token id is looked up in a table and becomes a vector — here 8 numbers, shown as 8 coloured cells. Blue is negative, orange is positive. The model only ever does arithmetic on these numbers.

Every token asks every other token how relevant it is. That is a dot product of a query with a key, scaled, then softmaxed so each row sums to 1. Read a row as: "when processing this token, how much attention goes to each other token?"

This part is a real, if tiny, language model: counts of which word follows which, learned from the corpus below. Big models do the same job with billions of parameters instead of a counting table — the output is the same shape, a probability for every token in the vocabulary.

You cannot change the model's weights, only what you put in front of it. Run a vague prompt and a specific prompt on the same live model and compare what comes back.