Click a preset or edit the numbers directly. The columns of the matrix are exactly where the basis vectors land.
The Toy Box
Forty interactive pieces, out of their boxes
The series has six hundred-some visualizations. These are forty favorites, pulled out of their posts so you can just play. Every one of them was built to make a single intuition stick — press play, drag things, click around. The link on each card goes to the post where it earns its keep.
Foundations
The math and the hardware everything else stands on.
Each output cell is one dot product: a row of A against a column of B. Watch the sweep, then you'll see it forever.
Same start, same bowl. SGD zig-zags off the steep walls, momentum glides, Adam adapts per-parameter and wins.
Drag T. Low temperature collapses the distribution onto the winner; high flattens it toward uniform. The ranking never changes.
Type a number. Then try 65504, 65505, and 100000, and watch fp16 fall off a cliff while bfloat16 shrugs.
Language before transformers
What prediction looked like before attention, and why it worked.
Watch the entropy of English collapse as context accumulates. This 1951 game is the entire premise of language modeling.
Click an analogy and watch the arithmetic. The offsets are parallel because training made the geometry mean something.
Step through the gates. Watch one slot get erased by the forget gate and immediately rewritten by the input gate.
Tokens
How raw bytes become the units a model actually thinks in.
One family emoji: 1 grapheme, 7 code points, 25 bytes. Click through the examples and watch the three layers come apart.
BPE merges the most frequent pair, over and over. Common words crystallize into single tokens; rare ones stay in pieces.
A word the tokenizer never saw still splits cleanly, because encoding replays the merge list in training order.
Viterbi considers every way to cut the word and keeps the best total. Greedy matching can't back out; this can.
Inside the transformer
The machine itself, one block at a time.
Four tokens, five steps. Click a token and trace its query through the scores, the softmax, and the weighted sum.
Toggle bidirectional against causal. Each token sees one more position than the last, and the upper triangle is exactly zero.
Three dimension pairs, three speeds. The fast pairs encode fine position, the slow pairs encode coarse. That's RoPE.
Click a neuron: its row of W₁ is a pattern to match, its column of W₂ is what gets written to the stream when it fires.
Turn specific neurons off and watch the model lose confidence in the correct answer. Facts live in surprisingly few places.
Most experts stay dark for any given token. That sparsity is the whole economy of mixture-of-experts.
Inference, one token at a time
What actually happens when a model generates.
Hit next-token. The attention matrix grows by exactly one row, and everything above it is already cached.
Toggle MHA, GQA, and MQA, and 7B against 70B. The KV cache is the scarce resource; head-sharing is why it fits.
Temperature, top-k, top-p, and min-p on live bars. Set a combination, then hit Sample and draw a token.
A small model guesses five tokens; the big one checks them all in a single pass. Identical output, far fewer passes.
The grammar masks illegal tokens before sampling, so the model literally cannot emit invalid JSON. Step through it.
Serving systems
The engineering that turns a model into a service.
Static batching piles up gray idle cells as short requests finish. Continuous batching refills every slot immediately.
Three requests scattered through a shared block pool, each holding a tiny block table. This is PagedAttention.
Two requests share a system prompt until one writes. Only the written block gets cloned; everything else stays shared.
Tiles stream through SRAM. The full score matrix is mathematically defined but never exists as bytes anywhere.
Pick a model, a precision, some GPUs. Whatever's left after the weights is your KV budget, and the KV budget is your concurrency.
Drag batch size. Throughput saturates while latency keeps climbing — everything right of the knee is user experience thrown away.
Training & post-training
How the weights get made, and made agreeable.
The forward pass covers the whole sequence, but loss only fires on the assistant's tokens. That mask flips a completer into an assistant.
Trace PPO's clipped objective. Once the policy moves too far, the gradient just goes flat — that plateau is why training doesn't explode.
With no balancing pressure, tiny initial advantages compound until two experts do all the work and six do nothing.
Retrieval & context
Getting the right information in front of the model.
A real query stepped through parse, chunk, embed, retrieve, rerank, assemble, generate, cite — with the payload at every boundary.
The hallucinated answer lands nearer the real passage than the query does. Then it gets thrown away. That's HyDE.
Old turns compact into a running digest while the last few stay verbatim. The model's view stays small as the conversation grows.
Agents & protocols
The loop, and the standards forming around it.
Generate, parse, execute, resume. The transcript on the right is the only memory the model has between turns.
Sequential tool calls stack three round-trips. Parallel calls fan out and come back together — same tokens, less wall-clock.
Add hosts and tools one at a time. Bespoke integrations grow like area; protocol adapters grow like perimeter.
initialize, tools/list, tools/call. Everything is plain JSON-RPC; the transport just carries the envelopes.
Commit to a hash before anyone sees the auth code; reveal the secret at token time. A stolen code is worthless on its own.