Transformer Unboxer is a transformer you can actually see. Type a short sentence and watch a real, from-scratch transformer turn it into emojis — with every matrix it computes shown on screen, with its real numbers. Most explanations of how transformers work are either diagrams that skip the real math, or real models too large to follow. This one is a genuine trained transformer, shrunk small enough to hold in your head: you can trace a single number from the word you typed all the way to the emoji it produces. Because it’s tiny — a handful of dimensions, a six-word input, one attention head — nothing is hidden. You see the embeddings, the attention scores, the feed-forward step, and the output distribution, each as a small table you can read. And unlike toy demos where attention is decorative, here the model must combine several words to choose the “mood” emoji — so you can watch attention do real work. It runs entirely in your browser: no install, no sign-up, no GPU. Step through the whole forward pass with the tabs — embedding, self-attention, feed-forward, and output. If you’ve ever read that transformers use “query, key, and value” matrices and wished you could see the actual numbers, this is where you look.
Transformer Unboxer — a transformer small enough to see
Frequently asked questions
Is this a real transformer or just an animation?
Real. It's a from-scratch model trained on a synthetic dataset; every matrix shown is an actual learned weight, and the forward pass is the real computation (verified against the PyTorch reference).
How does attention work in a transformer?
Each position builds a query, compares it against every key to get attention weights, and mixes the values accordingly. In the Unboxer you can watch the "mood" position attend to several words at once and combine them — attention doing real work, not decoration.
What are the query, key, and value matrices?
Three learned weight matrices that turn each token's vector into a query, a key, and a value. The Unboxer shows all three with their real numbers on the Self-Attending tab.
Do I need to install anything?
No. It runs entirely in your browser — no account, no download, no GPU.
How small is the model?
Deliberately tiny: hidden size 6, a six-word input, one layer, one head — about 25,000 parameters. Small enough that every matrix fits on screen.
Who is it for?
Anyone learning how transformers work — students, self-learners, and educators who want to see the mechanism, not just read about it.