Train a small GPT — the kind of model behind ChatGPT, a million times smaller — from scratch, right here in this tab. No server, no install. Watch the loss curve fall, then ask it to write you a sentence. Source ↗
Decision, forecasting & learned-terminal modes ↓ · one Mac-local specialist workspace
Train
Hyperparameters — preset above sets these. Click to edit individually.
Advanced
Loss curve wasm system: nominal
More metrics
Sample live
You can sample while training is still running — this is the trick. The model is a single instance in a Web Worker; Generate runs a forward pass against the same weights training is mutating, between steps. Sample twice during a run to watch what the model has picked up.
Inspect & evaluate
Benchmark
Score the loaded model against a registered held-out set.
Logit lens interpretability
Per-layer predictions for the current prompt. Reveals when specific knowledge appears in the residual stream. Optionally upload a tuned-lens .lenses sidecar trained via "posttrainllm tuned-lens" for calibrated probes.
Ablation interpretability
Zero out one layer's attention, MLP, or whole-block contribution, then sample. Reveals how load-bearing that component is.
Activation patching interpretability
Intervene on the residual stream at (layer, position) of the recipient prompt above. Zero: drop that token's representation at that depth — exposes whether it's load-bearing. Swap: substitute a hidden state captured from a separate donor prompt at the same coords — the full Meng et al. (2022) variant that reveals which downstream outputs causally depend on the donor's representation.
Watch the model think
Click any byte below to see what the model considered at that
position — the top-10 next-byte probabilities the
last forward pass produced, and the per-head attention
weights from the final transformer block (which earlier
bytes each head was looking at).
This is honest: it's the actual distribution that token was sampled
from, and the actual softmax weights from the last attention layer.
Nothing prettified. · marks bytes that are whitespace
or control characters.
Top-10 candidates
Attention from last layer · per head
Your machine detecting… computing…
computing…
4 Resources curated rabbit holes — 3Blue1Brown, Karpathy, papers, and this repo's own docs
Start here (visual + intuitive)
- 3Blue1Brown — Neural Networks — the maths, visually
- Jay Alammar — The Illustrated Transformer — the diagrams everyone has seen
- Karpathy — Let's build GPT — a 2 h build, from zero
- Karpathy — Neural Networks: Zero to Hero — the full lecture series
Go deeper (code + papers)
- karpathy/nanoGPT — the cleanest small GPT in PyTorch
- karpathy/minGPT — the original minimal build
- Attention Is All You Need — Vaswani et al., 2017
- LoRA: Low-Rank Adaptation — Hu et al., 2021
- AdamW — Loshchilov & Hutter, 2017
This project's own write-ups
- docs/notes.md — how every layer was built & tested
- docs/learn/README.md — the active learning index
- docs/techniques/lora_guide.md — LoRA in detail
- docs/performance/performance.md — the SIMD & WebGPU speed work
- docs/browser_notes.md — WASM, Workers, OPFS, WebGPU
Related projects
- mlc-ai/web-llm — run real LLMs in the browser (inference)
- ml-explore/mlx — Apple Silicon ML framework
- Hugging Face datasets — where the corpus picker pulls from
- MDN — WebGPU — the API powering the GPU path
5 WebGPU matmul benchmark Same matmul on WebGPU + WASM, side-by-side, with a live speedup number
Runs the same matmul through the WebGPU compute kernel and the WASM kernel, parity-checks they agree, and reports the per-shape speedup on your hardware. Needs Chrome / Edge 113+. For the historical end-to-end curve and its evidence limits, see /speedup.
Three ways to teach a small model.
Choose an answer, forecast a series, or predict a terminal screen. One workspace for inputs, training, evaluation, and saved models.
Start your local runtime
The specialist modes run on your Mac through Python. Your inputs stay between this page and the local runtime.
uv sync --project scripts/specialists --all-extras --frozen
pnpm --dir browser run build
uv run --project scripts/specialists python scripts/specialists/lab.py serveDownload the selected model first using the setup commands. The runtime loads cached weights. Training defaults to one step; a training smoke proves wiring, not model quality.
Make a choice in one forward pass.
Compare every allowed answer. Fit temperature on separate examples before interpreting the probabilities.
Result
Unqualified modelConnect your local runtime to run this example.
Every frame is predicted. No shell runs here. A tiny typing artifact does not know shell commands.
Inspect full output
These modes use a local Python runtime. Browser-only WASM/WebGPU execution for these specialists has not been built. The byte-level trainer above continues to run entirely in your tab.