Recorded on real hardware · replayed in real time
Watch a 125B model answer on one RTX 5070
Loading the recording…
Ready
Reading the prompt
Thinking
Exactly as the model wrote it, unedited. Runs in a sandbox. Click the game, press R to start, then steer with the arrow keys.
Inside the PC
Graphics card
VRAM · busiest experts
0expert uses
RAM + processor
every expert
0expert uses
SSD
lookup table & spill
0reads
24,576 experts · 48 layers × 512
each token: 10 per layer
on the graphics card
RAM, computed by the processor
read from SSD
Writing speed
-tok/s
GPU busy-%
GPU cache hit-%
Processor-%
Guess, then check
A small helper guesses the next words; the big model checks them all at once.
guessed 0
kept 0
Speed, timing, cache hit rate, RAM/SSD reads, helper guesses and GPU/RAM/CPU load are measured by Strata during the recording. Which exact experts light up is illustrative: Strata does not log per-token expert IDs, so the grid draws a pattern that matches the measured hit rate.
Run it on your own PC
Strata is free and open source. A 12 GB graphics card and 32 to 64 GB of RAM are enough.