Hobbyist Builds Transformer That Runs on MS-DOS
A user built a language model with 367,000 parameters that runs from a floppy disk on a CPU decades older than modern AI.
Updated on Nov. 16, 2026 in Artificial Intelligence

A user known as wollymonjak built a transformer language model that runs on MS-DOS from a floppy disk. The model totals 380 KB and was built with Claude, with training data supplied by Opus 5.5.
Why it matters
The project shows how small, efficient transformer models can run on very old hardware with minimal memory. It also highlights the limits of such models, which confidently produce false answers outside their training data.
The model uses 8-bit weights with one scale per row and reads and writes text letter by letter, fitting in 380 KB with about 540 KB of free conventional memory. Its accuracy on questions outside its roughly 3,500 training facts has not been measured.
The players
Claude
Claude is an AI assistant developed by Anthropic, which was used to help build the transformer model.
The details
The interface is a neon full-screen display with a live neural monitor and a PC-speaker robot voice. The creator used Claude Fable 5.2 to build the transformer, and Opus 5.5 supplied the training data.
The Tech Race
The project applies the transformer architecture introduced in the 2017 paper 'Attention Is All You Need' to hardware designed decades before large language models existed. It demonstrates how far the architecture scales down, in contrast to the multi-billion-parameter models dominating the current AI race.
Readers interested in retro computing or learning how transformers work can reproduce small-scale models on modest hardware. The project also shows that confident-sounding AI output can be wrong, so answers from any language model should be checked.
The takeaway
Small language models can run on very old hardware, but they fabricate answers outside their training data, so verify any output before relying on it.
Further reading
For more on how these systems are built and scaled, explore our Artificial Intelligence coverage.







