The ceiling for local AI processing on cheap edge hardware just shattered. A developer known as slvDev_ successfully booted a 28.9 million parameter Large Language Model directly on an $8 ESP32-S3 microcontroller. The tiny board generates text at roughly 9.5 tokens per second. It does this completely offline without pinging a single cloud server.
Before this project, the largest language model running on this class of microcontroller topped out at just 260,000 parameters. Squeezing something over 100 times larger onto the exact same hardware required a drastic architectural workaround. The ESP32-S3 simply lacks the fast RAM needed to hold a model of this size.
To bypass the hardware limits, slvDev_ borrowed a technique from Google’s Gemma architectures, specifically Gemma 3n/4. The developer used Google’s Per-Layer Embeddings trick to dump the massive 25-million-parameter embedding table straight into the chip’s slower flash memory.
This setup requires the chip to read just six rows of data per token generated. That amounts to about 450 bytes. The low read requirement keeps processing speeds high while consuming only about 560K of fast memory, metrics confirmed on the official GitHub repository documenting the token speeds and hardware specs.
The model is trained entirely on the TinyStories dataset built by Microsoft Research. It outputs simple, coherent short stories directly from the board. You can read a deeper technical breakdown of slvDev_’s breakthrough running the LLM in XDA Developers’ coverage published recently.
Local Edge AI Outgrows the API Dependency
OpenAI’s original ChatGPT launched with 117 million parameters. This ESP32-S3 setup holds roughly one-quarter of that architectural complexity on a piece of hardware that costs less than a fast-food meal. Devices that once required constant internet connections to parse text can now generate natural language responses locally.
That removes latency completely. It also eliminates API subscription costs for developers building smart home hardware or offline industrial sensors. By proving that a 28.9 million parameter model can run smoothly on $8 silicon, the project opens the door for fully autonomous voice assistants that never send audio data back to a corporate server.