Do you believe that an IoT chip ESP32-S3 microcontroller, which is only cheaper than two cups of coffee at US$8 (approximately NT$260), can run a language model with 28.9 million parameters completely offline? This isn’t just a dream, it’s a real open source project, and it’s much cheaper than both the Raspberry Pi and the NVIDIA Jetson. This project, called ESP32-AI, was shared on the ESP32 community version by developer slvDev and is now open source on GitHub. In comparison, the model Dave Bennett ran on similar chips in 2024 only had 260,000 parameters, and the results this time were improved by more than 100 times.
Open source players successfully used an $8 ESP32-S3 chip to run a 28.9 million parameter large language model offline
ESP32-S3 is an IoT chip launched by Espressif, which is widely used in smart home devices, sensor nodes and wearable devices. This chip has built-in 512KB of fast SRAM, 8MB of PSRAM and 16MB of flash memory, and costs only about $8. According to traditional practice, the entire model must be stuffed into fast memory to operate, which is why the previous model limit for similar chips was only 260,000 parameters.
Key breakthrough: Google’s Per-Layer Embeddings technology
The total size of the slvDev model is 14.9MB (4-bit quantization). It is impossible to load it all into SRAM. It is even difficult to put it into PSRAM. The past approach will encounter bottlenecks when the model becomes larger: every time a token is generated, the entire model must be read. The larger the model, the slower it will be, and eventually it will not work at all. The solution comes from the Per-Layer Embeddings technology proposed by Google in the Gemma model series. Most of the parameters in the language model are stored in a huge embedding table, and the model mainly reads data from it instead of performing intensive calculations. slvDev takes advantage of this feature to store a lookup table of approximately 25 million parameters in slower flash memory and access it through memory-mapped XIP. Each time a token is generated, the chip only needs to read about 6 lines and about 450 bytes of data from it. The actual portion of the core that requires intensive computation is only about 560KB, which can be left in fast SRAM.
28.9M LLM running on an $8 esp32, writing a story on a screen! fully on the chip, nothing goes to a server.
that’s ~100x bigger than the model people ran on this chip in 2023.
it grew out of @karpathy llama2.c and google’s per-layer embeddings, which is what lets a big model live… pic.twitter.com/m9PWTYCoKL— Slava S. (@slvDev) July 23, 2026
This design allows the huge embedded table to run almost “for free”. It just sits quietly in flash memory and is only sampled a small part at a time. slvDev pointed out in the explanation video that this idea is exactly the Per-Layer Embeddings technology of the Google Gemma model, but now it runs on the memory architecture of the microcontroller instead of the mobile phone or GPU. In addition, Andrej Karpathy’s llama2.c project also provides technical direction for training micro-language models and executing them in pure C language. ESP32-AI is built on the above-mentioned technology.
What can and cannot be done
The model was trained using Microsoft’s TinyStories dataset and is capable of writing short, coherent stories. It cannot answer questions, follow instructions, write code, or provide factual knowledge. The developers admit that the above limitations come from the scale of the core inference part of the model, and memory tricks cannot solve this problem.
When Brian Roemmele shared this news on He is testing three directions: embedding the model into simple nodes, matching it with a local voice front end, and exploring whether multiple chips can form a lightweight cluster network.
forget the $699 AI pins. this $8 chip just shattered the barrier for local AI hardware.
a developer just forced a 28.9 million-parameter LLM onto a standard ESP32-S3 microcontroller. it costs roughly 8 dollars, runs completely offline, and draws the power of a single LED.… https://t.co/ZRzlJH1NFP pic.twitter.com/SDllyZpENt
— ard (@ardchain) July 24, 2026
The original ChatGPT had 117 million parameters, and this model on the ESP32 is about a quarter of that. Although the inference capability is far less than that of ChatGPT, achieving such a scale on an $8 chip represents a breakthrough in architectural design rather than simple parameter stacking. The industry has been moving toward bigger models with more parameters over the past few years, but the more interesting frontier may be in the opposite direction: how small, cheap, and localized useful intelligence can become. When an $8 chip can generate a coherent story, it’s proof that the floor continues to move.
The significance of this project lies not only in the number of parameters, but also in the innovation of memory configuration strategies. Using XIP technology to place large embedded tables in flash memory instead of trying to cram all data into RAM, this design thinking has important reference value for the development of future edge AI devices.
LLM (Little Language Model) running on ESP32-S3 with screen output!
byu/Complex-Indication inesp32
Once it is accepted that a language model can live on an $8 chip and require no cloud connection at all, a host of new devices become feasible. Privacy is completely guaranteed by hardware, latency is only limited by local computing speed, and power consumption is so low that battery power becomes a reasonable option. Such devices do not need to answer all questions, but only need to be good at specific tasks, such as story generation, simple text generation, or local voice interaction.
The project for slvDev is already in Open source on GitHub, including firmware, wiring instructions, programming steps, and complete code for training, ablation experiments, and quantification. For developers interested in trying it out on their own hardware, all files are available for direct download.
In addition, slvDev candidly disclosed the tortuous history of the entire development process in the project description, including an error he himself discovered in parameter calculations, which once led to early published numbers being too high. Revision results are recorded in the repository along with the full commit history, a level of transparency that is rare in hardware-embedded AI projects.
From a larger perspective, this project represents an important signal in the field of edge AI. In the past few years, the development direction of language models has been the pursuit of larger parameter quantities and stronger reasoning capabilities, but ESP32-AI presents another possibility: deploying limited but useful intelligence capabilities into extremely low-cost hardware. When a chip costing less than NT$300 can generate coherent text locally, privacy-sensitive application scenarios, remote areas with insufficient network infrastructure, and extremely cost-sensitive IoT devices will usher in a new design space.
Source: KOCPC Chinese