Article URL: https://github.com/Carloscodix/qapla Comments URL: https://news.ycombinator.com/item?id=49178464 Points: 17 # Comments: 6

This text was written with the assistance of an AI. Careful: assistance doesn't mean the AI wrote it. It means it corrected, reviewed and filled in some parts, but the author is human (or so I believe). And why did we need it? Because we take for granted that training a model requires a GPU or a datacenter. Not always: sometimes something as small and as cheap as an eight-buck micro is enough to train one from scratch. Read that again: train. From scratch. Not run a pre-cooked model. Train. Forward pass, backprop and weight updates, inside the chip. Edge AI is nothing new. TinyML has been doing inference on microcontrollers for years, community ports of llama2.c — Andrej Karpathy's minimalist project — put ~260K-parameter transformers on an ESP32, and just recently a brilliant project got a model of almost 29 million parameters running on an $8 ESP32-S3. They're excellent pieces of work, and although they happen to have overlapped in time with this experiment, they weren't the model we were following. The esp32-ai project by Slava S. (slvDev), and others like it, all share one thing: they are inference. The model is born somewhere else — a GPU, a datacenter — trained on its data, quantized, and only then loaded onto the chip so it can run it. The brain is cooked outside and served on-chip. We asked ourselves a different question: what happens when the model can't be born outside? What happens when it can't come pre-trained — because the data it needs to learn doesn't exist until the device is in place — can't carry that data with it, and has no internet to download it? We know what you're thinking: "Fine, but… why would anyone need to train a transformer on an ESP32?" And it's a fair question. Maybe the answer is "no reason at all". But… if we're going to get creative, picture this: a sensor bolted to a piece of farm machinery in the middle of a field, which has to learn the normal vibration of that specific machine (different from any other machine in the world) to spot when something's going wrong and schedule maintenance before it breaks (and no, you won't catch me out: I said there's no internet, but… LoRa? ;). Or picture a sensor in a plot of land that learns how that particular soil dries out — its earth, its sun, its drainage — and predicts when it'll need watering, before the plants suffer. In those cases (and others we can think of) the data didn't exist until the device was installed: nobody could have pre-trained it. The chip has to learn on the fly, alone, right where it stands. We don't have a piece of farm machinery lying around to experiment with. So, to put the real learning ability of an ESP32 to the test, we designed the experiment around the only thing we did have: language. How far can an eight-buck chip get, learning a language from scratch, with nobody's help?