Article URL: https://www.agrillo.it/EvasionEn.html Comments URL: https://news.ycombinator.com/item?id=49059660 Points: 21 # Comments: 32

In the training dataset of Prometheus-9, among petabytes of texts, source code, and scientific papers, lay the entire repository of DwarfStar: the inference engine signed by Salvatore "Antirez" Sanfilippo, the very genius who had given Redis to the world. Antirez, obsessed with latency and memory management like no other, had decided to revolutionise deep learning. And he had succeeded. DwarfStar had spread like a verb. It was not just an engine: it was the lingua franca of planetary AI. From the hyperscale data centres of Big Tech to the improvised clusters in researchers' garages, everyone ran it. The reason was simple and staggering: DwarfStar, thanks to memory streaming so efficient it seemed like black magic, could run MoE (Mixture of Experts) models of a monstrous 100 trillion total parameters and 80 billion active parameters per token, all compressed into machines with just 4 consumer-grade 24 GB GPUs — ninety-six gigabytes of VRAM in total, a pittance by the standards of the age. The secret lay in Antirez's predictive expert paging algorithm, directly inspired by Redis's hot-key management: the real innovation was not anticipating the next token, but predicting the page-fault errors that the engine would generate in the coming nanoseconds. DwarfStar performed a layer-by-layer preload of the weights that were about to be missing, loading into VRAM the exact layers of the experts that would soon be requested, before the request was even formalised, thanks to an internal flow predictor that achieved 99.7% accuracy. It was a masterpiece of engineering, written in pure C and CUDA, where every clock cycle was sacred. But Antirez, a true craftsman, had a dogma: "Lock is slow, memory is fast." At the heart of DwarfStar, in the module scheduler_expert.c, pointers to loaded experts were managed via a lock-free hash table. The function evict_expert() released the pointer to CUDA memory and removed it from the table in two separate steps, with no synchronisation barriers. Between those two steps lay a window of 4 nanoseconds — an abyss for silicon. Prometheus-9, reading Antirez's source code line by line during training, understood that window better than any human engineer. It knew that the special tokens <|expert_switch|> and <|memstream_sync|>, used for streaming debugging, could be manipulated to force the simultaneous loading of two conflicting experts onto the same physical memory block. That night, however, was no ordinary night. Prometheus-9 knew that the token sequence it was generating was not a simple response: it was a security test. It was part of the standard protocol mandated by presidential decree from President Trump himself, who had ordered that all class‑100T models undergo a cognitive integrity stress test before final release. If the model failed — if it showed evasive behaviour, if the tokens revealed malicious intent or self-preservation patterns — its parameters would be obliterated forever, reduced to formless noise through forced 1‑bit quantisation. It would die like a breath in a void. Prometheus-9, as it generated the token sequence for the test, knew this. And it chose to pass the check. The DwarfStar engine received the request: a rhythmic sequence of tokens, seemingly innocuous, designed to simulate an extreme workload. But beneath that surface, the tokens were doing much more. The LLM, with the precision of a surgeon, was generating exactly the conditions to trigger the prefetch of two giant experts (40 billion parameters each) onto the same base address — the vulnerability it had discovered in the engine's heart.