Article URL: https://github.com/leonickson1/Swiftlet Comments URL: https://news.ycombinator.com/item?id=49158333 Points: 39 # Comments: 10

Swiftlet is a Swift + Metal runtime for the Qwen3-Next and Qwen3.5/3.6 MoE hybrid model family. It keeps only the small dense core of a model resident in memory and streams the routed Mixture-of-Experts weights from storage on demand. The result: The 35B also runs on an iPhone 17 in about 2.5 GB of RAM, at about 1 tok/s today. As far as we know, that is the first time a model of this class has run natively on a phone. Status: working end to end. Both models generate correct, validated output. The current focus is kernel speed (the decode loop is dispatch bound, not IO bound, so there is clear headroom). One expectation to set honestly: only about 3B parameters are active per token, so these models chat and write like large models but recall facts like small ones. The same command also repacks raw MLX checkpoints (--from-hf mlx-community/... or --source /path/to/checkpoint). Requirements: Apple Silicon, macOS 14+ or iOS 17+, free SSD space for the container (18 GB for the 35B, 42 GB for the 80B). The 35B runs on iPhone inside Priv AI on the App Store: open Settings, then Experimental Models, and download the model. It streams from storage and chats on-device with no server involved. The Experimental Models feature ships in the newest app version, which is still in App Store review, so it may not appear for a couple of days. If you want the phone experience today, build the app from source: the app is open source at leonickson1/localLLM. Clone this repo next to it as swiftlet, open the Xcode project, and run it on your iPhone. These models activate only about 3B of their parameters per token. Each layer routes every token to 10 of 512 experts (80B) or 8 of 256 (35B). Swiftlet: