Article URL: https://mlugg.co.uk/posts/incremental-compilation-internals/ Comments URL: https://news.ycombinator.com/item?id=49085666 Points: 56 # Comments: 8

As a member of the Zig core team, one of the most impactful projects I’ve been involved with is the implementation of incremental compilation into the Zig compiler. This feature allows the compiler to detect which individual functions and declarations have changed since a project was last built, recompile only that code, and directly patch the resulting bytes into the output binary, making the rebuild extremely fast. The Zig project has been working towards this feature for a long time, and over the last few release cycles, it has finally gone from a proof-of-concept quality feature to one which is viable for real-world projects and which most of the Zig core team makes daily use of. Today, using Zig’s incremental compilation, you can make changes to real, complex applications in a matter of milliseconds. But don’t just take my word for it! Here’s a simple video (no audio) demonstrating me using Zig to quickly make and test some changes to Fizzy, a pixel editor application. The initial build takes around 5 seconds, and then every time I make a change, a rebuild completes in 50–70ms. For this demo, I had to upgrade Fizzy to Zig’s master branch. This is because while Zig 0.16.0 does have support for incremental compilation, it is missing some important linker features which have since been implemented. This means that if you prefer to stick to tagged releases of Zig, you likely won’t be able to try this out until 0.17.0 drops; sorry! If you’re already convinced and just want to know how to use this, great! Head on down to the last section of this post to find out. But perhaps you’re understandably skeptical that this is applicable to most projects, or, like me, you just enjoy learning how stuff like this works. For all of you folks, let’s dig into the details! The Zig compiler’s pipeline can be split up into a few different parts, which we’ll look at in order. The first part works at the granularity of entire source files, and basically consists of running the following process in a loop: If you’re curious, ZIR (Zig Intermediate Representation) is an untyped SSA-form IR—but don’t worry if you have no idea what that means, because it won’t really matter here. All we care about is that we’re converting an entire source file into a different format.