Zero Cost Abstractions

The idea of a zero cost abstraction is very important to certain programming languages, like Rust and C++, which intend to enable users to write programs with excellent performance profiles with relatively little effort. Since this idea is fundamental to the design of Rust and my work, I want to investigate, for a moment, what exactly a zero cost abstraction even is.

The idea is summarized in its original by Bjarne Stroustrup, the original developer of C++:

What you don’t use, you don’t pay for. And further: What you do use, you couldn’t hand code any better.

In this definition, there are two factors that make something a proper zero cost abstraction:

• No global costs: A zero cost abstraction ought not to negatively impact the performance of programs that don’t use it. For example, it can’t require every program carry a heavy language runtime to benefit the only programs that use the feature.
• Optimal performance: A zero cost abstractoin ought to compile to the best implementation of the solution that someone would have written with the lower level primitives. It can’t introduce additional costs that could be avoided without the abstraction.

However, I think its important to keep in mind that there is a third requirement for something to be a zero cost abstraction. It’s often overlooked because its simply a requirement of all good abstractions, zero cost or not:

• Improve users’ experience: The point of abstraction is to provide a new tool, assembled from lower level components, which enable users to more easily write the programs they want ot write. A zero cost abstraction, like all abstractions, must actually offer a better experience than the alternative.

All of these are of course ideals that can never be fully realized, and this third in particular, well - there’s no accounting for taste. But for zero cost abstractions, the third point is actually especially important, because zero cost abstractions are competing with two different alternatives. On the one hand, it must be better than handwriting code with the same performance profile yourself, that’s rather clear. But on the other, it must also be better than eating the performance cost and using an abstraction which isn’t zero cost. That doesn’t mean we need to be strictly better than the non-zero cost abstraction, because the performance cost is a factor, but we do have to be within spitting distance so that whatever additional programmer overhead you take on to be “zero cost” feels worthwhile.

(I think to some extent Rust has tried to cheat this by making non-zero cost abstractions actively unpleasant to write, making it feel like the zero cost abstraction is better. I think this is a mistake that hurts the overall UX of the language and probably just keeps some people from using it at all, hurting our overall goals.)

I want to say, also, that actually creating a zero cost abstraction that achieves this trifecta is both incredibly difficult and incredibly awesome. Rust has only really done this excellently a few times (all extremely high impact), and having been involved in what I think is going to be one of those success stories - async/await of course - it feels like holding fire in your hands. Last September I commented to a friend that I was worried I would never do any work as good as the work I had just done (in reference to the Pin API), and I really did feel that. But we rarely achieve something so great, because its extremely difficult and involves a fair amount of luck as well (many problems probably just don’t have a really great zero cost abstraction yet to be discovered, at least within the design constraints prior decisions have put around them).

To be clear about what I mean, I want to list a few of the really great zero cost abstractions in Rust:

• Ownership and borrowing. Of course this is the biggest one, guaranteed memory and thread safety without a garbage collector is Rust’s original, huge success story.
• Iterator and closure APIs. This is another classic. While there are some cases where internal iteration might be optimized better, the fact that you can write map, filter, iterative for loops, and so on over slices and be optimized to the equivalent of some handwritten C is absolutely astounding.
• Async/await and Futures. The Futures API is an important example, because the early versions of futures hit the “zero cost” part of zero cost abstraction really well, but was not actually providing a good enough UX to drive adoption. By adding pinning to support async/await, references across awaits, and so on, we’ve made a product that I really think will solve users’ problems and make Rust more viable for writing high performance network services.
• Unsafe and the module boundary. Underlying all of these, and every one of Rust’s success stories, is the notion of unsafe blocks and privacy that allow us to dip into raw pointer manipulation to build these zero cost abstractions. None of Rust’s brilliant features would be possible without this really fundamental ability to break the rules locally to extend the system beyond what the typechecker can handle. This is the zero cost abstraction that is the mother of all other zero cost abstractions in Rust.

In other areas, we still haven’t succeeded so much at finding zero cost abstractions. One example of this is the trait object as a solution for dynamically dispatched polymorphism. (Note here that the dynamic dispatch is a part of the goal here, so the virtual call is not non-zero cost.) The problem is that the concept of object safety, as well as sized and unsized types, and probably just some bad ergonomics around the coercions, has made trait objects really unwieldly to work with; I’m usually pretty annoyed when I have to use them. For a good 18 months at least I’ve wanted to really dig into this problem space, but other things have always taken priority.