Without boats, dreams dry up Async/await syntax in Rust was initially released to much fanfare and excitement. To quote Hacker
News at the time: This is going to open the flood gates. I am sure lot of people were just waiting for this moment
for Rust adoption. I for one was definitely in this boat. Also, this has all the goodness: open-source, high quality engineering, design in open, large
contributors to a complex piece of software. Truly inspiring! Recently, the reception has been a bit more mixed. To a quote a comment on Hacker News again,
discussing a recent blog post on the subject: I genuinely can’t understand how anybody could look at the mess that’s Rust’s async and think that
it was a good design for a language that already had the reputation of being very complicated to
write. I tried to get it, I really did, but my god what a massive mess that is. And it contaminates
everything it touches, too. I really love Rust and I do most of my coding in it these days, but
every time I encounter async-heavy Rust code my jaw clenches and my vision blurs. Of course, neither of these comments are completely representative: even four years ago, some people
had pointed concerns. And in the same thread as this comment about jaws clenching and vision
blurring, there were many people defending async Rust with equal fervor. But I don’t think I would
be out of pocket to say that the nay-sayers have grown more numerous and their tone more strident as
time has gone on. To some extent this is just the natural progression of the hype cycle, but I also
think as we have become more distant from the original design process, some of the context has been
lost. Between 2017 and 2019, I drove the design of async/await syntax, in collaboration with others and
building on the work of those who came before me. Forgive me if I am put a bit off when someone says
that they don’t know how anyone could look at that “mess” and “think that it was a good design,” and
please indulge me in this imperfectly organized and overly long explanation of how async Rust came
to exist, what its purpose was, and why, in my opinion, for Rust there was no viable alternative. I
hope that along the way I might shed more light on the design of Rust in a broader and deeper sense,
at least slightly, and not merely regurgitate the justifications of the past. I want to address a controversy that has gripped the Rust community for the past year or so: the
choice by the prominent async “runtimes” to default to multi-threaded executors that perform
work-stealing to balance work dynamically among their many tasks. Some Rust users are
unhappy with this decision, so unhappy that they use language I would characterize as
melodramatic: The Original Sin of Rust async programming is making it multi-threaded by default. If premature
optimization is the root of all evil, this is the mother of all premature optimizations, and it
curses all your code with the unholy It’s always off-putting to me that claims written this way can be taken seriously as a technical
criticism, but our industry is rather unserious. I want to wrap up my consideration of the idea of adding new auto traits to Rust with some notes
from a conversation I had with Ariel Ben-Yehuda. You can read these two previous posts for context: In my previous post, I described the idea of using an edition mechanism to introduce a new
auto trait. I wrote that the compiler would need to create an “unbreakable firewall” to prevent
using The response has been pretty optimistic that ensuring this would be possible, even though I wrote in
the post myself that I “despair” over how difficult it was. I’ve received a great example from Ariel
Ben-Yehuda which demonstrates how this problem is more difficult to solve than you would probably
think. In Rust, there are certain API decisions about what is and isn’t sound that impact all Rust code.
That is, a decision was made to allow or not allow types which have certain safety requirements, and
now all users are committed to that decision. They can’t just use a different API with different
rules: all APIs must conform to these rules. These rules are determined through certain “marker” traits. If a safe API could do something to a
value of a type which some types don’t support, the API must be bound by that marker trait, so that
users can not pass values of those types which don’t support that behavior to that API. In contrast,
if Rust allows APIs to perform that behavior on any type, without any sort of marker trait bound,
then types which don’t support that behavior cannot exist. I’m going to give three examples to show what I mean, each of which Rust has considered at different
points, though only the first one actually exists in Rust. One of the most famous anecdotes that forms the basis of the United States’ political self-identity
is the story of an interaction between Benjamin Franklin and Elizabeth Willing Powel after the
Constitutional Convention of 1787, which established the United States’ present form of government.
Powel asked Franklin what sort of government the U.S. was to have, to which he replied: “a
republic, if you can keep it.” Given the self-conscious references to “constitutions” and “checks and balances” in the Rust
project’s recent governance RFC and the discourse around it, some further reflection on
this quote and its implications about governance as such might now be appropriate for the project
and its community. I have been devoting a lot of my free time in the past month to thinking about structured
concurrency, and a blog post about that is coming soon, but first I want to revisit iterators and
generators. In a previous post, I wrote about one of the hardest problems for generators:
self-referential generators. Unlike the Future trait when we were designing async functions, the
Iterator trait is already stable, and it does not take a pinned reference to itself. This means an
Iterator cannot be self-referential. This is the first post in a series of posts about concurrency in Rust, and the different APIs that
can exist to support it. Unlike my recent series on control-flow effects, this series isn’t driving
toward any particular vision of what I think the Rust project should do. Instead, I am just trying
to publicly explore the problem space and build tools for thinking about the issues involved. I’m
not sure what the “right” concurrency API is. One of the main emphases of my recent posts has been that I believe shipping generators would solve
a lot of user problems by making it easy to write imperative iterative code, and especially to make
that iterative code interact well with asynchrony and fallibility as well. One thing that frustrates
me about the situation is that generators have been nearly ready to ship for years now, but very
little visible progress has been made. In particular, the core compiler transform to take a
generator and produce a state machine already exists, because it’s exactly how async functions are
implemented. In a previous post, I established the notion of “registers” - code in Rust can be
written in different registers, and it’s important to adequately support all registers. I
specifically discussed the low-level interface of the AsyncIterator trait, about which there is
currently a debate. The interface it currently has is a method called Why async Rust?
Thread-per-core
Send + 'static
, or worse yet Send + Sync + 'static
, which
just kills all the joy of actually writing Rust.Generic trait methods and new auto traits
Follow up to "Changing the rules of Rust"
!Leak
types from the new edition with code from the old edition that assumes values of all
types can be leaked.Changing the rules of Rust
A governance system, if you can keep it
Iterator, Generator
The Scoped Task trilemma
Generators
The AsyncIterator interface
poll_next
, which is a “poll”
method like Future::poll
. Poll methods are very “low-level” and are harder to write correctly than
async functions. Some people would like to see AsyncIterator
shifted to have an async next method,
simply the “asyncified” Iterator
trait.