Without boats, dreams dry up
In my previous post, I wrote about the distinction between “multi-task” and “intra-task” concurrency in async Rust. I want to open this post by considering a common pattern that users encounter, and how they might implement a solution using each technique.
Let’s call this “sub-tasking.” You have a unit of work that you need to perform, and you want to divide that unit into many smaller units of work, each of which can be run concurrently. This is intentionally extremely abstract: basically every program of any significance contains an instance of this pattern at least once (often many times), and the best solution will depend on the kind of work being done, how much work there is, the arity of concurrency, and so on.
Using multi-task concurrency, each smaller of work would be its own task. The user would spawn each of these tasks onto an executor. The results of the task would be collected with a synchronization primitive like a channel, or the tasks would be awaited together with a JoinSet.
Using intra-task concurrency, each smaller unit will be a future run concurrently within the same task. The user would construct all of the futures and then use a concurrency primitive like join! or select! to combine them into a single future, depending on the exact access pattern.
Each of these approaches has its advantages and disadvantages. Spawning multiple tasks requires that
each task be
'static, which means they cannot borrow data from their parent task. This is
often a very annoying limitation, not only because it might be costly to use shared ownership
Arc and possibly
Mutex), but also because even if it isn’t going to be problematic
in this context to use shared ownership,
(I’d love to see this change! Cheap shared ownership constructs like
Rc should have
non-affine semantics so you don’t have to call clone on them.)
When you join multiple futures, they can borrow from state outside of them within the same task, but as I wrote in the previous post, you can only join a static number of futures. Users that don’t want to deal with shared ownership but have a dynamic number of sub-tasks they need to execute are left searching for another solution. Enter FuturesUnordered.
FuturesUnordered is an odd duck of an abstraction from the futures library, which represents a
collection of futures as a
Stream (in std parlance, an
AsyncIterator). This makes it a lot like
JoinSet in surface appearance, but unlike
JoinSet the futures you push into it are not
spawned separately onto the executor, but are polled as the
FuturesUnordered is polled. Much like
spawning a task, every future pushed into
FuturesUnordered is separately allocated, so
representationally its very similar to multi-task concurrency. But because the
what polls each of these futures, they don’t execute independently and they don’t need to be
'static. They can borrow surrounding state as long as the
FuturesUnordered doesn’t outlive that
In a sense,
FuturesUnordered is a sort of hybrid between intra-task concurrency and multi-task
concurrency: you can borrow state from the same task like intra-task, but you can execute
arbitrarily many concurrent futures like multi-task. So it seems like a natural fit for the use case
I was just describing when the user wants that exact combination of features. But
has also been a culprit in some of the more frustrating bugs that users encounter when writing async
Rust. In the rest of this post, I want to investigate the reasons why that is.
In the early-to-mid 2010s, there was a renaissance in languages exploring new ways of doing concurrency. In the midst of this renaissance, one abstraction for achieving concurrent operations that was developed was the “future” or “promise” abstraction, which represented a unit of work that will maybe eventually complete, allowing the programmer to use this to manipulate control flow in their program. Building on this, syntactic sugar called “async/await” was introduced to take futures and shape them into the ordinary, linear control flow that is most common. This approach has been adopted in many mainstream languages, a series of developments that has been controversial among practitioners.
There are two excellent posts from that period which do a very good job of making the case for the two sides of the argument. I couldn’t more strongly recommend reading each these posts in full:
The thesis of Eriksen’s post is that futures provide a fundamentally different model of concurrency from threads. Threads provide a model in which all operations occur “synchronously,” because the execution of the program is modeled as a stack of function calls, which block when they need to wait for concurrently executing operations to complete. In contrast, by representing concurrent operations as asynchronously completing “futures,” the futures model enabled several advantages cited by Eriksen. These are the ones I find particularly compelling:
flatMap operator to chain many
concurrent network requests after one initial network request.
Nystrom takes the counter-position. He starts by imagining a language in which all functions are
. In his imaginary language, the important difference
between the two colors of function is that
functions can only be called from other
functions. He posits this distinction as a great frustration for users of the language,
because having to track two different kinds of functions is annoying and in his language
functions must be called using an annoyingly baroque syntax. Of course, what he’s referring to is
the difference between synchronous functions and asynchronous functions. Exactly what Eriksen cites
as an advantage of futures - that functions returning futures are different from functions that
don’t return futures - is for Nystrom it’s greatest weakness.
Some of the remarks Nystrom makes are not relevant to async Rust. For example, he says that if you call a function of one color as if it were a function of the other, dreadful things could happen:
When calling a function, you need to use the call that corresponds to its color. If you get it wrong … it does something bad. Dredge up some long-forgotten nightmare from your childhood like a clown with snakes for arms hiding under your bed. That jumps out of your monitor and sucks out your vitreous humour.
One of his main points is also that calling a
function is much more “painful” than
function. As Nystrom later elaborates in his post, he is referring to the
[Async/await] lets you make asynchronous calls just as easily as you can synchronous ones, with the tiny addition of a cute little keyword. You can nest
awaitcalls in expressions, use them in exception handling code, stuff them inside control flow.
Of course, he also says this, which is the crux of the argument about the “function coloring problem”:
But… you still have divided the world in two. Those async functions are easier to write, but they’re still async functions.
You’ve still got two colors. Async-await solves annoying rule #4: they make red functions not much worse to call than blue ones. But all of the other rules are still there.
Futures represent asynchronous operations differently from synchronous operations. For Eriksen, this provides additional affordances which are the key advantage of futures. For Nystrom, this is just an another hurdle to calling functions which return futures instead of blocking.
As you might expect if you’re familiar with this blog, I fall pretty firmly on the side of Eriksen. So it has not been easy on me to find that Nystrom’s views have been much more popular with the sort of people who comment on Hacker News or write angry, over-confident rants on the internet. A few months ago I wrote a post exploring the history of how Rust came to have the futures abstraction and async/await syntax on top of that, as well as a follow-up post describing the features I would like to see added to async Rust to make it easier to use.
Now I would like to take a step back and re-examine the design of async Rust in the context of this question about the utility of the futures model of concurrency. What has the use of futures actually gotten us in async Rust? I would like us to imagine that there could be a world in which the difficulties of using futures have been mitigated or resolved & the additional affordances they provide make async Rust not only just as easy to use as non-async Rust, but actually a better experience overall.…