iou version 0.3 released
Today I made a new release of the iou library, which contains idiomatic Rust bindings to the liburing library. This library allows users to manipulate the new io-uring interface for asynchronous IO on Linux. For more context, you can read my previous post on the first release of iou last year.
This new release greatly expands the API of iou, introduces some valuable improvements, and contains some breakages. I figured I would let this blog post serve as some basic release notes.
Changes to the Submission Queue
The submission queue side of things has seen the biggest API expansion. The most interesting change
has been that the
SQE type now can prepare many more types of events. This has been a big
shift in the io-uring interface over the past year: though it was originally intended for doing file
IO asynchronously, it now allows many different kinds of system calls to be passed through io-uring
instead of doing a blocking call. An application could now perform all or almost all of its
necessary IO using io-uring, from opening files to accepting connections to calls like
In addition, iou now provides a method for reserving multiple contiguous SQEs from the submission queue at once. The iterator of SQEs returned by this also supports adapters which return hard-linked and soft-linked sequences of SQEs.
Changes to the Completion Queue
The completion side of things has had a pretty substantial API shift, as
CQEs are no longer
references into the completion queue. Instead, the data from the queue is copied out and stored on
the stack (because a CQE is actually quite small: only two words). This makes the interface easier
to work with in several ways.
As a result, the CompletionQueue now also supports iteration APIs: the
cqes method and the
cqes_blocking method. These two iterators iterate over the completion queue
yielding CQEs, they only differ in how they behave when the completion queue has no outstanding
Changes to the Registrar
Another big change has been to the registrar API with the introduction of the
Registered type. When registering buffers or file descriptors, an iterator is
returned which gives the user a type representing a buffer or file descriptor that has been
registered with the kernel. These types provide a typesafe and guaranteed correct API for performing
Users can pass these
RegisteredBuf types to
SQE::prep_write, and the prep will automatically prepare an event for the pre-registered resource.
This avoids users having to remember that they want to use different op codes or set flags and other
The API has also been expanded in other ways, reflecting all of the work that has gone into the
io-uring interface underneath iou. The
Probe type allows users to investigate which
io-uring operations their kernel supports. Advanced use cases can use
(in addition the previously existing
SetupFlags) to configure an IoUring instance being
set up. The
provide_buffers feature is now supported by iou, for automatic
The future of iou & ringbahn
iou provide an idiomatic Rust API to the low level liburing library. ringbahn, built on top of iou, provides a much higher level, 100% safe API that integrates with the Rust async/await ecosystem. I hope to also release a version 0.1 of ringbahn in the next few days. I’ve also put on GitHub a crate called maglev, which is a demonstration of a driver for ringbahn that uses the io-uring interface in an efficient manner (making as few syscalls as possible).
maglev+ringbahn is designed to act just as a “reactor” (or “proactor”) built on io-uring; ideally it should work with any executor, but it integrates most well with the ecosystem around the smol crate because of the very open-ended way smol has been built. maglev + ringbahn, when completed, would be a replacement for smol’s async-io crate to run all of your IO on io-uring instead of epoll.
However, I will soon be starting a job doing work that is not open source. I am not sure if I will be able to continue to develop ringbahn and iou in my free time. I would be interested in chatting with other potential contributors about mentoring them to continue this work. I’ve created a GitHub organization to contain all of these crates.