iou: Rust bindings for liburing

Today I’m releasing a library called iou. This library provides idiomatic Rust bindings to the C library called liburing, which itself is a higher interface for interacting with the io_uring Linux kernel interface. Here are the answers to some questions I expect that may provoke.

What is io_uring?

io_uring is an interface added to the Linux kernel in version 5.1. Concurrent with that, the primary maintainer of that interface has also been publishing a library for interacting with it called liburing.

The io_uring interface is a new way of performing async IO on Linux. Unlike epoll, which is readiness based, io_uring is completion based: the user submits IO events to the kernel, which returns for the user when those events have completed. io_uring can be used for any kind of IO, including both files and network sockets.

The io_uring interface involves two queues (implementing as ring buffers; hence the name), the submission queue and the completion queue. The user’s program submits events by adding them to the submission queue, and the kernel adds finished events to the completion queue. This has performance advantages over a traditional epoll interface: used correctly, it requires fewer syscalls and memcpys than epoll would.

Does io_uring present a problem for Rust’s futures model?

This is a misunderstanding that I’ve seen repeated several times. It is wrong.

Completion based APIs are inherently difficult to make memory safe because they involve sharing memory between the user program and the kernel. Kernel APIs are opaque to us, we cannot statically typecheck that the borrow is not mutated until the completion completes. In particular, you cannot be guaranteed that a future will be held until the kernel completes because it could be dropped before the completion resolves.

This has been called the “completion/cancellation problem” but I want to emphasize that this problem is inherent to the nature of Rust’s memory model and completion based APIs and there’s nothing about the design of futures in my opinion that makes this problem worse or could alleviate it. The way Rust works is that borrows can be ended whenever the user desires by dropping the borrowing reference, and there’s no way to guarantee that a borrow would last until the completion ends.

Should I care about io_uring?

Maybe. But if you need to ask: probably not.

io_uring can be provide significant performance improvements, especially if you modify your code to take advantage of its “shared buffer” property and eliminate the memcpy between the kernel and userspace. But if you don’t stand a lot to gain by improving your IO throughput, why bother?

In fact, I’d say that many users don’t even need async IO at all. If you don’t have a good reason to think you need async IO, you probably don’t. And even though async/await make it much easier to deal with than the futures APIs alone, it’s still not without friction and pitfalls (especially around cooperative multitasking) that are not worth dealing with if you don’t need the performance.

However, one of Rust’s main areas of growth is users writing very high throughput or low latency network services who know that they do need absolute maximum performance. I hope by providing good APIs for dealing with io_uring, we can better support those sorts of users. But don’t go running to rewrite your code unless you have a good reason to think you need to.

What APIs does iou provide?

iou provides mostly the same APIs as liburing (some things are missing, and will be added eventually). However, it does so in a way that is memory safe and idiomatic Rust, unlike liburing, which is idiomatic C.

In particular, it provides an IoUring type, which enables you to set up and manage an io_uring instance. It also provides the ability to split the queues of the IoUring into separate types, so that you can mutate each of them without synchronizing with the other (for example, allowing you to process the completion queue on one thread while submitting to the submission queue from one or many other threads).

I’d recommend anyone who wants to experiment with io_uring use iou instead of trying to deal with liburing or the syscall interface directly. PRs are welcome to add any missing APIs to liburing.

Is iou a safe interface to io_uring?

iou is mostly safe, but not completely. The methods of SubmissionQueueEvent that you can use to prepare an actual IO event are unsafe. This is because you are responsible for guaranteeing that the kernel’s borrow of the buffer and file descriptor in these IO events is respected by your program.

Even though this is only a small part of the interface in iou, it ultimately means that you cannot use iou to complete IO without a bit of unsafe code.

I’m also working on a higher level library that integrates with futures and provides a completely safe API. But I thought I’d publish iou so that anyone else who wants to work on these kinds of APIs can experiment with iou.

How well tested is iou / is it correct and well documented?

I’ve only done basic smoke tests with iou to prove it can be used to actually read and write to the file system. I have read the source of liburing, some of the io_uring code in the kernel, and Jens Axboe’s notes on io_uring, and I believe the API is correct. However, it’s possible there are safety aspects I’ve overlooked, and its very possible my code contains bugs big and small. That’s why I’m releasing it: so that if other people who want to experiment with io_uring build on top of a single wrapper, we can be more sure that we have a solid foundation.

The library also has basically no documentation of its own. I would be very eager to review PRs which add documentation to some of the APIs in this crate.

What is the longer term future of iou?

I’ve published iou at version 0.1 and the sys bindings (uring-sys) at 1.0.0-alpha. Since liburing shouldn’t make breaking changes, I don’t expect uring-sys to make breaking changes either. I’ll let it live at a pre-release for a few weeks to receive any bug reports about APIs that are mistaken somehow, then move uring-sys to 1.0.0.

I would be excited to mentor other people to take over maintainership of both of these crates in the longer term. I don’t hope to be actively involved in maintaining this code once it is stable and off the ground.