Io uring: Difference between revisions

The core system calls of uring are wrapped by the C API of [https://github.com/axboe/liburing liburing]. Windows added a very similar interface, IoRing, in 2020. In my opinion, uring ought largely displace [[epoll]] in new Linux code. [[FreeBSD]] seems to be sticking with [[kqueue]], meaning code using uring won't run there, but neither did epoll (save through FreeBSD's somewhat dubious Linux compatibility layer). Both the system calls and liburing have fairly comprehensive man page coverage, including the [https://www.man7.org/linux/man-pages/man7/io_uring.7.html io_uring.7] top-level page.

Central to every uring are two ringbuffers holding CQEs (Completion Queue Entries) and SQE (Submission Queue Entries) descriptors (as best I can tell, this terminology was borrowed from the [https://nvmexpress.org/specifications/ NVMe specification]). Note that SQEs are allocated externally to the SQ descriptor ring (the same is not true for CQEs). SQEs roughly correspond to a single system call: they are tagged with an operation type, and filled in with the values that would traditionally be supplied as arguments to the appropriate function. Userspace is provided references to SQEs on the SQE ring, which it fills in and submits. Submission operates up through a specified SQE, and thus all SQEs before it in the ring must also be ready to go (this is likely the main reason why the SQ holds descriptors to an external ring of SQEs: you can acquire SQEs, and then submit them out of order). The kernel places results in the CQE ring. These rings are shared between kernel- and userspace. The rings must be distinct unless the kernel specifies the <tt>IORING_FEAT_SINGLE_MMAP</tt> feature (see below).

<tt>io_uring_setup(2)</tt> is wrapped by liburing's <tt>io_uring_queue_init(3)</tt> and <tt>io_uring_queue_init_params(3)</tt>. When using these wrappers, <tt>io_uring_queue_exit(3)</tt> should be used to clean up. These wrappers operate on a <tt>struct io_uring</tt>:

struct io_uring {                                                                                                                  

   struct io_uring_sq sq;                                                                                                          

   struct io_uring_cq cq;                                                                                                          

   unsigned flags;                                                                                                                  

   int ring_fd;                                                                                                                    

   unsigned features;                                                                                                              

   int enter_ring_fd;                                                                                                              

   __u8 int_flags;                                                                                                                  

   __u8 pad[3];                                                                                                                    

   unsigned pad2;                                                                                                                  

The details of ring structure are only relevant when using the low-level API, and they are not exposed via liburing. They're primarily used to prepare the three (or two, see <tt>IORING_FEAT_SINGLE_MMAP</tt>) backing memory maps. You'll need set these up yourself if you want to use [[Pages|huge pages]].

     __u32 flags;

     __u32 resv[3];

     __u32 flags;

     __u32 resv[3];

===Flags===

| IORING_SETUP_SQPOLL || 5.1 (5.11 for full features) || Create a kernel thread to poll on the submission queue. If the submission queue is kept busy, this thread will reap SQEs without the need for a system call. If enough time goes by without new submissions, the kernel thread goes to sleep, and <tt>io_uring_enter(2)</tt> must be called to wake it.

| IORING_SETUP_COOP_TASKRUN || 5.19 || Don't interrupt userspace processes to indicate CQE availability. It's usually desirable to allow events to be processed at arbitrary kernelspace transitions, in which case this flag can be provided to improve performance.

| IORING_SETUP_TASKRUN_FLAG || 5.19 || Requires <tt>IORING_SETUP_COOP_TASKRUN</tt>. When completions are pending awaiting processing, the <tt>IORING_SQ_TASKRUN</tt> flag will be set in the submission ring. This will be checked by <tt>io_uring_peek_cqe()</tt>, which will enter the kernel to process them.

| IORING_SETUP_DEFER_TASKRUN || 6.1 || Requires <tt>IORING_SETUP_SINGLE_ISSUER</tt>. Don't process completions at arbitrary kernel/scheduler transitions, but only <tt>io_uring_enter(2)</tt> when called with <tt>IORING_ENTER_GETEVENTS</tt> by the thread that submitted the SQEs.

! SQE flag !! Description

| <tt>IOSQE_FIXED_FILE</tt> || References a registered descriptor.

| <tt>IOSQE_IO_DRAIN</tt> || Issue after in-flight I/O.

| <tt>IOSQE_IO_LINK</tt> || Links next SQE.

| <tt>IOSQE_IO_HARDLINK</tt> || Same as <tt>IOSQE_IO_LINK</tt>, but a failure does not sever the chain.

| <tt>IOSQE_ASYNC</tt> || Always operate asynchronously.

| <tt>IOSQE_BUFFER_SELECT</tt> || Use a registered buffer.

| <tt>IOSQE_CQE_SKIP_SUCCESS</tt> || Don't post a CQE on success.

<tt>ioprio</tt> includes the following flags (most of which are useless when using the higher-level liburing API):

| <tt>IORING_RECVSEND_FIXED_BUF</tt> ||

If using the higher-level API, <tt>io_uring_get_sqe(3)</tt> is the primary means to acquiring an SQE. If none are available, it will return <tt>NULL</tt>, and you should submit outstanding SQEs to free one up.

Here's an example C++ data type that encodes eight bits as an operation type, eight bits as an index, and forty-eight bits as other data. I typically use something like this to reflect the operation which was used, the index into some relevant data structure, and other information about the operation (perhaps an offset or a length). Salt to taste, of course.

All of these liburing functions call the internal functions <tt>__io_uring_flush_sq()</tt> and <tt>__io_uring_submit()</tt>. The former updates the ring tail with a release-semantics write, while the latter calls <tt>io_uring_enter()</tt> if necessary. Note that timeouts are implemented internally using a SQE, and thus will kick off work if the submission ring is full pursuant to acquiring the entry.

Completions can be detected by four different means:

* Checking the completion queue speculatively. This either means a periodic check, which will suffer latency up to the period, or a busy check, which will churn CPU, but is probably the lowest-latency solution. This is best accomplished with <tt>io_uring_peek_cqe(3)</tt>, perhaps in conjunction with <tt>io_uring_cq_ready(3)</tt> (neither involves a system call).

* Using processor-dependent memory watch instructions. On x86, there's MONITOR+MWAIT, but they require you to be in ring 0, so you'd probably want UMONITOR/UMWAIT. This ought allow a very low-latency wake that consumes very little power.

It is possible for a single submission to result in multiple completions (e.g. <tt>io_uring_prep_multishot_accept(3)</tt>); this is known as ''multishot''. Errors on a multishot SQE will typically terminate the work request; a multishot SQE will set <tt>IORING_CQE_F_MORE</tt> high in generated CQEs so long as it remains active. A CQE without this flag indicates that the multishot is no longer operational, and must be reposted if further events are desired.

urings (and especially the <tt>struct io_uring</tt> object of liburing) are [https://github.com/axboe/liburing/issues/850#issuecomment-1526163360 not intended for multithreaded use] (quoth Axboe, "don't share a ring between threads"), though they can be used in several threaded paradigms. A single thread submitting and a single thread reaping is definitely supported. Descriptors can be sent among rings with <tt>IORING_OP_MSG_RING</tt>. Multiple submitters definitely must be serialized in userspace.

I am aware of no unannoying way to share some elements between threads in a single uring, while also monitoring distinct urings for each thread.

If you want to monitor both an epoll and a uring in a single thread without busy waiting, you will run into problems. You can't directly <tt>poll()</tt> a uring for CQE readiness, so it can't be added to your epoll watchset. If you set a zero timeout, you're busy waiting; if you set a non-zero timeout, one is dependent on the other's readiness. There are two solutions:

* Add the epoll fd to your uring with <tt>IORING_OP_POLL_ADD</tt>, and wait only for uring readiness. When you get a CQE for this submitted event, check the epoll.

Similarly, [[XDP]]'s native means of notification is via <tt>poll(2)</tt>; XDP can be unified with uring using either of these two methods.

I'd like to see [[signalfd|signalfds]] and pidfds integrated for purposes of linked operations (you can read from them with the existing infrastructure, but you can't create them, and thus can't link their creation to other events).

 

 

It's kind of annoying that chains can't extend over submissions. If I've got a lot of data I need delivered in order, it seems I'm limited to a single chain in-flight at a time, or else I risk out-of-order delivery due to one chain aborting, followed by items from a subsequent chain succeeding.

It would be awesome if I could indicate in each SQE some other uring to which the CQE ought be delivered, basically combining inter-ring messages and SQE submission. This would be the <i>ne plus ultra</i> IMHO of multithreaded uring (and be very much like my [[Libtorque|masters thesis]], not that I claim any inspiration whatsoever for uring).