Io uring

(52 intermediate revisions by the same user not shown)

Line 8:

* Both polling- and interrupt-driven I/O on the kernel side

The core system calls of uring are wrapped by the C API of [https://github.com/axboe/liburing liburing]. ~~Windows~~ added a very similar interface, IoRing, in 2020. In my opinion, uring ought largely displace [[epoll]] in new Linux code. [[FreeBSD]] seems to be sticking with [[kqueue]], meaning code using uring won't run there, but neither did epoll (save through FreeBSD's somewhat dubious Linux compatibility layer). Both the system calls and liburing have fairly comprehensive man page coverage, including the [https://www.man7.org/linux/man-pages/man7/io_uring.7.html io_uring.7] top-level page.

The core system calls of uring are wrapped by the C API of [https://github.com/axboe/liburing liburing]. Win32 added a very similar interface, [https://learn.microsoft.com/en-us/windows/win32/api/ioringapi/ IoRing], in 2020. In my opinion, uring ought largely displace [[epoll]] in new Linux code. [[FreeBSD]] seems to be sticking with [[kqueue]], meaning code using uring won't run there, but neither did epoll (save through FreeBSD's somewhat dubious Linux compatibility layer). Both the system calls and liburing have fairly comprehensive man page coverage, including the [https://www.man7.org/linux/man-pages/man7/io_uring.7.html io_uring.7] top-level page.

==Rings==

Central to every uring are two ringbuffers holding CQEs (Completion Queue Entries) and SQE (Submission Queue Entries) descriptors (as best I can tell, this terminology was borrowed from the [https://nvmexpress.org/specifications/ NVMe specification]). Note that SQEs are allocated externally to the SQ descriptor ring (the same is not true for CQEs). SQEs roughly correspond to a single system call: they are tagged with an operation type, and filled in with the values that would traditionally be supplied as arguments to the appropriate function. Userspace is provided references to SQEs on the SQE ring, which it fills in and submits. Submission operates up through a specified SQE, and thus all SQEs before it in the ring must also be ready to go (this is likely the main reason why the SQ holds descriptors to an external ring of SQEs: you can acquire SQEs, and then submit them out of order). The kernel places results in the CQE ring. These rings are shared between kernel- and userspace. The rings must be distinct unless the kernel specifies the <tt>IORING_FEAT_SINGLE_MMAP</tt> feature (see below).

Central to every uring are two ringbuffers holding CQEs (Completion Queue Entries) and SQE (Submission Queue Entries) descriptors (as best I can tell, this terminology was borrowed from the [https://nvmexpress.org/specifications/ NVMe specification]; I've also seen it with Mellanox NICs). Note that SQEs are allocated externally to the SQ descriptor ring (the same is not true for CQEs). SQEs roughly correspond to a single system call: they are tagged with an operation type, and filled in with the values that would traditionally be supplied as arguments to the appropriate function. Userspace is provided references to SQEs on the SQE ring, which it fills in and submits. Submission operates up through a specified SQE, and thus all SQEs before it in the ring must also be ready to go (this is likely the main reason why the SQ holds descriptors to an external ring of SQEs: you can acquire SQEs, and then submit them out of order, but see <tt>IORING_SETUP_NO_SQARRAY</tt>). The kernel places results in the CQE ring. These rings are shared between kernel- and userspace. The rings must be distinct unless the kernel specifies the <tt>IORING_FEAT_SINGLE_MMAP</tt> feature (see below).

uring does not generally make use of <tt>errno</tt>. Synchronous functions return the negative error code as their result. Completion queue entries have the negated error code placed in their <tt>res</tt> fields.

Line 57:

|-

| <tt>IORING_ENTER_REGISTERED_RING</tt> || <tt>ring_fd</tt> is an offset into the registered ring pool rather than a normal file descriptor.

|-

| <tt>IORING_ENTER_ABS_TIMER</tt> || (Since Linux 6.12) The timeout argument in the <tt>io_uring_getevents_arg</tt> is an absolute time, using the registered clock.

|-

|}

Line 81:

Line 83:

<tt>resv</tt> must be zeroed out. In the absence of flags, the uring uses interrupt-driven I/O. Calling <tt>close(2)</tt> on the returned descriptor frees all resources associated with the uring.

<tt>io_uring_setup(2)</tt> is wrapped by liburing's <tt>io_uring_queue_init(3)</tt> and <tt>io_uring_queue_init_params(3)</tt>. When using these wrappers, <tt>io_uring_queue_exit(3)</tt> should be used to clean up. ~~These wrappers operate on a <tt>struct io_uring</tt>:~~

<tt>io_uring_setup(2)</tt> is wrapped by liburing's <tt>io_uring_queue_init(3)</tt> and <tt>io_uring_queue_init_params(3)</tt>. When using these wrappers, <tt>io_uring_queue_exit(3)</tt> should be used to clean up.

struct io_uring {

int io_uring_queue_init(unsigned entries, struct io_uring *ring, unsigned flags);

struct io_uring_sq sq;

int io_uring_queue_init_params(unsigned entries, struct io_uring *ring, struct io_uring_params *params);

struct io_uring_cq cq;

void io_uring_queue_exit(struct io_uring *ring);

unsigned flags;

</syntaxhighlight>

int ring_fd;

unsigned features;

These wrappers operate on a <tt>struct io_uring</tt>:

int enter_ring_fd;

__u8 int_flags;

__u8 pad[3];

struct io_uring {

unsigned pad2;

struct io_uring_sq sq;

struct io_uring_cq cq;

unsigned flags;

int ring_fd;

unsigned features;

int enter_ring_fd;

__u8 int_flags;

__u8 pad[3];

unsigned pad2;

};

struct io_uring_sq {

unsigned *khead;

unsigned *ktail;

unsigned *kring_mask; // Deprecated: use `ring_mask`

unsigned *kring_entries; // Deprecated: use `ring_entries`

unsigned *kflags;

unsigned *kdropped;

unsigned *array;

struct io_uring_sqe *sqes;

unsigned sqe_head;

unsigned sqe_tail;

size_t ring_sz;

void *ring_ptr;

unsigned ring_mask;

unsigned ring_entries;

unsigned pad[2];

};

struct io_uring_cq {

unsigned *khead;

unsigned *ktail;

unsigned *kring_mask; // Deprecated: use `ring_mask`

unsigned *kring_entries; // Deprecated: use `ring_entries`

unsigned *kflags;

unsigned *koverflow;

struct io_uring_cqe *cqes;

size_t ring_sz;

void *ring_ptr;

unsigned ring_mask;

unsigned ring_entries;

unsigned pad[2];

};

</syntaxhighlight>

Line 99:

Line 142:

<tt>io_uring_queue_init(3)</tt> takes an <tt>unsigned flags</tt> argument, which is passed as the <tt>flags</tt> field of <tt>io_uring_params</tt>. <tt>io_uring_queue_init_params(3)</tt> takes a <tt>struct io_uring_params*</tt> argument, which is passed through directly to <tt>io_uring_setup(2)</tt>. It's best to avoid mixing the low-level API and that provided by liburing.

===Ring structure===

The details of ring structure are only relevant when using the low-level API~~, and they~~ are ~~not exposed via liburing. They're primarily~~ used to prepare the three (or two, see <tt>IORING_FEAT_SINGLE_MMAP</tt>) backing memory maps~~. You'll need set these up yourself if you want to use [[Pages|huge pages]]~~.

The details of ring structure are typically only relevant when using the low-level API. Two offset structs are used to prepare and control the three (or two, see <tt>IORING_FEAT_SINGLE_MMAP</tt>) backing memory maps.

struct io_sqring_offsets {

Line 106:

Line 149:

__u32 ring_mask;

__u32 ring_entries;

__u32 flags;

__u32 flags; // IORING_SQ_*

__u32 dropped;

__u32 array;

__u32 ~~resv[3]~~;

__u32 resv1;

__u64 user_addr; // sqe mmap for IORING_SETUP_NO_MMAP

};

Line 119:

Line 163:

__u32 overflow;

__u32 cqes;

__u32 flags;

__u32 flags; // IORING_CQ_*

__u32 ~~resv[3]~~;

__u32 resv1;

__u64 user_addr; // sq/cq mmap for IORING_SETUP_NO_MMAP

};

</syntaxhighlight>

Line 172:

Line 217:

|-

| IORING_SETUP_DEFER_TASKRUN || 6.1 || Requires <tt>IORING_SETUP_SINGLE_ISSUER</tt>. Don't process completions at arbitrary kernel/scheduler transitions, but only <tt>io_uring_enter(2)</tt> when called with <tt>IORING_ENTER_GETEVENTS</tt> by the thread that submitted the SQEs. Cannot be used with <tt>IORING_SETUP_SQPOLL</tt>.

|-

| IORING_SETUP_NO_MMAP || 6.5 || The caller will provide two memory regions (the queue rings in <tt>cq_off.user_data</tt>, the SQEs in <tt>sq_off.user_data</tt>) rather than having them allocated by the kernel (they may both come from a single map). This allows for the use of huge pages.

|-

| IORING_SETUP_REGISTERED_FD_ONLY || 6.6 || Registers the ring fd for use with <tt>IORING_REGISTER_USE_REGISTERED_RING</tt>. Returns a registered fd index.

|-

| IORING_SETUP_NO_SQARRAY || 6.6 || Do not use an indirection array for the submission queue, instead submitting them strictly in order.

|-

|}

Line 213:

Line 264:

|-

| <tt>IORING_FEAT_REG_REG_RING</tt> || 6.3 || Ring descriptors can be registered with <tt>IORING_REGISTER_USE_REGISTERED_RING</tt>.

|-

| <tt>IORING_FEAT_RECVSEND_BUNDLE</tt> || 6.10 || <tt>send</tt> and <tt>recv</tt> operations can be bundled.

|-

| <tt>IORING_FEAT_MIN_TIMEOUT</tt> || 6.12 || A minimum batch wait timeout is supported.

|-

|}

Line 312:

Line 367:

|-

| <tt>IORING_UNREGISTER_PERSONALITY</tt> || 5.6 ||

|-

|}

====NAPI settings====

{| class="wikitable" border="1"

! Flag !! Kernel !! Description

|-

| <tt>IORING_REGISTER_NAPI</tt> || ||

|-

| <tt>IORING_UNREGISTER_PERSONALITY</tt> || ||

|-

|}

Line 358:

Line 422:

|-

|}

<tt>ioprio</tt> includes the following flags ~~(most of which are useless when using the higher-level liburing API)~~:

<tt>ioprio</tt> includes the following flags:

{| class="wikitable" border="1"

! SQE ioprio flag !! Description

|-

| <tt>IORING_RECV_MULTISHOT</tt> ||

|-

| <tt>IORING_SEND_MULTISHOT</tt> || Introduced in 6.8.

|-

| <tt>IORING_ACCEPT_MULTISHOT</tt> ||

Line 373:

Line 439:

===Acquiring SQEs===

If using the higher-level API, <tt>io_uring_get_sqe(3)</tt> is the primary means to ~~acquiring~~ an SQE. If none are available, it will return <tt>NULL</tt>, and you should submit outstanding SQEs to free one up.

If using the higher-level API, <tt>io_uring_get_sqe(3)</tt> is the primary means to acquire an SQE. If none are available, it will return <tt>NULL</tt>, and you should submit outstanding SQEs to free one up.

struct io_uring_sqe *io_uring_get_sqe(struct io_uring *ring);

Line 390:

Line 456:

__u64 io_uring_cqe_get_data64(struct io_uring_cqe *cqe);

</syntaxhighlight>

Here's an example C++ data type that encodes eight bits as an operation type, eight bits as an index, and forty-eight bits as other data. I typically use something like this to reflect the operation which was used, the index into some relevant data structure, and other information about the operation (perhaps an offset or a length). Salt to taste, of course.

Here's an example [[C++]] data type that encodes eight bits as an operation type, eight bits as an index, and forty-eight bits as other data. I typically use something like this to reflect the operation which was used, the index into some relevant data structure, and other information about the operation (perhaps an offset or a length). Salt to taste, of course.

union URingCtx {

Line 435:

Line 501:

====Opening and closing file descriptors====

// IORING_OP_OPENAT (5.15)

void io_uring_prep_openat(struct io_uring_sqe *sqe, int dfd, const char *path,

int flags, mode_t mode);

void io_uring_prep_openat_direct(struct io_uring_sqe *sqe, int dfd, const char *path,

int flags, mode_t mode, unsigned file_index);

// IORING_OP_OPENAT2 (5.15)

void io_uring_prep_openat2(struct io_uring_sqe *sqe, int dfd, const char *path,

int flags, struct open_how *how);

Line 452:

Line 520:

void io_uring_prep_multishot_accept_direct(struct io_uring_sqe *sqe, int sockfd, struct sockaddr *addr,

socklen_t *addrlen, int flags);

// IORING_OP_CLOSE (5.6, direct support since 5.15)

void io_uring_prep_close(struct io_uring_sqe *sqe, int fd);

void io_uring_prep_close_direct(struct io_uring_sqe *sqe, unsigned file_index);

// IORING_OP_SOCKET (5.19)

void io_uring_prep_socket(struct io_uring_sqe *sqe, int domain, int type,

int protocol, unsigned int flags);

Line 467:

Line 537:

// IORING_OP_SEND (5.6)

void io_uring_prep_send(struct io_uring_sqe *sqe, int sockfd, const void *buf, size_t len, int flags);

// IORING_OP_SEND_ZC (6.0)

void io_uring_prep_send_zc(struct io_uring_sqe *sqe, int sockfd, const void *buf, size_t len, int flags, int zc_flags);

// IORING_OP_SENDMSG (5.3)

Line 477:

Line 548:

void io_uring_prep_recvmsg(struct io_uring_sqe *sqe, int fd, struct msghdr *msg, unsigned flags);

void io_uring_prep_recvmsg_multishot(struct io_uring_sqe *sqe, int fd, struct msghdr *msg, unsigned flags);

// IORING_OP_READ (5.6)

void io_uring_prep_read(struct io_uring_sqe *sqe, int fd, void *buf, unsigned nbytes, __u64 offset);

// IORING_OP_READV

~~void io_uring_prep_read(struct io_uring_sqe *sqe, int fd, void *buf, unsigned nbytes, __u64 offset);~~

void io_uring_prep_readv(struct io_uring_sqe *sqe, int fd, const struct iovec *iovecs, unsigned nr_vecs, __u64 offset);

void io_uring_prep_readv2(struct io_uring_sqe *sqe, int fd, const struct iovec *iovecs,

Line 489:

Line 561:

// IORING_OP_SYNC_FILE_RANGE (5.2)

void io_uring_prep_sync_file_range(struct io_uring_sqe *sqe, int fd, unsigned len, __u64 offset, int flags);

// IORING_OP_TEE (5.8)

void io_uring_prep_tee(struct io_uring_sqe *sqe, int fd_in, int fd_out, unsigned int nbytes, unsigned int splice_flags);

// IORING_OP_WRITE (5.6)

void io_uring_prep_write(struct io_uring_sqe *sqe, int fd, const void *buf, unsigned nbytes, __u64 offset);

// IORING_OP_WRITEV

~~void io_uring_prep_write(struct io_uring_sqe *sqe, int fd, const void *buf, unsigned nbytes, __u64 offset);~~

void io_uring_prep_writev(struct io_uring_sqe *sqe, int fd, const struct iovec *iovecs,

unsigned nr_vecs, __u64 offset);

Line 501:

Line 575:

// IORING_OP_CONNECT (5.5)

void io_uring_prep_connect(struct io_uring_sqe *sqe, int sockfd, const struct sockaddr *addr, socklen_t addrlen);

</syntaxhighlight>

====Operations on file descriptors====

// IORING_OP_URING_CMD

void io_uring_prep_cmd_sock(struct io_uring_sqe *sqe, int cmd_op, int fd, int level,

int optname, void *optval, int optlen);

// IORING_OP_GETXATTR

void io_uring_prep_getxattr(struct io_uring_sqe *sqe, const char *name, char *value,

const char *path, unsigned int len);

// IORING_OP_SETXATTR

void io_uring_prep_setxattr(struct io_uring_sqe *sqe, const char *name, const char *value,

const char *path, int flags, unsigned int len);

// IORING_OP_FGETXATTR

void io_uring_prep_fgetxattr(struct io_uring_sqe *sqe, int fd, const char *name,

char *value, unsigned int len);

// IORING_OP_FSETXATTR

void io_uring_prep_fsetxattr(struct io_uring_sqe *sqe, int fd, const char *name,

const char *value, int flags, unsigned int len);

</syntaxhighlight>

Line 541:

Line 634:

All transring messages are prepared using the target uring's file descriptor, and thus aren't particularly compatible with the higher-level liburing API (of course, you can just yank <tt>ring_fd</tt> out of <tt>struct iouring</tt>). That's unfortunate, because this functionality is very useful in a multithreaded environment. Note that the ring to which the SQE is submitted can itself be the target of the message!

// IORING_OP_MSG_RING

void io_uring_prep_msg_ring(struct io_uring_sqe *sqe, int fd, unsigned int len, __u64 data, unsigned int flags);

void io_uring_prep_msg_ring_cqe_flags(struct io_uring_sqe *sqe, int fd, unsigned int len, __u64 data, unsigned int flags, unsigned int cqe_flags);

void io_uring_prep_msg_ring_fd(struct io_uring_sqe *sqe, int fd, int source_fd, int target_fd, __u64 data, unsigned int flags);

void io_uring_prep_msg_ring_fd_alloc(struct io_uring_sqe *sqe, int fd, int source_fd, __u64 data, unsigned int flags);

</syntaxhighlight>

====Futexes====

// IORING_OP_FUTEX_WAKE

void io_uring_prep_futex_wake(struct io_uring_sqe *sqe, uint32_t *futex, uint64_t val,

uint64_t mask, uint32_t futex_flags, unsigned int flags);

// IORING_OP_FUTEX_WAIT

void io_uring_prep_futex_wait(struct io_uring_sqe *sqe, uint32_t *futex, uint64_t val,

uint64_t mask, uint32_t futex_flags, unsigned int flags);

// IORING_OP_FUTEX_WAITV

void io_uring_prep_futex_waitv(struct io_uring_sqe *sqe, struct futex_waitv *futex,

uint32_t nr_futex, unsigned int flags);

</syntaxhighlight>

Line 633:

Line 740:

|-

| <tt>IORING_CQE_F_NOTIF</tt> || Notification CQE for zero-copy sends

|-

| <tt>IORING_CQE_F_BUF_MORE</tt> || There will be more CQEs to the specified registered buffer

|-

|}

Completions can be detected by at least ~~four~~ different means:

Completions can be detected by at least five different means:

* Checking the completion queue speculatively. This either means a periodic check, which will suffer latency up to the period, or a busy check, which will churn CPU, but is probably the lowest-latency solution. This is best accomplished with <tt>io_uring_peek_cqe(3)</tt>, perhaps in conjunction with <tt>io_uring_cq_ready(3)</tt> (neither involves a system call).

* Checking the completion queue speculatively. This either means a periodic check, which will suffer latency up to the period, or a busy check, which will churn CPU, but is probably the lowest-latency portable solution. This is best accomplished with <tt>io_uring_peek_cqe(3)</tt>, perhaps in conjunction with <tt>io_uring_cq_ready(3)</tt> (neither involves a system call).

int io_uring_peek_cqe(struct io_uring *ring, struct io_uring_cqe **cqe_ptr);

Line 653:

Line 762:

</syntaxhighlight>

* Using an [[eventfd]] together with <tt>io_uring_register_eventfd(3)</tt>. See below for the full API. This eventfd can be combined with e.g. regular [[epoll]].

* Polling the uring's file descriptor directly.

* Using processor-dependent memory watch instructions. On x86, there's [https://www.felixcloutier.com/x86/monitor MONITOR]+[https://www.felixcloutier.com/x86/mwait MWAIT], but they require you to be in ring 0, so you'd probably want [https://www.felixcloutier.com/x86/umonitor UMONITOR]+[https://www.felixcloutier.com/x86/umwait UMWAIT]. This ought allow a very low-latency wake that consumes very little power.

Line 680:

Line 790:

===Multithreaded use===

urings (and especially the <tt>struct io_uring</tt> object of liburing) are [https://github.com/axboe/liburing/issues/850#issuecomment-1526163360 not intended for multithreaded use] (quoth Axboe, "don't share a ring between threads"), though they can be used in several threaded paradigms. A single thread submitting and a single thread reaping is definitely supported. Descriptors can be sent among rings with <tt>IORING_OP_MSG_RING</tt>. Multiple submitters definitely must be serialized in userspace.

urings (and especially the <tt>struct io_uring</tt> object of liburing) are [https://github.com/axboe/liburing/issues/850#issuecomment-1526163360 not intended for multithreaded use] (quoth Axboe, "don't share a ring between threads"), though they can be used in several threaded paradigms. A single thread submitting and some different single thread reaping is definitely supported. Descriptors can be sent among rings with <tt>IORING_OP_MSG_RING</tt>. Multiple submitters definitely must be serialized in userspace.

If an op will be completed via a kernel task, the thread that submitted that SQE must remain alive until the op's completion. It will otherwise error out with <tt>-ECANCELED</tt>. If you must submit the SQE from a thread which will die, consider creating it disabled (see <tt>IORING_SETUP_R_DISABLED</tt>), and enabling it from the thread which will reap the completion event using <tt>IORING_REGISTER_ENABLE_RINGS</tt> with <tt>io_uring_register(2)</tt>.

Line 688:

Line 798:

When <tt>IORING_SETUP_SQPOLL</tt> is used, the kernel poller thread is considered to have performed the submission, providing another possible way around this problem.

I am aware of no unannoying way to share some elements between threads in a single uring, while also monitoring distinct urings for each thread.

I am aware of no unannoying way to share some elements between threads in a single uring, while also monitoring distinct urings for each thread. I suppose you could poll on both.

===Coexistence with [[epoll]]/[[XDP]]===

If you want to monitor both an epoll and a uring in a single thread without busy waiting, you will run into problems~~. You can't directly <tt>poll()</tt> a uring for CQE readiness, so it can't be added to your epoll watchset~~. If you set a zero timeout, you're busy waiting; if you set a non-zero timeout, one is dependent on the other's readiness. There are ~~two~~ solutions:

If you want to monitor both an epoll and a uring in a single thread without busy waiting, you will run into problems. If you set a zero timeout, you're busy waiting; if you set a non-zero timeout, one is dependent on the other's readiness. There are three solutions:

* Add the epoll fd to your uring with <tt>IORING_OP_POLL_ADD</tt> (ideally multishot), and wait only for uring readiness. When you get a CQE for this submitted event, check the epoll.

* Poll on the uring's file descriptor directly for completion queue events.

* Register an [[eventfd]] with your uring with <tt>io_uring_register_eventfd(3)</tt>, add that to your epoll, and when you get <tt>POLLIN</tt> for this fd, check the completion ring.

The full API here is:

Line 702:

Line 813:

<tt>io_uring_register_eventfd_async(3)</tt> only posts to the eventfd for events that completed out-of-line. There is not necessarily a bijection between completion events and posts even with the regular form; multiple CQEs can post only a single event, and spurious posts can occur.

Similarly, [[XDP]]'s native means of notification is via <tt>poll(2)</tt>; XDP can be unified with uring using ~~either~~ of these ~~two~~ methods.

Similarly, [[XDP]]'s native means of notification is via <tt>poll(2)</tt>; XDP can be unified with uring using any of these methods.

===Queue overflows===

Line 716:

Line 827:

==What's missing==

I'd like to see [[signalfd|signalfds]] and pidfds integrated for purposes of linked operations (you can read from them with the existing infrastructure, but you can't create them, and thus can't link their creation to other events).

I'd like to see [[signalfd|signalfds]] and [[Pidfd|pidfds]] integrated for purposes of linked operations (you can read from them with the existing infrastructure, but you can't create them, and thus can't link their creation to other events).

Why is there no <tt>vmsplice(2)</tt> action? How about <tt>fork(2)</tt>/<tt>pthread_create(3)</tt> (or more generally <tt>clone(2)</tt>)? We could have coroutines with I/O.

It would be nice to have tight integration with condition variables or even mutex/futex (allow me to submit a request to get a lock, and when i get the CQE, i have that lock). Bonus points if the fast (uncontended) path never needs a system call (like mutexes built atop futexes today).

It's kind of annoying that chains can't extend over submissions. If I've got a lot of data I need delivered in order, it seems I'm limited to a single chain in-flight at a time, or else I risk out-of-order delivery due to one chain aborting, followed by items from a subsequent chain succeeding. <tt>recv_multishot</tt> can help with this.

I understand that making huge pages as unpleasant as possible to use is a central tenet (perhaps the central tenet) underpinning the entire Linux mission, but it would be great if hugepage-backed urings were only a flag/field away.

It's kind of annoying that chains can't extend over submissions. If I've got a lot of data I need delivered in order, it seems I'm limited to a single chain in-flight at a time, or else I risk out-of-order delivery due to one chain aborting, followed by items from a subsequent chain succeeding.

It seems that the obvious next step would be providing small snippets of arbitrary computation to be run in kernelspace, linked with SQEs. Perhaps [[eBPF]] would be a good starting place.

It would be awesome if I could indicate in each SQE some other uring to which the CQE ought be delivered, basically combining inter-ring messages and SQE submission. This would be the ne plus ultra IMHO of multithreaded uring (and be very much like my [[Libtorque|masters thesis]], not that I claim any inspiration whatsoever for uring).

It would be awesome if I could indicate in each SQE some other uring to which the CQE ought be delivered, basically combining inter-ring messages and SQE submission. This would be the ne plus ultra IMHO of multithreaded uring (and be very much like my [[Libtorque|masters thesis]], not that I claim any inspiration/precedence whatsoever for uring).

==See also==

Line 748:

Line 855:

** "[https://lwn.net/Articles/863071/ Discriptorless files for io_uring]" 2021-07-19

** "[https://lwn.net/Articles/879724/ Zero-copy network transmission with io_uring]" 2021-12-30

** "[https://lwn.net/Articles/1002371/ Process creation in io_uring]" 2024-12-20

[[CATEGORY: Networking]]

Io uring: Difference between revisions

Dank (talk | contribs)

@@ Line 8: / Line 8: @@
 * Both polling- and interrupt-driven I/O on the kernel side
-The core system calls of uring are wrapped by the C API of [https://github.com/axboe/liburing liburing]. Windows added a very similar interface, IoRing, in 2020. In my opinion, uring ought largely displace [[epoll]] in new Linux code. [[FreeBSD]] seems to be sticking with [[kqueue]], meaning code using uring won't run there, but neither did epoll (save through FreeBSD's somewhat dubious Linux compatibility layer). Both the system calls and liburing have fairly comprehensive man page coverage, including the [https://www.man7.org/linux/man-pages/man7/io_uring.7.html io_uring.7] top-level page.
+The core system calls of uring are wrapped by the C API of [https://github.com/axboe/liburing liburing]. Win32 added a very similar interface, [https://learn.microsoft.com/en-us/windows/win32/api/ioringapi/ IoRing], in 2020. In my opinion, uring ought largely displace [[epoll]] in new Linux code. [[FreeBSD]] seems to be sticking with [[kqueue]], meaning code using uring won't run there, but neither did epoll (save through FreeBSD's somewhat dubious Linux compatibility layer). Both the system calls and liburing have fairly comprehensive man page coverage, including the [https://www.man7.org/linux/man-pages/man7/io_uring.7.html io_uring.7] top-level page.
 ==Rings==
-Central to every uring are two ringbuffers holding CQEs (Completion Queue Entries) and SQE (Submission Queue Entries) descriptors (as best I can tell, this terminology was borrowed from the [https://nvmexpress.org/specifications/ NVMe specification]). Note that SQEs are allocated externally to the SQ descriptor ring (the same is not true for CQEs). SQEs roughly correspond to a single system call: they are tagged with an operation type, and filled in with the values that would traditionally be supplied as arguments to the appropriate function. Userspace is provided references to SQEs on the SQE ring, which it fills in and submits. Submission operates up through a specified SQE, and thus all SQEs before it in the ring must also be ready to go (this is likely the main reason why the SQ holds descriptors to an external ring of SQEs: you can acquire SQEs, and then submit them out of order). The kernel places results in the CQE ring. These rings are shared between kernel- and userspace. The rings must be distinct unless the kernel specifies the <tt>IORING_FEAT_SINGLE_MMAP</tt> feature (see below).
+Central to every uring are two ringbuffers holding CQEs (Completion Queue Entries) and SQE (Submission Queue Entries) descriptors (as best I can tell, this terminology was borrowed from the [https://nvmexpress.org/specifications/ NVMe specification]; I've also seen it with Mellanox NICs). Note that SQEs are allocated externally to the SQ descriptor ring (the same is not true for CQEs). SQEs roughly correspond to a single system call: they are tagged with an operation type, and filled in with the values that would traditionally be supplied as arguments to the appropriate function. Userspace is provided references to SQEs on the SQE ring, which it fills in and submits. Submission operates up through a specified SQE, and thus all SQEs before it in the ring must also be ready to go (this is likely the main reason why the SQ holds descriptors to an external ring of SQEs: you can acquire SQEs, and then submit them out of order, but see <tt>IORING_SETUP_NO_SQARRAY</tt>). The kernel places results in the CQE ring. These rings are shared between kernel- and userspace. The rings must be distinct unless the kernel specifies the <tt>IORING_FEAT_SINGLE_MMAP</tt> feature (see below).
 uring does not generally make use of <tt>errno</tt>. Synchronous functions return the negative error code as their result. Completion queue entries have the negated error code placed in their <tt>res</tt> fields.
@@ Line 57: / Line 57: @@
 |-
 | <tt>IORING_ENTER_REGISTERED_RING</tt> || <tt>ring_fd</tt> is an offset into the registered ring pool rather than a normal file descriptor.
+|-
+| <tt>IORING_ENTER_ABS_TIMER</tt> || (Since Linux 6.12) The timeout argument in the <tt>io_uring_getevents_arg</tt> is an absolute time, using the registered clock.
 |-
 |}
@@ Line 81: / Line 83: @@
 <tt>resv</tt> must be zeroed out. In the absence of flags, the uring uses interrupt-driven I/O. Calling <tt>close(2)</tt> on the returned descriptor frees all resources associated with the uring.
-<tt>io_uring_setup(2)</tt> is wrapped by liburing's <tt>io_uring_queue_init(3)</tt> and <tt>io_uring_queue_init_params(3)</tt>. When using these wrappers, <tt>io_uring_queue_exit(3)</tt> should be used to clean up. These wrappers operate on a <tt>struct io_uring</tt>:
+<tt>io_uring_setup(2)</tt> is wrapped by liburing's <tt>io_uring_queue_init(3)</tt> and <tt>io_uring_queue_init_params(3)</tt>. When using these wrappers, <tt>io_uring_queue_exit(3)</tt> should be used to clean up.
 <syntaxhighlight lang="c">
-struct io_uring {
+int io_uring_queue_init(unsigned entries, struct io_uring *ring, unsigned flags);
-   struct io_uring_sq sq;
+int io_uring_queue_init_params(unsigned entries, struct io_uring *ring, struct io_uring_params *params);
-   struct io_uring_cq cq;
+void io_uring_queue_exit(struct io_uring *ring);
-   unsigned flags;
+</syntaxhighlight>
-   int ring_fd;
-   unsigned features;
+These wrappers operate on a <tt>struct io_uring</tt>:
-   int enter_ring_fd;
-   __u8 int_flags;
+<syntaxhighlight lang="c">
-   __u8 pad[3];
+struct io_uring {
-   unsigned pad2;
+   struct io_uring_sq sq;
+   struct io_uring_cq cq;
+   unsigned flags;
+   int ring_fd;
+   unsigned features;
+   int enter_ring_fd;
+   __u8 int_flags;
+   __u8 pad[3];
+   unsigned pad2;
+};
+struct io_uring_sq {
+  unsigned *khead;
+  unsigned *ktail;
+  unsigned *kring_mask;     // Deprecated: use `ring_mask`
+  unsigned *kring_entries;  // Deprecated: use `ring_entries`
+  unsigned *kflags;
+  unsigned *kdropped;
+  unsigned *array;
+  struct io_uring_sqe *sqes;
+  unsigned sqe_head;
+  unsigned sqe_tail;
+  size_t ring_sz;
+  void *ring_ptr;
+  unsigned ring_mask;
+  unsigned ring_entries;
+  unsigned pad[2];
+};
+struct io_uring_cq {
+  unsigned *khead;
+  unsigned *ktail;
+  unsigned *kring_mask;     // Deprecated: use `ring_mask`
+  unsigned *kring_entries;  // Deprecated: use `ring_entries`
+  unsigned *kflags;
+  unsigned *koverflow;
+  struct io_uring_cqe *cqes;
+  size_t ring_sz;
+  void *ring_ptr;
+  unsigned ring_mask;
+  unsigned ring_entries;
+  unsigned pad[2];
 };
 </syntaxhighlight>
@@ Line 99: / Line 142: @@
 <tt>io_uring_queue_init(3)</tt> takes an <tt>unsigned flags</tt> argument, which is passed as the <tt>flags</tt> field of <tt>io_uring_params</tt>. <tt>io_uring_queue_init_params(3)</tt> takes a <tt>struct io_uring_params*</tt> argument, which is passed through directly to <tt>io_uring_setup(2)</tt>. It's best to avoid mixing the low-level API and that provided by liburing.
 ===Ring structure===
-The details of ring structure are only relevant when using the low-level API, and they are not exposed via liburing. They're primarily used to prepare the three (or two, see <tt>IORING_FEAT_SINGLE_MMAP</tt>) backing memory maps. You'll need set these up yourself if you want to use [[Pages|huge pages]].
+The details of ring structure are typically only relevant when using the low-level API. Two offset structs are used to prepare and control the three (or two, see <tt>IORING_FEAT_SINGLE_MMAP</tt>) backing memory maps.
 <syntaxhighlight lang="c">
 struct io_sqring_offsets {
@@ Line 106: / Line 149: @@
      __u32 ring_mask;
      __u32 ring_entries;
-     __u32 flags;
+     __u32 flags;                 // IORING_SQ_*
      __u32 dropped;
      __u32 array;
-     __u32 resv[3];
+     __u32 resv1;
+    __u64 user_addr;             // sqe mmap for IORING_SETUP_NO_MMAP
 };
@@ Line 119: / Line 163: @@
      __u32 overflow;
      __u32 cqes;
-     __u32 flags;
+     __u32 flags;                 // IORING_CQ_*
-     __u32 resv[3];
+     __u32 resv1;
+    __u64 user_addr;             // sq/cq mmap for IORING_SETUP_NO_MMAP
 };
 </syntaxhighlight>
@@ Line 172: / Line 217: @@
 |-
 | IORING_SETUP_DEFER_TASKRUN || 6.1 || Requires <tt>IORING_SETUP_SINGLE_ISSUER</tt>. Don't process completions at arbitrary kernel/scheduler transitions, but only <tt>io_uring_enter(2)</tt> when called with <tt>IORING_ENTER_GETEVENTS</tt> by the thread that submitted the SQEs. Cannot be used with <tt>IORING_SETUP_SQPOLL</tt>.
+|-
+| IORING_SETUP_NO_MMAP || 6.5 || The caller will provide two memory regions (the queue rings in <tt>cq_off.user_data</tt>, the SQEs in <tt>sq_off.user_data</tt>) rather than having them allocated by the kernel (they may both come from a single map). This allows for the use of huge pages.
+|-
+| IORING_SETUP_REGISTERED_FD_ONLY || 6.6 || Registers the ring fd for use with <tt>IORING_REGISTER_USE_REGISTERED_RING</tt>. Returns a registered fd index.
+|-
+| IORING_SETUP_NO_SQARRAY || 6.6 || Do not use an indirection array for the submission queue, instead submitting them strictly in order.
 |-
 |}
@@ Line 213: / Line 264: @@
 |-
 | <tt>IORING_FEAT_REG_REG_RING</tt> || 6.3 || Ring descriptors can be registered with <tt>IORING_REGISTER_USE_REGISTERED_RING</tt>.
+|-
+| <tt>IORING_FEAT_RECVSEND_BUNDLE</tt> || 6.10 || <tt>send</tt> and <tt>recv</tt> operations can be bundled.
+|-
+| <tt>IORING_FEAT_MIN_TIMEOUT</tt> || 6.12 || A minimum batch wait timeout is supported.
 |-
 |}
@@ Line 312: / Line 367: @@
 |-
 | <tt>IORING_UNREGISTER_PERSONALITY</tt> || 5.6 ||
+|-
+|}
+====NAPI settings====
+{| class="wikitable" border="1"
+! Flag !! Kernel !! Description
+|-
+| <tt>IORING_REGISTER_NAPI</tt> ||  ||
+|-
+| <tt>IORING_UNREGISTER_PERSONALITY</tt> ||  ||
 |-
 |}
@@ Line 358: / Line 422: @@
 |-
 |}
-<tt>ioprio</tt> includes the following flags (most of which are useless when using the higher-level liburing API):
+<tt>ioprio</tt> includes the following flags:
 {| class="wikitable" border="1"
 ! SQE ioprio flag !! Description
 |-
 | <tt>IORING_RECV_MULTISHOT</tt> ||
+|-
+| <tt>IORING_SEND_MULTISHOT</tt> || Introduced in 6.8.
 |-
 | <tt>IORING_ACCEPT_MULTISHOT</tt> ||
@@ Line 373: / Line 439: @@
 ===Acquiring SQEs===
-If using the higher-level API, <tt>io_uring_get_sqe(3)</tt> is the primary means to acquiring an SQE. If none are available, it will return <tt>NULL</tt>, and you should submit outstanding SQEs to free one up.
+If using the higher-level API, <tt>io_uring_get_sqe(3)</tt> is the primary means to acquire an SQE. If none are available, it will return <tt>NULL</tt>, and you should submit outstanding SQEs to free one up.
 <syntaxhighlight lang="c">
 struct io_uring_sqe *io_uring_get_sqe(struct io_uring *ring);
@@ Line 390: / Line 456: @@
 __u64 io_uring_cqe_get_data64(struct io_uring_cqe *cqe);
 </syntaxhighlight>
-Here's an example C++ data type that encodes eight bits as an operation type, eight bits as an index, and forty-eight bits as other data. I typically use something like this to reflect the operation which was used, the index into some relevant data structure, and other information about the operation (perhaps an offset or a length). Salt to taste, of course.
+Here's an example [[C++]] data type that encodes eight bits as an operation type, eight bits as an index, and forty-eight bits as other data. I typically use something like this to reflect the operation which was used, the index into some relevant data structure, and other information about the operation (perhaps an offset or a length). Salt to taste, of course.
 <syntaxhighlight lang="cpp">
 union URingCtx {
@@ Line 435: / Line 501: @@
 ====Opening and closing file descriptors====
 <syntaxhighlight lang="c">
+// IORING_OP_OPENAT (5.15)
 void io_uring_prep_openat(struct io_uring_sqe *sqe, int dfd, const char *path,
                            int flags, mode_t mode);
 void io_uring_prep_openat_direct(struct io_uring_sqe *sqe, int dfd, const char *path,
                                   int flags, mode_t mode, unsigned file_index);
+// IORING_OP_OPENAT2 (5.15)
 void io_uring_prep_openat2(struct io_uring_sqe *sqe, int dfd, const char *path,
                             int flags, struct open_how *how);
@@ Line 452: / Line 520: @@
 void io_uring_prep_multishot_accept_direct(struct io_uring_sqe *sqe, int sockfd, struct sockaddr *addr,
                                             socklen_t *addrlen, int flags);
+// IORING_OP_CLOSE (5.6, direct support since 5.15)
 void io_uring_prep_close(struct io_uring_sqe *sqe, int fd);
 void io_uring_prep_close_direct(struct io_uring_sqe *sqe, unsigned file_index);
+// IORING_OP_SOCKET (5.19)
 void io_uring_prep_socket(struct io_uring_sqe *sqe, int domain, int type,
                            int protocol, unsigned int flags);
@@ Line 467: / Line 537: @@
 // IORING_OP_SEND (5.6)
 void io_uring_prep_send(struct io_uring_sqe *sqe, int sockfd, const void *buf, size_t len, int flags);
+// IORING_OP_SEND_ZC (6.0)
 void io_uring_prep_send_zc(struct io_uring_sqe *sqe, int sockfd, const void *buf, size_t len, int flags, int zc_flags);
 // IORING_OP_SENDMSG (5.3)
@@ Line 477: / Line 548: @@
 void io_uring_prep_recvmsg(struct io_uring_sqe *sqe, int fd, struct msghdr *msg, unsigned flags);
 void io_uring_prep_recvmsg_multishot(struct io_uring_sqe *sqe, int fd, struct msghdr *msg, unsigned flags);
+// IORING_OP_READ (5.6)
+void io_uring_prep_read(struct io_uring_sqe *sqe, int fd, void *buf, unsigned nbytes, __u64 offset);
 // IORING_OP_READV
-void io_uring_prep_read(struct io_uring_sqe *sqe, int fd, void *buf, unsigned nbytes, __u64 offset);
 void io_uring_prep_readv(struct io_uring_sqe *sqe, int fd, const struct iovec *iovecs, unsigned nr_vecs, __u64 offset);
 void io_uring_prep_readv2(struct io_uring_sqe *sqe, int fd, const struct iovec *iovecs,
@@ Line 489: / Line 561: @@
 // IORING_OP_SYNC_FILE_RANGE (5.2)
 void io_uring_prep_sync_file_range(struct io_uring_sqe *sqe, int fd, unsigned len, __u64 offset, int flags);
+// IORING_OP_TEE (5.8)
 void io_uring_prep_tee(struct io_uring_sqe *sqe, int fd_in, int fd_out, unsigned int nbytes, unsigned int splice_flags);
+// IORING_OP_WRITE (5.6)
+void io_uring_prep_write(struct io_uring_sqe *sqe, int fd, const void *buf, unsigned nbytes, __u64 offset);
 // IORING_OP_WRITEV
-void io_uring_prep_write(struct io_uring_sqe *sqe, int fd, const void *buf, unsigned nbytes, __u64 offset);
 void io_uring_prep_writev(struct io_uring_sqe *sqe, int fd, const struct iovec *iovecs,
                            unsigned nr_vecs, __u64 offset);
@@ Line 501: / Line 575: @@
 // IORING_OP_CONNECT (5.5)
 void io_uring_prep_connect(struct io_uring_sqe *sqe, int sockfd, const struct sockaddr *addr, socklen_t addrlen);
+</syntaxhighlight>
+====Operations on file descriptors====
+<syntaxhighlight lang="c">
+// IORING_OP_URING_CMD
+void io_uring_prep_cmd_sock(struct io_uring_sqe *sqe, int cmd_op, int fd, int level,
+                            int optname, void *optval, int optlen);
+// IORING_OP_GETXATTR
+void io_uring_prep_getxattr(struct io_uring_sqe *sqe, const char *name, char *value,
+                            const char *path, unsigned int len);
+// IORING_OP_SETXATTR
+void io_uring_prep_setxattr(struct io_uring_sqe *sqe, const char *name, const char *value,
+                            const char *path, int flags, unsigned int len);
+// IORING_OP_FGETXATTR
+void io_uring_prep_fgetxattr(struct io_uring_sqe *sqe, int fd, const char *name,
+                             char *value, unsigned int len);
+// IORING_OP_FSETXATTR
+void io_uring_prep_fsetxattr(struct io_uring_sqe *sqe, int fd, const char *name,
+                             const char *value, int flags, unsigned int len);
 </syntaxhighlight>
@@ Line 541: / Line 634: @@
 All transring messages are prepared using the target uring's file descriptor, and thus aren't particularly compatible with the higher-level liburing API (of course, you can just yank <tt>ring_fd</tt> out of <tt>struct iouring</tt>). That's unfortunate, because this functionality is very useful in a multithreaded environment. Note that the ring to which the SQE is submitted can itself be the target of the message!
 <syntaxhighlight lang="c">
+// IORING_OP_MSG_RING
 void io_uring_prep_msg_ring(struct io_uring_sqe *sqe, int fd, unsigned int len, __u64 data, unsigned int flags);
 void io_uring_prep_msg_ring_cqe_flags(struct io_uring_sqe *sqe, int fd, unsigned int len, __u64 data, unsigned int flags, unsigned int cqe_flags);
 void io_uring_prep_msg_ring_fd(struct io_uring_sqe *sqe, int fd, int source_fd, int target_fd, __u64 data, unsigned int flags);
 void io_uring_prep_msg_ring_fd_alloc(struct io_uring_sqe *sqe, int fd, int source_fd, __u64 data, unsigned int flags);
+</syntaxhighlight>
+====Futexes====
+<syntaxhighlight lang="c">
+// IORING_OP_FUTEX_WAKE
+void io_uring_prep_futex_wake(struct io_uring_sqe *sqe, uint32_t *futex, uint64_t val,
+                              uint64_t mask, uint32_t futex_flags, unsigned int flags);
+// IORING_OP_FUTEX_WAIT
+void io_uring_prep_futex_wait(struct io_uring_sqe *sqe, uint32_t *futex, uint64_t val,
+                              uint64_t mask, uint32_t futex_flags, unsigned int flags);
+// IORING_OP_FUTEX_WAITV
+void io_uring_prep_futex_waitv(struct io_uring_sqe *sqe, struct futex_waitv *futex,
+                               uint32_t nr_futex, unsigned int flags);
 </syntaxhighlight>
@@ Line 633: / Line 740: @@
 |-
 | <tt>IORING_CQE_F_NOTIF</tt> || Notification CQE for zero-copy sends
+|-
+| <tt>IORING_CQE_F_BUF_MORE</tt> || There will be more CQEs to the specified registered buffer
 |-
 |}
-Completions can be detected by at least four different means:
+Completions can be detected by at least five different means:
-* Checking the completion queue speculatively. This either means a periodic check, which will suffer latency up to the period, or a busy check, which will churn CPU, but is probably the lowest-latency solution. This is best accomplished with <tt>io_uring_peek_cqe(3)</tt>, perhaps in conjunction with <tt>io_uring_cq_ready(3)</tt> (neither involves a system call).
+* Checking the completion queue speculatively. This either means a periodic check, which will suffer latency up to the period, or a busy check, which will churn CPU, but is probably the lowest-latency portable solution. This is best accomplished with <tt>io_uring_peek_cqe(3)</tt>, perhaps in conjunction with <tt>io_uring_cq_ready(3)</tt> (neither involves a system call).
 <syntaxhighlight lang="c">
 int io_uring_peek_cqe(struct io_uring *ring, struct io_uring_cqe **cqe_ptr);
@@ Line 653: / Line 762: @@
 </syntaxhighlight>
 * Using an [[eventfd]] together with <tt>io_uring_register_eventfd(3)</tt>. See below for the full API. This eventfd can be combined with e.g. regular [[epoll]].
+* Polling the uring's file descriptor directly.
 * Using processor-dependent memory watch instructions. On x86, there's [https://www.felixcloutier.com/x86/monitor MONITOR]+[https://www.felixcloutier.com/x86/mwait MWAIT], but they require you to be in ring 0, so you'd probably want [https://www.felixcloutier.com/x86/umonitor UMONITOR]+[https://www.felixcloutier.com/x86/umwait UMWAIT]. This ought allow a very low-latency wake that consumes very little power.
@@ Line 680: / Line 790: @@
 ===Multithreaded use===
-urings (and especially the <tt>struct io_uring</tt> object of liburing) are [https://github.com/axboe/liburing/issues/850#issuecomment-1526163360 not intended for multithreaded use] (quoth Axboe, "don't share a ring between threads"), though they can be used in several threaded paradigms. A single thread submitting and a single thread reaping is definitely supported. Descriptors can be sent among rings with <tt>IORING_OP_MSG_RING</tt>. Multiple submitters definitely must be serialized in userspace.
+urings (and especially the <tt>struct io_uring</tt> object of liburing) are [https://github.com/axboe/liburing/issues/850#issuecomment-1526163360 not intended for multithreaded use] (quoth Axboe, "don't share a ring between threads"), though they can be used in several threaded paradigms. A single thread submitting and some different single thread reaping is definitely supported. Descriptors can be sent among rings with <tt>IORING_OP_MSG_RING</tt>. Multiple submitters definitely must be serialized in userspace.
 If an op will be completed via a kernel task, the thread that submitted that SQE must remain alive until the op's completion. It will otherwise error out with <tt>-ECANCELED</tt>. If you must submit the SQE from a thread which will die, consider creating it disabled (see <tt>IORING_SETUP_R_DISABLED</tt>), and enabling it from the thread which will reap the completion event using <tt>IORING_REGISTER_ENABLE_RINGS</tt> with <tt>io_uring_register(2)</tt>.
@@ Line 688: / Line 798: @@
 When <tt>IORING_SETUP_SQPOLL</tt> is used, the kernel poller thread is considered to have performed the submission, providing another possible way around this problem.
-I am aware of no unannoying way to share some elements between threads in a single uring, while also monitoring distinct urings for each thread.
+I am aware of no unannoying way to share some elements between threads in a single uring, while also monitoring distinct urings for each thread. I suppose you could poll on both.
 ===Coexistence with [[epoll]]/[[XDP]]===
-If you want to monitor both an epoll and a uring in a single thread without busy waiting, you will run into problems. You can't directly <tt>poll()</tt> a uring for CQE readiness, so it can't be added to your epoll watchset. If you set a zero timeout, you're busy waiting; if you set a non-zero timeout, one is dependent on the other's readiness. There are two solutions:
+If you want to monitor both an epoll and a uring in a single thread without busy waiting, you will run into problems. If you set a zero timeout, you're busy waiting; if you set a non-zero timeout, one is dependent on the other's readiness. There are three solutions:
 * Add the epoll fd to your uring with <tt>IORING_OP_POLL_ADD</tt> (ideally multishot), and wait only for uring readiness. When you get a CQE for this submitted event, check the epoll.
+* Poll on the uring's file descriptor directly for completion queue events.
 * Register an [[eventfd]] with your uring with <tt>io_uring_register_eventfd(3)</tt>, add that to your epoll, and when you get <tt>POLLIN</tt> for this fd, check the completion ring.
 The full API here is:
@@ Line 702: / Line 813: @@
 <tt>io_uring_register_eventfd_async(3)</tt> only posts to the eventfd for events that completed out-of-line. There is not necessarily a bijection between completion events and posts even with the regular form; multiple CQEs can post only a single event, and spurious posts can occur.
-Similarly, [[XDP]]'s native means of notification is via <tt>poll(2)</tt>; XDP can be unified with uring using either of these two methods.
+Similarly, [[XDP]]'s native means of notification is via <tt>poll(2)</tt>; XDP can be unified with uring using any of these methods.
 ===Queue overflows===
@@ Line 716: / Line 827: @@
 ==What's missing==
-I'd like to see [[signalfd|signalfds]] and pidfds integrated for purposes of linked operations (you can read from them with the existing infrastructure, but you can't create them, and thus can't link their creation to other events).
+I'd like to see [[signalfd|signalfds]] and [[Pidfd|pidfds]] integrated for purposes of linked operations (you can read from them with the existing infrastructure, but you can't create them, and thus can't link their creation to other events).
 Why is there no <tt>vmsplice(2)</tt> action? How about <tt>fork(2)</tt>/<tt>pthread_create(3)</tt> (or more generally <tt>clone(2)</tt>)? We could have coroutines with I/O.
-It would be nice to have tight integration with condition variables or even mutex/futex (allow me to submit a request to get a lock, and when i get the CQE, i have that lock). Bonus points if the fast (uncontended) path never needs a system call (like mutexes built atop futexes today).
+It's kind of annoying that chains can't extend over submissions. If I've got a lot of data I need delivered in order, it seems I'm limited to a single chain in-flight at a time, or else I risk out-of-order delivery due to one chain aborting, followed by items from a subsequent chain succeeding. <tt>recv_multishot</tt> can help with this.
-I understand that making huge pages as unpleasant as possible to use is a central tenet (perhaps <i>the</i> central tenet) underpinning the entire Linux mission, but it would be great if hugepage-backed urings were only a flag/field away.
-It's kind of annoying that chains can't extend over submissions. If I've got a lot of data I need delivered in order, it seems I'm limited to a single chain in-flight at a time, or else I risk out-of-order delivery due to one chain aborting, followed by items from a subsequent chain succeeding.
 It seems that the obvious next step would be providing small snippets of arbitrary computation to be run in kernelspace, linked with SQEs. Perhaps [[eBPF]] would be a good starting place.
-It would be awesome if I could indicate in each SQE some other uring to which the CQE ought be delivered, basically combining inter-ring messages and SQE submission. This would be the <i>ne plus ultra</i> IMHO of multithreaded uring (and be very much like my [[Libtorque|masters thesis]], not that I claim any inspiration whatsoever for uring).
+It would be awesome if I could indicate in each SQE some other uring to which the CQE ought be delivered, basically combining inter-ring messages and SQE submission. This would be the <i>ne plus ultra</i> IMHO of multithreaded uring (and be very much like my [[Libtorque|masters thesis]], not that I claim any inspiration/precedence whatsoever for uring).
 ==See also==
@@ Line 748: / Line 855: @@
 ** "[https://lwn.net/Articles/863071/ Discriptorless files for io_uring]" 2021-07-19
 ** "[https://lwn.net/Articles/879724/ Zero-copy network transmission with io_uring]" 2021-12-30
+** "[https://lwn.net/Articles/1002371/ Process creation in io_uring]" 2024-12-20
+[[CATEGORY: Networking]]