DDIO

Data Direct I/O is an Intel technology allowing PCIe devices to interact directly with local processor LLCs (Last Level Caches). It supersedes and extends Direct Cache Access, and is present on E7 Xeons of the second generation and later, and all E5 Xeons.

Software

DDIO is architecturally transparent to hardware and software, but since it can only function with directly-connected ("local") sockets, threads ought be bound to those cores if they want to take advantage of DDIO. Model-specific register 0xc8b controls which LLC ways are used by DDIO (per PCIe root). There is currently no CPUID element corresponding to DDIO.

If DDIO is providing useful advantage, it can be best detected by a decrease in L3 data cache misses (normally, input involves a compulsory LLC miss, since it's routed to memory from the card, and only then into the processor). Use perf to check the relevant performance counters:

perf stat -e cycles:u,instructions:u,mem_load_retired.l3_miss:u,mem_load_retired.l2_miss:u

External links

• "Reexamining Direct Cache Access to Optimize I/O Intensive Applications for Multi-hundred-gigabit Networks", Farshin + Roozbeh + Maguire + Kostić, USENIX ATC 2020