Check out my first novel, midnight's simulacra!
DDIO: Difference between revisions
(Created page with "Data Direct I/O is an [https://www.intel.com/content/www/us/en/io/data-direct-i-o-technology.html Intel technology] allowing PCIe devices to interact directly with local processor LLCs (Last Level Caches). It supersedes and extends Direct Cache Access, and is present on E7 Xeons of the second generation and later, and all E5 Xeons. ==Configuration== Model-specific register 0xc8b controls which LLC ways are used by DDIO. There is currently no CPUID element co...") |
|||
| (One intermediate revision by the same user not shown) | |||
| Line 1: | Line 1: | ||
Data Direct I/O is an [https://www.intel.com/content/www/us/en/io/data-direct-i-o-technology.html Intel technology] allowing PCIe devices to interact directly with local processor LLCs (Last Level Caches). It supersedes and extends [[Direct Cache Access]], and is present on E7 Xeons of the second generation and later, and all E5 Xeons. | Data Direct I/O is an [https://www.intel.com/content/www/us/en/io/data-direct-i-o-technology.html Intel technology] allowing PCIe devices to interact directly with local processor LLCs (Last Level Caches). It supersedes and extends [[Direct Cache Access]], and is present on E7 Xeons of the second generation and later, and all E5 Xeons. | ||
== | ==Software== | ||
[[MSR|Model-specific register]] 0xc8b controls which LLC ways are used by DDIO. There is currently no CPUID element corresponding to DDIO. | DDIO is architecturally transparent to hardware and software, but since it can only function with directly-connected ("local") sockets, threads ought be bound to those cores if they want to take advantage of DDIO. [[MSR|Model-specific register]] 0xc8b controls which LLC ways are used by DDIO (per PCIe root). There is currently no CPUID element corresponding to DDIO. | ||
If DDIO is providing useful advantage, it can be best detected by a decrease in L3 data cache misses (normally, input involves a compulsory LLC miss, since it's routed to memory from the card, and only then into the processor). Use [[perf]] to check the relevant [[Performance Counters|performance counters]]: | |||
<tt>perf stat -e cycles:u,instructions:u,mem_load_retired.l3_miss:u,mem_load_retired.l2_miss:u</tt> | |||
==External links== | ==External links== | ||
Latest revision as of 01:05, 22 January 2023
Data Direct I/O is an Intel technology allowing PCIe devices to interact directly with local processor LLCs (Last Level Caches). It supersedes and extends Direct Cache Access, and is present on E7 Xeons of the second generation and later, and all E5 Xeons.
Software
DDIO is architecturally transparent to hardware and software, but since it can only function with directly-connected ("local") sockets, threads ought be bound to those cores if they want to take advantage of DDIO. Model-specific register 0xc8b controls which LLC ways are used by DDIO (per PCIe root). There is currently no CPUID element corresponding to DDIO.
If DDIO is providing useful advantage, it can be best detected by a decrease in L3 data cache misses (normally, input involves a compulsory LLC miss, since it's routed to memory from the card, and only then into the processor). Use perf to check the relevant performance counters:
perf stat -e cycles:u,instructions:u,mem_load_retired.l3_miss:u,mem_load_retired.l2_miss:u
External links
- "Reexamining Direct Cache Access to Optimize I/O Intensive Applications for Multi-hundred-gigabit Networks", Farshin + Roozbeh + Maguire + Kostić, USENIX ATC 2020
