Fast UNIX Servers: Difference between revisions
tarnsclusion of mteventqueues from libtorque |
No edit summary Tags: Mobile edit Mobile web edit |
||
| (8 intermediate revisions by the same user not shown) | |||
| Line 2: | Line 2: | ||
Dan Kegel's classic site "[http://www.kegel.com/c10k.html The C10K Problem]" (still updated from time to time) put a Promethean order to the arcana of years, with Jeff Darcy's "[http://pl.atyp.us/content/tech/servers.html High-Performance Server Architecture]" adding to our understanding. I'm collecting here some followup material to these excellent works (and of course the books of W. Richard Stevens, whose torch we merely carry). Some of these techniques have found, or will find, their way into [[libtorque]], my multithreaded event unification library (and master's thesis). | Dan Kegel's classic site "[http://www.kegel.com/c10k.html The C10K Problem]" (still updated from time to time) put a Promethean order to the arcana of years, with Jeff Darcy's "[http://pl.atyp.us/content/tech/servers.html High-Performance Server Architecture]" adding to our understanding. I'm collecting here some followup material to these excellent works (and of course the books of W. Richard Stevens, whose torch we merely carry). Some of these techniques have found, or will find, their way into [[libtorque]], my multithreaded event unification library (and master's thesis). | ||
''FIXME'' as of 2019, I need update this with information on [[DPDK]], [[io_uring]], and [[eBPF]]/[[XDP]] at a minimum. | |||
==Central Design Principles== | ==Central Design Principles== | ||
| Line 7: | Line 9: | ||
* Principle 1: '''Exploit all cycles/bandwidth.''' Avoid blocking I/O and unnecessary evictions of cache, but prefetch into cache where appropriate (this applies to page caches just as much as processor caches or any other layer of the [[Architecture#Memory_Hierarchies|memory hierarchy]]). Be prepared to exploit multiple processing elements. Properly align data and avoid cache-aliasing effects. Use jumbo frames in appropriate scenarios and proactively warn on network degradation (e.g., half-duplex Ethernet due to failed link negotiation). | * Principle 1: '''Exploit all cycles/bandwidth.''' Avoid blocking I/O and unnecessary evictions of cache, but prefetch into cache where appropriate (this applies to page caches just as much as processor caches or any other layer of the [[Architecture#Memory_Hierarchies|memory hierarchy]]). Be prepared to exploit multiple processing elements. Properly align data and avoid cache-aliasing effects. Use jumbo frames in appropriate scenarios and proactively warn on network degradation (e.g., half-duplex Ethernet due to failed link negotiation). | ||
* Principle 2: '''Don't duplicate work.''' Avoid unnecessary copies, context switches, system calls and signals. Use double-buffering or ringbuffers, and calls like [[Linux APIs|Linux's]] <tt>splice(2)</tt>. | * Principle 2: '''Don't duplicate work.''' Avoid unnecessary copies, context switches, system calls and signals. Use double-buffering or ringbuffers, and calls like [[Linux APIs|Linux's]] <tt>splice(2)</tt>. | ||
* Principle 3: '''Measure, measure, and measure again, preferably automatically.''' Hardware, software and networks will all surprise you. Become friends with your hardware's [[Performance Counters|performance counters]] and tools like [[ | * Principle 3: '''Measure, measure, and measure again, preferably automatically.''' Hardware, software and networks will all surprise you. Become friends with your hardware's [[Performance Counters|performance counters]] and tools like [[eBPF]], dtrace, ktrace, etc. Build explicit support for performance analysis into the application, especially domain-specific statistics. | ||
<blockquote>"I thought of another moral, more down to earth and concrete, and I believe that every militant chemist can confirm it: that one must distrust the almost-the-same (sodium is almost the same as potassium, but with sodium nothing would have happened), the practically identical, the approximate, all surrogates, and all patchwork. The differences can be small, but they can lead to radically different consequences, like a railroad's switch points: the chemist's trade consists in good part of being aware of these differences, knowing them close up and foreseeing their effects. And not only the chemist's trade." - Primo Levi, ''The Periodic Table''</blockquote> | <blockquote>"I thought of another moral, more down to earth and concrete, and I believe that every militant chemist can confirm it: that one must distrust the almost-the-same (sodium is almost the same as potassium, but with sodium nothing would have happened), the practically identical, the approximate, all surrogates, and all patchwork. The differences can be small, but they can lead to radically different consequences, like a railroad's switch points: the chemist's trade consists in good part of being aware of these differences, knowing them close up and foreseeing their effects. And not only the chemist's trade." - Primo Levi, ''The Periodic Table''</blockquote> | ||
| Line 16: | Line 18: | ||
==Event Cores== | ==Event Cores== | ||
* as of Linux 5, [[io_uring]] is the only game in town on Linux | |||
* [[epoll]] on [[Linux APIs|Linux]], <tt>/dev/poll</tt> on Solaris, [[kqueue]] on [[FreeBSD APIs|FreeBSD]] | * [[epoll]] on [[Linux APIs|Linux]], <tt>/dev/poll</tt> on Solaris, [[kqueue]] on [[FreeBSD APIs|FreeBSD]] | ||
* [http://msdn.microsoft.com/en-us/library/aa365198%28VS.85%29.aspx I/O Completion Ports] on Win32, [http://developers.sun.com/solaris/articles/event_completion.html Event Completion Framework] on Solaris 10, POSIX.1b asynchronous I/O | * [http://msdn.microsoft.com/en-us/library/aa365198%28VS.85%29.aspx I/O Completion Ports] on Win32, [http://developers.sun.com/solaris/articles/event_completion.html Event Completion Framework] on Solaris 10, POSIX.1b asynchronous I/O | ||
| Line 50: | Line 53: | ||
** I'm actively using <tt>PACKET_TX_RING</tt> in [[Omphalos]] as of June 2011; it works magnificently. | ** I'm actively using <tt>PACKET_TX_RING</tt> in [[Omphalos]] as of June 2011; it works magnificently. | ||
* "Zero-copy" gets banded about a good bit; be sure you're aware of hardware limitations (see FreeBSD's <tt>zero_copy(9)</tt>, for instance) | * "Zero-copy" gets banded about a good bit; be sure you're aware of hardware limitations (see FreeBSD's <tt>zero_copy(9)</tt>, for instance) | ||
* Linux 5 introduced <tt>io_uring</tt> and really locked down and expanded [[eBPF]] | |||
==The Full Monty: A Theory of UNIX Servers== | ==The Full Monty: A Theory of UNIX Servers== | ||
| Line 81: | Line 85: | ||
===Hardware Esoterica=== | ===Hardware Esoterica=== | ||
* [[Direct Cache Access]] must be supported by NICs, northbridge chipset, OS and microarchitecture | * [[Direct Cache Access]] must be supported by NICs, northbridge chipset, OS and microarchitecture | ||
* IOMMU / I/OAT | * [[IOMMU]] / I/OAT | ||
* Checksum offloading / TSO / [http://lwn.net/Articles/243949/ LRO] / Frame descriptors | * Checksum offloading / TSO / [http://lwn.net/Articles/243949/ LRO] / Frame descriptors | ||
** Use [http://packages.debian.org/search?keywords=ethtool ethtool] on Linux to configure NICs (try <tt>ethtool -g, -k and -c</tt>) | ** Use [http://packages.debian.org/search?keywords=ethtool ethtool] on Linux to configure NICs (try <tt>ethtool -g, -k and -c</tt>) | ||
| Line 95: | Line 99: | ||
===Tuning for the Network=== | ===Tuning for the Network=== | ||
* All hosts ought employ the | * All hosts ought employ the RFC 1323 options (see [[Syncookies]] regarding contraindications there) | ||
* Avoid fragmentation: datagram services (UDP, DCCP) ought ensure they're not exceeding PMTUs | * Avoid fragmentation: datagram services (UDP, DCCP) ought ensure they're not exceeding PMTUs | ||
* LAN services ought consider jumbo frames. | * LAN services ought consider jumbo frames. | ||
| Line 115: | Line 119: | ||
==See Also== | ==See Also== | ||
* [https://raw.github.com/dankamongmen/libtorque/master/doc/mteventqueues "mteventqueues"] by Your Humble Wikist, and likewise [[libtorque]] | * [https://raw.github.com/dankamongmen/libtorque/master/doc/mteventqueues "mteventqueues"] by Your Humble Wikist, and likewise [[libtorque]] | ||
* [https://theojulienne.io/2020/07/03/scaling-linux-services-before-accepting-connections.html "Scaling Linux Services Before Accepting Connections"] by Theo Julienne, 2020-07-03 | |||
* [http://jacquesmattheij.com/Poll+vs+Epoll+once+again "Poll vs Epoll, Once Again"] by Jacques Mattheij, 2010-08-04 | * [http://jacquesmattheij.com/Poll+vs+Epoll+once+again "Poll vs Epoll, Once Again"] by Jacques Mattheij, 2010-08-04 | ||
* [http://sheddingbikes.com/posts/1280829388.html "Poll, Epoll, Science, and Superpoll"] by Zed Shaw | * [http://sheddingbikes.com/posts/1280829388.html "Poll, Epoll, Science, and Superpoll"] by Zed Shaw | ||