Fast UNIX Servers - revision history

Dank at 20:49, 22 May 2023

2023-05-22T20:49:11Z

← Older revision		Revision as of 20:49, 22 May 2023
Line 3:		Line 3:
	Dan Kegel's classic site "[http://www.kegel.com/c10k.html The C10K Problem]" (still updated from time to time) put a Promethean order to the arcana of years, with Jeff Darcy's "[http://pl.atyp.us/content/tech/servers.html High-Performance Server Architecture]" adding to our understanding. I'm collecting here some followup material to these excellent works (and of course the books of W. Richard Stevens, whose torch we merely carry). Some of these techniques have found, or will find, their way into [[libtorque]], my multithreaded event unification library (and master's thesis).		Dan Kegel's classic site "[http://www.kegel.com/c10k.html The C10K Problem]" (still updated from time to time) put a Promethean order to the arcana of years, with Jeff Darcy's "[http://pl.atyp.us/content/tech/servers.html High-Performance Server Architecture]" adding to our understanding. I'm collecting here some followup material to these excellent works (and of course the books of W. Richard Stevens, whose torch we merely carry). Some of these techniques have found, or will find, their way into [[libtorque]], my multithreaded event unification library (and master's thesis).

	''FIXME'' as of 2019, I need update this with information on [[DPDK]] and [[eBPF]]/[[XDP]] at a minimum.		''FIXME'' as of 2019, I need update this with information on [[DPDK]], [[io_uring]], and [[eBPF]]/[[XDP]] at a minimum.

	==Central Design Principles==		==Central Design Principles==

Dank: /* Hardware Esoterica */

2022-03-12T23:44:06Z

Hardware Esoterica

← Older revision		Revision as of 23:44, 12 March 2022
Line 85:		Line 85:
	===Hardware Esoterica===		===Hardware Esoterica===
	* [[Direct Cache Access]] must be supported by NICs, northbridge chipset, OS and microarchitecture		* [[Direct Cache Access]] must be supported by NICs, northbridge chipset, OS and microarchitecture
	* IOMMU / I/OAT		* [[IOMMU]] / I/OAT
	* Checksum offloading / TSO / [http://lwn.net/Articles/243949/ LRO] / Frame descriptors		* Checksum offloading / TSO / [http://lwn.net/Articles/243949/ LRO] / Frame descriptors
	** Use [http://packages.debian.org/search?keywords=ethtool ethtool] on Linux to configure NICs (try <tt>ethtool -g, -k and -c</tt>)		** Use [http://packages.debian.org/search?keywords=ethtool ethtool] on Linux to configure NICs (try <tt>ethtool -g, -k and -c</tt>)

Dank: /* Event Cores */

2021-05-19T08:31:51Z

Event Cores

← Older revision		Revision as of 08:31, 19 May 2021
Line 18:		Line 18:

	==Event Cores==		==Event Cores==
			* as of Linux 5, [[io_uring]] is the only game in town on Linux
	* [[epoll]] on [[Linux APIs\|Linux]], <tt>/dev/poll</tt> on Solaris, [[kqueue]] on [[FreeBSD APIs\|FreeBSD]]		* [[epoll]] on [[Linux APIs\|Linux]], <tt>/dev/poll</tt> on Solaris, [[kqueue]] on [[FreeBSD APIs\|FreeBSD]]
	* [http://msdn.microsoft.com/en-us/library/aa365198%28VS.85%29.aspx I/O Completion Ports] on Win32, [http://developers.sun.com/solaris/articles/event_completion.html Event Completion Framework] on Solaris 10, POSIX.1b asynchronous I/O		* [http://msdn.microsoft.com/en-us/library/aa365198%28VS.85%29.aspx I/O Completion Ports] on Win32, [http://developers.sun.com/solaris/articles/event_completion.html Event Completion Framework] on Solaris 10, POSIX.1b asynchronous I/O

Dank: link to Theo Julienne

2020-07-05T08:29:49Z

link to Theo Julienne

← Older revision		Revision as of 08:29, 5 July 2020
Line 118:		Line 118:
	==See Also==		==See Also==
	* [https://raw.github.com/dankamongmen/libtorque/master/doc/mteventqueues "mteventqueues"] by Your Humble Wikist, and likewise [[libtorque]]		* [https://raw.github.com/dankamongmen/libtorque/master/doc/mteventqueues "mteventqueues"] by Your Humble Wikist, and likewise [[libtorque]]
			* [https://theojulienne.io/2020/07/03/scaling-linux-services-before-accepting-connections.html "Scaling Linux Services Before Accepting Connections"] by Theo Julienne, 2020-07-03
	* [http://jacquesmattheij.com/Poll+vs+Epoll+once+again "Poll vs Epoll, Once Again"] by Jacques Mattheij, 2010-08-04		* [http://jacquesmattheij.com/Poll+vs+Epoll+once+again "Poll vs Epoll, Once Again"] by Jacques Mattheij, 2010-08-04
	* [http://sheddingbikes.com/posts/1280829388.html "Poll, Epoll, Science, and Superpoll"] by Zed Shaw		* [http://sheddingbikes.com/posts/1280829388.html "Poll, Epoll, Science, and Superpoll"] by Zed Shaw

Dank at 18:21, 10 December 2019

2019-12-10T18:21:08Z

← Older revision		Revision as of 18:21, 10 December 2019
Line 3:		Line 3:
	Dan Kegel's classic site "[http://www.kegel.com/c10k.html The C10K Problem]" (still updated from time to time) put a Promethean order to the arcana of years, with Jeff Darcy's "[http://pl.atyp.us/content/tech/servers.html High-Performance Server Architecture]" adding to our understanding. I'm collecting here some followup material to these excellent works (and of course the books of W. Richard Stevens, whose torch we merely carry). Some of these techniques have found, or will find, their way into [[libtorque]], my multithreaded event unification library (and master's thesis).		Dan Kegel's classic site "[http://www.kegel.com/c10k.html The C10K Problem]" (still updated from time to time) put a Promethean order to the arcana of years, with Jeff Darcy's "[http://pl.atyp.us/content/tech/servers.html High-Performance Server Architecture]" adding to our understanding. I'm collecting here some followup material to these excellent works (and of course the books of W. Richard Stevens, whose torch we merely carry). Some of these techniques have found, or will find, their way into [[libtorque]], my multithreaded event unification library (and master's thesis).

	''FIXME'' as of 2019, I need update this with information on [[DPDK]] and [[eBPF]] at a minimum.		''FIXME'' as of 2019, I need update this with information on [[DPDK]] and [[eBPF]]/[[XDP]] at a minimum.

	==Central Design Principles==		==Central Design Principles==

Dank at 18:18, 10 December 2019

2019-12-10T18:18:40Z

← Older revision		Revision as of 18:18, 10 December 2019
Line 2:		Line 2:

	Dan Kegel's classic site "[http://www.kegel.com/c10k.html The C10K Problem]" (still updated from time to time) put a Promethean order to the arcana of years, with Jeff Darcy's "[http://pl.atyp.us/content/tech/servers.html High-Performance Server Architecture]" adding to our understanding. I'm collecting here some followup material to these excellent works (and of course the books of W. Richard Stevens, whose torch we merely carry). Some of these techniques have found, or will find, their way into [[libtorque]], my multithreaded event unification library (and master's thesis).		Dan Kegel's classic site "[http://www.kegel.com/c10k.html The C10K Problem]" (still updated from time to time) put a Promethean order to the arcana of years, with Jeff Darcy's "[http://pl.atyp.us/content/tech/servers.html High-Performance Server Architecture]" adding to our understanding. I'm collecting here some followup material to these excellent works (and of course the books of W. Richard Stevens, whose torch we merely carry). Some of these techniques have found, or will find, their way into [[libtorque]], my multithreaded event unification library (and master's thesis).

			''FIXME'' as of 2019, I need update this with information on [[DPDK]] and [[eBPF]] at a minimum.

	==Central Design Principles==		==Central Design Principles==

Dank: /* A Garden of Interfaces */

2019-11-02T01:35:57Z

A Garden of Interfaces

← Older revision		Revision as of 01:35, 2 November 2019
Line 50:		Line 50:
	** I'm actively using <tt>PACKET_TX_RING</tt> in [[Omphalos]] as of June 2011; it works magnificently.		** I'm actively using <tt>PACKET_TX_RING</tt> in [[Omphalos]] as of June 2011; it works magnificently.
	* "Zero-copy" gets banded about a good bit; be sure you're aware of hardware limitations (see FreeBSD's <tt>zero_copy(9)</tt>, for instance)		* "Zero-copy" gets banded about a good bit; be sure you're aware of hardware limitations (see FreeBSD's <tt>zero_copy(9)</tt>, for instance)
			* Linux 5 introduced <tt>io_uring</tt> and really locked down and expanded [[eBPF]]

	==The Full Monty: A Theory of UNIX Servers==		==The Full Monty: A Theory of UNIX Servers==

Dank: /* Central Design Principles */

2019-11-02T01:34:35Z

Central Design Principles

← Older revision		Revision as of 01:34, 2 November 2019
Line 7:		Line 7:
	* Principle 1: '''Exploit all cycles/bandwidth.''' Avoid blocking I/O and unnecessary evictions of cache, but prefetch into cache where appropriate (this applies to page caches just as much as processor caches or any other layer of the [[Architecture#Memory_Hierarchies\|memory hierarchy]]). Be prepared to exploit multiple processing elements. Properly align data and avoid cache-aliasing effects. Use jumbo frames in appropriate scenarios and proactively warn on network degradation (e.g., half-duplex Ethernet due to failed link negotiation).		* Principle 1: '''Exploit all cycles/bandwidth.''' Avoid blocking I/O and unnecessary evictions of cache, but prefetch into cache where appropriate (this applies to page caches just as much as processor caches or any other layer of the [[Architecture#Memory_Hierarchies\|memory hierarchy]]). Be prepared to exploit multiple processing elements. Properly align data and avoid cache-aliasing effects. Use jumbo frames in appropriate scenarios and proactively warn on network degradation (e.g., half-duplex Ethernet due to failed link negotiation).
	* Principle 2: '''Don't duplicate work.''' Avoid unnecessary copies, context switches, system calls and signals. Use double-buffering or ringbuffers, and calls like [[Linux APIs\|Linux's]] <tt>splice(2)</tt>.		* Principle 2: '''Don't duplicate work.''' Avoid unnecessary copies, context switches, system calls and signals. Use double-buffering or ringbuffers, and calls like [[Linux APIs\|Linux's]] <tt>splice(2)</tt>.
	* Principle 3: '''Measure, measure, and measure again, preferably automatically.''' Hardware, software and networks will all surprise you. Become friends with your hardware's [[Performance Counters\|performance counters]] and tools like [[~~Oprofile~~]], dtrace, ktrace, etc. Build explicit support for performance analysis into the application, especially domain-specific statistics.		* Principle 3: '''Measure, measure, and measure again, preferably automatically.''' Hardware, software and networks will all surprise you. Become friends with your hardware's [[Performance Counters\|performance counters]] and tools like [[eBPF]], dtrace, ktrace, etc. Build explicit support for performance analysis into the application, especially domain-specific statistics.
	<blockquote>"I thought of another moral, more down to earth and concrete, and I believe that every militant chemist can confirm it: that one must distrust the almost-the-same (sodium is almost the same as potassium, but with sodium nothing would have happened), the practically identical, the approximate, all surrogates, and all patchwork. The differences can be small, but they can lead to radically different consequences, like a railroad's switch points: the chemist's trade consists in good part of being aware of these differences, knowing them close up and foreseeing their effects. And not only the chemist's trade." - Primo Levi, ''The Periodic Table''</blockquote>		<blockquote>"I thought of another moral, more down to earth and concrete, and I believe that every militant chemist can confirm it: that one must distrust the almost-the-same (sodium is almost the same as potassium, but with sodium nothing would have happened), the practically identical, the approximate, all surrogates, and all patchwork. The differences can be small, but they can lead to radically different consequences, like a railroad's switch points: the chemist's trade consists in good part of being aware of these differences, knowing them close up and foreseeing their effects. And not only the chemist's trade." - Primo Levi, ''The Periodic Table''</blockquote>

Dank: make rfc 1323 a link

2012-06-14T15:31:19Z

make rfc 1323 a link

← Older revision		Revision as of 15:31, 14 June 2012
Line 95:		Line 95:

	===Tuning for the Network===		===Tuning for the Network===
	* All hosts ought employ the ~~RFC1323~~ options (see [[Syncookies]] regarding contraindications there)		* All hosts ought employ the RFC 1323 options (see [[Syncookies]] regarding contraindications there)
	* Avoid fragmentation: datagram services (UDP, DCCP) ought ensure they're not exceeding PMTUs		* Avoid fragmentation: datagram services (UDP, DCCP) ought ensure they're not exceeding PMTUs
	* LAN services ought consider jumbo frames.		* LAN services ought consider jumbo frames.

Dank: tarnsclusion of mteventqueues from libtorque

2012-06-14T15:27:58Z

tarnsclusion of mteventqueues from libtorque

← Older revision		Revision as of 15:27, 14 June 2012
Line 27:		Line 27:
	* [[Asynchronous I/O]] is pretty much by definition edge-triggered.		* [[Asynchronous I/O]] is pretty much by definition edge-triggered.
	* <tt>epoll</tt> (via <tt>EPOLLET</tt>) and <tt>kqueue</tt> (via <tt>EV_CLEAR</tt>) provide edge-triggered semantics		* <tt>epoll</tt> (via <tt>EPOLLET</tt>) and <tt>kqueue</tt> (via <tt>EV_CLEAR</tt>) provide edge-triggered semantics
	* '''fixme''' a thorough comparison of these is sorely needed -- in short, edge-triggered saves syscalls, speeds up event cores, and makes concurrency easier		* see the <tt>mteventqueue</tt> [https://raw.github.com/dankamongmen/libtorque/master/doc/mteventqueues document from libtorque's documentation]:
	** see the <tt>mteventqueue</tt> [https://raw.github.com/dankamongmen/libtorque/master/doc/mteventqueues document from libtorque's documentation]		<include src="https://raw.github.com/dankamongmen/libtorque/master/doc/mteventqueues " />

	==A Garden of Interfaces==		==A Garden of Interfaces==