dankwiki

Libtorque: Difference between revisions

← Older edit

Dank (talk | contribs)

2 user groups
11,302 edits
m hotpar backed 'dat ass up
update links
 
(20 intermediate revisions by the same user not shown)
Line 1: Line 1:
[[File:Libtorque.svg|thumb|right|alt="architecture model"|Architecture of a libtorque-enabled process.]]
[[File:Libtorque.svg|thumb|right|alt="architecture model"|Architecture of a libtorque-enabled process.]]
'''My [[Fast UNIX Servers]] page is a useful companion to this article.'''
<tt>libtorque</tt> is a multithreaded event library for UNIX designed to take full advantage of the manycore, heterogenous, [[NUMA]] future. The [[media:Libtorque-proposal.pdf|project proposal]] suggests motivation for <tt>libtorque</tt>: I believe it necessary to take scheduling and memory-placement decisions into account to most optimally handle events, especially on manycore machines and ''especially'' to handle unexpected traffic sets (denial of service attacks, oversubscribed pipes, mixed-latency connections, etc). Along the way, I intend to shake up the UNIX programming idiom; my hope is that <tt>libtorque</tt> leads to more network programmers thinking about the complex issues involved, but simplifies rather than aggravates a task already fraught with difficulty.
<tt>libtorque</tt> is a multithreaded event library for UNIX designed to take full advantage of the manycore, heterogenous, [[NUMA]] future. The [[media:Libtorque-proposal.pdf|project proposal]] suggests motivation for <tt>libtorque</tt>: I believe it necessary to take scheduling and memory-placement decisions into account to most optimally handle events, especially on manycore machines and ''especially'' to handle unexpected traffic sets (denial of service attacks, oversubscribed pipes, mixed-latency connections, etc). Along the way, I intend to shake up the UNIX programming idiom; my hope is that <tt>libtorque</tt> leads to more network programmers thinking about the complex issues involved, but simplifies rather than aggravates a task already fraught with difficulty.


Previous, non-threaded open source event libraries include [http://www.monkey.org/~provos/libevent/ libevent], [http://software.schmorp.de/pkg/libev.html libev] and [http://liboop.ofb.net liboop].
Other open source event libraries include [http://www.monkey.org/~provos/libevent/ libevent], [http://software.schmorp.de/pkg/libev.html libev] and [http://liboop.ofb.net liboop].


'''I am currently focused on producing a submission for [http://www.usenix.org/events/hotpar10/cfp/ HotPar 2010], due 2010-01-31. That document will provide a unified, current, readable explanation of libtorque, with much information not yet recorded here. Watch this space!'''
[[User:Dank|Dank]] 21:36, 17 January 2010 (UTC)
==Resources==
==Resources==
* "[[Media:hotpar2010.pdf|libtorque: Portable Multithreaded Continuations for Scalable Event-Driven Programs]]"
* [[git]] hosting from [http://github.com GitHub] (dankamongmen/libtorque [http://github.com/dankamongmen/libtorque project page])
* [[git]] hosting from [http://github.com GitHub] (dankamongmen/libtorque [http://github.com/dankamongmen/libtorque project page])
** Lots of good data in the [http://github.com/dankamongmen/libtorque/blob/master/README README!]
** Lots of good data in the [http://github.com/dankamongmen/libtorque/blob/master/README README!]
** <tt>git clone</tt> from git://github.com/dankamongmen/libtorque.git
** <tt>git clone</tt> from git://github.com/dankamongmen/libtorque.git
* [http://www.bugzilla.org/ bugzilla], hosted here on http://dank.qemfd.net/bugzilla/. See [[#Current_Issues|below]] for a snapshot bug report.
* [http://www.bugzilla.org/ bugzilla], hosted here on https://nick-black.com/bugzilla/. See [[#Current_Issues|below]] for a snapshot bug report.
* A [[Media:Libtorque-presentation.pdf|presentation]] I did for GT's [http://comparch.gatech.edu/arch_whisky/fall09.html Arch-Whiskey] seminar, 2009-11-13
* A [[Media:Libtorque-presentation.pdf|presentation]] I did for GT's [http://comparch.gatech.edu/arch_whisky/fall09.html Arch-Whiskey] seminar, 2009-11-13
* Fo sho there's a [http://groups.google.com/group/libtorque-devel mailing list]!
* Fo sho there's a [http://groups.google.com/group/libtorque-devel mailing list]!
Line 17: Line 18:
* <tt>[http://github.com/dankamongmen/libtorque/raw/master/doc/mteventqueues mteventqueues]</tt> - "Multithreaded Event Queues". Details of [[epoll]] and [[kqueue]] semantics, especially with regard to locking. Correctness and performance implications thereof.
* <tt>[http://github.com/dankamongmen/libtorque/raw/master/doc/mteventqueues mteventqueues]</tt> - "Multithreaded Event Queues". Details of [[epoll]] and [[kqueue]] semantics, especially with regard to locking. Correctness and performance implications thereof.
* <tt>[http://github.com/dankamongmen/libtorque/raw/master/doc/termination termination]</tt> - "Termination". Interaction with POSIX cancellation and signals. API and semantics for initiating and blocking on a libtorque context's shutdown. Design justification and details.
* <tt>[http://github.com/dankamongmen/libtorque/raw/master/doc/termination termination]</tt> - "Termination". Interaction with POSIX cancellation and signals. API and semantics for initiating and blocking on a libtorque context's shutdown. Design justification and details.
===Recent commits (via [http://github.com/feeds/dankamongmen/commits/libtorque/master Atom])===
<!-- ===Recent commits (via [http://github.com/feeds/dankamongmen/commits/libtorque/master Atom])===
<rss>http://github.com/feeds/dankamongmen/commits/libtorque/master</rss>
<rss>http://github.com/feeds/dankamongmen/commits/libtorque/master</rss> -->


==Design/Functionality==
==Design/Functionality==
Line 58: Line 59:
====archdetect====
====archdetect====
* Utility built/packaged with libtorque:
* Utility built/packaged with libtorque:
<pre> Package 0: (8 threads total)
<pre>Testing archdetect: env LD_LIBRARY_PATH=.out/lib .out/bin/archdetect
Core 0:  1   9 (2x processor type 1)
Package 0: (8 threads total)
Core 1:  3  11 (2x processor type 1)
Core 0:  0   4 (2x processor type 1)
Core 2:   5 13 (2x processor type 1)
Core 1:  1  5 (2x processor type 1)
Core 3:  7  15 (2x processor type 1)
Core 2:  2   6 (2x processor type 1)
Package 1: (8 threads total)
Core 3:  3  7 (2x processor type 1)
Core 0:   0  8 (2x processor type 1)
Core 1:  2  10 (2x processor type 1)
Core 2:  4  12 (2x processor type 1)
Core 3:   6  14 (2x processor type 1)
(  1x) Memory node 1 of 1:
(  1x) Memory node 1 of 1:
12292928KB (11.723 GB) total, 4KB pages
8,131,244KB (7.754 GB) total in 4KB and 4MB pages
( 16x) Processing unit type 1 of 1:
(   8x) Processing unit type 1 of 2: x86
Brand name: Intel(R) Xeon(R) CPU E5520 @ 2.27GHz (OEM)
Extensions: MMX SSE SSE2 SSE3 SSSE3 SSE4.1 SSE4.2 SSE4a
Family: 0x006 (6) Model: 0x1a (26) Stepping: 5
Family: 0x006 (6) Model: 0x1e (30) Stepping: 5 (OEM)
Brand name: Intel(R) Core(TM) i7 CPU Q 720 @ 1.60GHz
2 threads per processing core, 8 cores (4 logical) per package
2 threads per processing core, 8 cores (4 logical) per package
Cache 1 of 4: 32KB total, 64B line, 8-assoc, 2-shared (L1 data)
Cache 1 of 4: 32KB total, 64B line, 4-assoc, 2-shared (L1 code)
Cache 2 of 4: 32KB total, 64B line, 4-assoc, 2-shared (L1 code)
Cache 2 of 4: 32KB total, 64B line, 8-assoc, 2-shared (L1 data)
Cache 3 of 4: 256KB total, 64B line, 8-assoc, 2-shared (L2 unified)
Cache 3 of 4: 256KB total, 64B line, 8-assoc, 2-shared (L2 unified)
Cache 4 of 4: 8MB total, 64B line, 16-assoc, 16-shared (L3 unified)
Cache 4 of 4: 6MB total, 64B line, 12-assoc, 16-shared (L3 unified)
TLB 1 of 5: 4KB pages, 7-entry, 7-assoc, 2-shared (L2 code)
TLB 1 of 5: 4KB pages, 64-entry, 4-assoc, 2-shared (L1 code)
TLB 2 of 5: 4KB pages, 64-entry, 4-assoc, 2-shared (L2 data)
TLB 2 of 5: 4MB pages, 32-entry, 4-assoc, 2-shared (L1 data)
TLB 3 of 5: 4MB pages, 32-entry, 4-assoc, 2-shared (L1 data)
TLB 3 of 5: 4KB pages, 7-entry, 7-assoc, 2-shared (L2 code)
TLB 4 of 5: 4KB pages, 64-entry, 4-assoc, 2-shared (L1 code)
TLB 4 of 5: 4KB pages, 64-entry, 4-assoc, 2-shared (L2 data)
TLB 5 of 5: 4KB pages, 512-entry, 4-assoc, 2-shared (L2 data)</pre>
TLB 5 of 5: 4KB pages, 512-entry, 4-assoc, 2-shared (L2 data)
(  1x) Processing unit type 2 of 2: CUDA
CUDA compute capabilities: 1.2
Brand name: GeForce GTS 360M
1 thread per processing core, 96 cores per package</pre>


===Scheduling===
===Scheduling===
Line 218: Line 220:


===Projects===
===Projects===
* Matt Welsh's [http://www.eecs.harvard.edu/~mdw/proj/seda/ SEDA] (Staged Event-Driven Architecture)
* [http://eventlet.net/ Eventlet] is a python implementation of asynchronous triggers
* The [[Radovic-Hagersten lock]] team has a page on [http://www.it.uu.se/research/group/uart/projects/nucasynch NUMA locking]
* The [[Radovic-Hagersten lock]] team has a page on [http://www.it.uu.se/research/group/uart/projects/nucasynch NUMA locking]
* Emery Berger's [http://www.hoard.org/ Hoard] and other manycore-capable [[allocators|allocators]] (libumem aka magazined slab, Google's [http://goog-perftools.sourceforge.net/doc/tcmalloc.html ctmalloc], etc).
* Emery Berger's [http://www.hoard.org/ Hoard] and other manycore-capable [[allocators|allocators]] (libumem aka magazined slab, Google's [http://goog-perftools.sourceforge.net/doc/tcmalloc.html ctmalloc], etc).
Line 231: Line 235:
<tt>libtorque</tt>, under that name, began as a project for Professor [http://vuduc.org/ Rich Vuduc's] Fall 2009 [[High Performance Parallel Computing|CSE6230]]. It really began gestating in three parts:
<tt>libtorque</tt>, under that name, began as a project for Professor [http://vuduc.org/ Rich Vuduc's] Fall 2009 [[High Performance Parallel Computing|CSE6230]]. It really began gestating in three parts:
* Work on intrusion prevention at [http://www.reflexsystems.com/ Reflex] got me hooked on automata-based networking activations and [[Fast UNIX Servers|fast networking]]
* Work on intrusion prevention at [http://www.reflexsystems.com/ Reflex] got me hooked on automata-based networking activations and [[Fast UNIX Servers|fast networking]]
* Work on [[ICAP]] servers and  [http://en.wikipedia.org/wiki/Reverse_proxy reverse proxies] at [http://www.mcafee.com/ McAfee], especially <tt>snare</tt>, got me thinking about networking API's
* Work on [[ICAP]] servers and  [http://en.wikipedia.org/wiki/Reverse_proxy reverse proxies] at [http://www.mcafee.com/ McAfee], especially <tt>snare</tt>, got me thinking about networking APIs
* Professor [http://www.cc.gatech.edu/directory/faculty/faculty/school-of-computer-science/directory/thomas-conte Tom Conte's] Spring 2009 "CS 8803 MCA: Multicore and Manycore Architecture" lit up the parallelism fire
* Professor [http://www.cc.gatech.edu/directory/faculty/faculty/school-of-computer-science/directory/thomas-conte Tom Conte's] Spring 2009 "CS 8803 MCA: Multicore and Manycore Architecture" lit up the parallelism fire


===Milestones===
===Milestones===
* 2009-10-22: [http://github.com/dankamongmen/libtorque/commit/e7429294beb9dc581a7cdab2371d2ddca3169047 First commit] (e7429294beb9dc581a7cdab2371d2ddca3169047)
* 2009-10-22: [http://github.com/dankamongmen/libtorque/commit/e7429294beb9dc581a7cdab2371d2ddca3169047 First commit] (e7429294beb9dc581a7cdab2371d2ddca3169047)
* 2009-11-12: CSE 6230 checkpoint (see [http://dank.qemfd.net/tabpower/cse6230proposal.pdf proposal])
* 2009-11-12: CSE 6230 checkpoint (see [https://nick-black.com/tabpower/cse6230proposal.pdf proposal])
* 2009-11-13: CS 8001-CAS [http://comparch.gatech.edu/arch_whisky/fall09.html Arch-Whiskey] presentation
* 2009-11-13: CS 8001-CAS [http://comparch.gatech.edu/arch_whisky/fall09.html Arch-Whiskey] presentation
* 2009-12-10: CSE 6230 [http://dank.qemfd.net/tabpower/cse6230finalpaper.pdf final report] (again, see proposal)
* 2009-12-10: CSE 6230 [https://nick-black.com/tabpower/cse6230finalpaper.pdf final report] (again, see proposal)
* 2010-01-24: [http://www.usenix.org/events/hotpar10/cfp/ HotPar2010] submission deadline
* 2010-01-24: [http://www.usenix.org/events/hotpar10/cfp/ HotPar 2010] submission deadline


===Current Issues===
===Logo===
This report lists all unresolved bugs having severity "major", "critical", or "blocker":
{{#bugzilla:
  |product=libtorque
  |severity=blocker,critical,major
  |status=new,reopened,assigned
  |lastcomment=1
}}
 
===Logos===
<pre>888 ,e, 888        d8          "...tear the roof off the sucka..."
<pre>888 ,e, 888        d8          "...tear the roof off the sucka..."
888  "  888 88e  d88    e88 88e  888,8,  e88 888 8888 8888  ,e e,
888  "  888 88e  d88    e88 88e  888,8,  e88 888 8888 8888  ,e e,
Line 258: Line 253:
_____________________________________________ 888 _________________
_____________________________________________ 888 _________________
continuation-based unix i/o for manycore numa\888/© nick black 2009</pre>
continuation-based unix i/o for manycore numa\888/© nick black 2009</pre>
{|width="100%"
 
|align="right"|'''''A work of tightly-controlled audacity'''''
==Corrections==
|}
* Marc Lehmann of the [http://software.schmorp.de/pkg/libev.html libev] project pointed out some errors regarding my characterization of that library. Thanks, Marc!
[[Category: Projects]]
Retrieved from "https://nick-black.com/dankwiki/index.php?title=Libtorque"