UNIX Weapons School: Difference between revisions

No edit summary
No edit summary
Tags: Mobile edit Mobile web edit
 
(32 intermediate revisions by the same user not shown)
Line 1: Line 1:
[[File:UWS.png|right]]
[[File:UWS.png|right]]
CS4803 UWS may well get offered at GT this summer! Amazing.


This syllabus is '''''TENTATIVE''''' and '''''PRELIMINARY'''''!
Lecture slides are available [http://nick-black.com/lectures.html here]. These slides are very much a work in progress; LaTeX source is available on [https://github.com/dankamongmen/unix-weapons-school GitHub].


''Accept the challenges so that you can feel the exhilaration of victory.'' -- George S. Patton
''Accept the challenges so that you can feel the exhilaration of victory.'' -- George S. Patton
[[File:topgun.png|right|thumb|On March 3, 1969, the United States Navy established an elite school for the top one percent of its pilots. Its purpose was to teach the lost art of aerial combat and to ensure that the handful of students who graduated were the best fighter pilots in the world. They succeeded. Today, the Navy calls it Fighter Weapons School. The flyers call it: TOP GUN.]]


UNIX Weapons School is an intense elective practicum. '''Its goal is to train formidably competent systems programmers.'''
UNIX Weapons School is an intense elective practicum. '''Its goal is to train formidably competent systems programmers.'''
* The name owes heritage to Richard Bejtlich's and TaoSecurity's superb [http://www.taosecurity.com/training.html TCP/IP Weapons School] from BlackHat 2007. Thanks Richard!
* ...and through them, of course, to the United States Navy's [http://en.wikipedia.org/wiki/United_States_Navy_Fighter_Weapons_School Fighter Weapons School] aka "Top Gun".
GoogleGroups hosts our (permanent, public) [https://groups.google.com/forum/?fromgroups#!forum/unix-weapons-school mailing list].


Week 0's [http://dank.qemfd.net/tabpower/intro.pdf introductory slides] (draft).
''update from 2021: this class would now absolutely go into [[Io_uring|io_uring]] and [[eBPF]]. also SSDs and the resultant changes to the I/O paradigm. there would probably be some talk of Spectre/Meltdown in conjunction with the architecture lecture, also BIG.little and core scheduling. probably a bit more coverage of [[ARM]] details, though i'll not yet yeet the x86 stuff. the "future of systems programming section" would be updated. a bit more talk about distributed systems, probably, and i'd discard the current material about machine learning. other than that, this still pretty much stands.''
 
==Lectures thus far==
===2013-05-14 Greetings and Salutations===
* "[http://nick-black.com/intro.pdf Intro + official junk]"
** Motivation
** Class policies
** Self-assessment
* "[http://nick-black.com/bigo.pdf Big-O Ain't What it Used to Be]"
** Review of asymptotic complexity
** Case study: The impact of hardware upon matrix multiplication
 
===2013-05-16 Computer Architecture===
* "[http://nick-black.com/comparch.pdf Your Friend the Computer]"
** CMOS power equations and pipelining
** Superscaler and out-of-order
** Delays in dynamically pipelined processors
 
===2013-05-21 The x86===
* "[http://nick-black.com/x86.pdf The x86 is dead. Long live the x86!]"
** Guided tour of the x86's history
** Core frontend + backend
** x86+SSE+VEX instruction set
 
===2013-05-23 UNIX Development===
* "[http://nick-black.com/practicum.pdf Not Sucking in the UNIX Environment]"
** Shell
** C development: Autotools, gcc, binutils
** C++ development: CMake, g++/clang
** Debugging: gdb, valgrind, strace, ltrace
 
===2013-08-01 FINAL EXAM SUMMER 2013===
[[UWC Summer 2013 Final Exam]]


==Cheating==
==Cheating==
Line 17: Line 47:


==Motivation==
==Motivation==
Without systems programmers, hardware would sit silent, stillborn. Without systems programmers, application designers shout mutely into a void.
<blockquote>"The TOPGUN course is designed to train already experienced Navy and Marine Corps aircrews at the graduate level in all aspects of strike-fighter aircraft employment, including tactics, hardware, techniques and the current world threat for air-to-air and air-to-ground missions. The course includes eighty hours of lectures and twenty-five flights that pit students against TOPGUN instructors. When a pilot or WSO completes the TOPGUN course he/she will return as a Training Officer carrying the latest tactical doctrine back to their operational squadron, or go directly to an FRS squadron to teach new aircrews."</blockquote>


The reconfigurability of software combined with the system bounding of hardware allows the systems programmer the broadest space with which to innovate. Unfortunately, most of these possible innovations are just bugs, or (within the innermost ring) superbugs. When that signed vs unsigned comparison means the wrong hard drive sector is selected, it can ruin your whole day.
Without systems programmers, hardware sits silent, stillborn. Without systems programmers, application designers shout mutely into a void.


Systems programming unites the most fundamental realms of our science: [[Programming Language Theory|programming language theory]], [[Compiler Design|compiler design]], [[architecture]], and algorithms come together in almost every project. The systems programmer, sitting in the neck of abstractions' hourglass, need never ask "and at the level below me, how does it work?" When the systems programmer fails, computations become uncomputable. It's important to do it right.
The reconfigurability of software combined with the system bounding of hardware allows the systems programmer the broadest space with which to innovate. Unfortunately, most of these possible innovations are just bugs, or (within the innermost ring) superbugs. When that signed vs unsigned comparison means the wrong hard drive sector is selected, it can ruin your whole day. Systems programming unites the most fundamental realms of our science: [[Programming Language Theory|programming language theory]], [[Compiler Design|compiler design]], [[architecture]], and algorithms come together in practical application. These theories are complex enough to demand classes of their own. Here, we will integrate these prerequisite theories to deliver efficient, robust systems software solutions. Furthermore, you'll become acquainted with the gritty details of cutting-edge technique.


==Grading==
==Grading==
Line 30: Line 60:
<br>            -- Dante Alighieri, ''The Inferno'', Canto III, 14-15</blockquote>
<br>            -- Dante Alighieri, ''The Inferno'', Canto III, 14-15</blockquote>


Because I am hard, you will not like me. But the more you hate me, the more you will learn.
[https://www.youtube.com/watch?v=tHxf17yJsKs Because I am hard, you will not like me. But the more you hate me, the more you will learn.]


Each project will be evaluated for both correctness (in terms of unit tests passed) and performance (on several reference platforms). Performance of incorrect code is, of course, moot. I will:
Each project will be evaluated for both correctness (in terms of unit tests passed) and performance (on several reference platforms). Performance of incorrect code is, of course, moot. I will make each project public two weeks prior to its due date, and provide a set of basic unit tests and a web-accessible oracular reference implementation (you can submit inputs and receive outputs). For details of grading, see the first week's slides. I am using a scheme designed to maximize competition and conflict between class members, which I hope will reduce cheating, lead to great personal exertions, and possibly get a chair thrown.
* Made the project public two weeks prior to its due date
* Provide a set of basic unit tests
* Provide a web-accessible oracular reference implementation (you can submit inputs and receive outputs)
* Collect projects
* Review them by hand, looking for lacunae. Any problems spotted will be added to the unit tests.
* Run all projects against the collected unit tests.
* Eliminate (award a score of zero to) any projects which irregularly halt, or do not halt within some time.
* Each unit test T_n will be successfully completed by some S_n projects, ordered (0--S_n-1) by decreasing time to completion (i.e. slowest is 0).
** If S_n is 1, the single successful project receives 1.5 points.
** Otherwise, each of the S_n successful projects receives points equal to |sortpos| / S_n-1.
** Thus, the fastest will receive 1 point, and those between always get more points for beating more people while being bested by fewer
*** And the slowest might as well have been wrong, which I love.
*** Unless, of course, the slowest was also the fastest. Good for you!


This scheme ought maximize competition and conflict between class members, which I hope will reduce cheating, lead to great personal exertions, and possibly get a chair thrown.
{{:UNIX Weapons School Weekplan}}


{{:UNIX Weapons School Weekplan}}
==Thanks==
* Bill Leahy, for causing GT to pick UWS up as a class, at a time when it was nothing more than a single-page PDF flyer.
* David Kanter of [http://realworldtech.com Real World Tech] for use of his superb processor diagrams, and his years of insightful writing besides.
* Richard Bejtlich's and TaoSecurity's [http://www.taosecurity.com/training.html TCP/IP Weapons School] from BlackHat 2007, for nomenclature.
** ...and through them, of course, to the United States Navy's [http://en.wikipedia.org/wiki/United_States_Navy_Fighter_Weapons_School Fighter Weapons School] aka "Top Gun".