Schwarzgerät

“Money has only a different value in the eyes of each.” ―William Makepeace Thackeray, Vanity Fair (1847), Chapter XLIV

ABSTRACT: In August of 2016, I pulled the trigger on a long-planned workstation build. That same month, Intel and NVIDIA dropped new product. Hot, salivation-provoking product: densely packed marvels bursting with FLOPS, rough dank beasts woven up from high-κ 16nm and 14nm strained-silicon FinFETs. Both companies, utterly dominant at their markets' high ends, announced pricing that effectively shoved atom bombs up the asses of computing enthusiasts throughout the free world.

This is the rambling, poorly-edited, relentlessly technical story of that build. Also: epigraphs.

Intro: 10 cores of garbage

“Their way to the City lay through this town of Vanity, they contrived here to set up a fair; a fair wherein should be sold of all sorts of vanity, and that it should last all the year long. Therefore at this fair are all such merchandise sold: as houses, lands, trades, places, honours, preferments, titles, countries, kingdoms; lusts, pleasures, and delights of all sorts – as whores, bawds, wives, husbands, children, masters, servants, lives, blood, bodies, souls, silver, gold, pearls, precious stones, and what not.

And moreover, at this fair there is at all times to be deceivers, cheats, games, plays, fools, apes, knaves, and rogues and that of every kind.”

―John Bunyan, The Pilgrim's Progress (1678), Part I

As I began researching recent years' incremental improvements to high-end desktop technologies, one thing seemed obvious: the Intel Broadwell-E Core i7 6950X was laughably useless, a processor distinguished only by its price, a money-grab narrowly targeted at suckers. Broadly dismissed in reviews, its clock speed and microarchitecture are inferior to the much cheaper Skylake i7 6700K, its cost per core exceeds that of all but the largest Broadwell-EP Xeon E5 v4s, and it doesn't support multisocket configurations. Arstechnica put it best in their Broadwell-E review:

Intel is somewhat shooting itself in the foot with the pricing on the i7-6950X. The recently released Xeon Broadwell-EP processor list includes the Xeon E5-2640 v4: a 10-core 2.4 GHz/3.4 GHz part that runs at 90W, and is priced at $939, which compares favorably to the i7-6950X and its 10-cores at a 3.0 GHz/3.5 GHz clockspeeds. And because it’s a Xeon E5 processor processor, with the right motherboard a user can put two into the same machine for 20 cores/40 threads for only $1878, or only $150 more than the 10-core i7-6950X.

I thus felt weird several days later buying a 6950X ($1650 at Amazon). I'd since concluded that the black sheep Broadwell-E Extreme decacore is actually the best option for a very small class of users, including myself. Keep reading. I promise to explain this (tl;dr: you can't overclock xeons, and C612 multisocket motherboards don't give me what I want).

With that said, Intel and NVIDIA sure are fucking us for all we're worth on premium components.

Build Goals

“There is no way to tell his story without telling my own. And if his story really is a confession, then so is mine.” ―Apocalypse Now (1979)

I am not a gamer. If you're building a gaming rig, buy the excellent Skylake 6700K and be done with it. Game performance is almost entirely based on on GPUs, games generally make poor use of extra cores, and the reference 6700K clocks in at 4GHz. You'll get a better interface to the southbridge via DMI 3.0, you'll draw less power, and you'll save money sufficient to buy a video card. You'll have a lot less PCIe capability, though: Skylake offers 16 PCIe 3.0 lanes, and another 20 via the Southbridge, but the Southbridge only gets 4 DMI 3.0 lanes to the processor. A 16 lane video card is thus going to monopolize your die's direct PCIe hookup.

My intense compute tasks include:

• Compilation of large source packages
• Developing and testing my own software, some of it using GPGPU
• Running large simulations and data analysis, some of it using GPGPU
• Research into high-performance computing and networking
• Occasional virtualization, console emulation, video editing and transcoding

In addition, I require several dozen terabytes of spinning disk, significant solid state storage, XLR audio output driven by a quality DAC, and lots of USB/wireless connectivity. All non-volatile storage must employ redundancy.

Ideally, I want a multisocket NUMA solution to facilitate research on optimizing for such environments. Likewise, I'd like the 512-bit AVX extensions and TSX, for research into making use of these advanced features, and I prefer ECC memory. Beyond that, it's a matter of cores, clocks, lanes, and RAM, the more the better.

I'd like to avoid things I don't intend to use: Intel graphics, VGA outputs, IPMI and other spooky hidden network control stacks, and anything lacking Linux support.

My budget is essentially unlimited, and I expect to drop somewhere between three and five thousand dollars on the machine (not including spinning disks or case). If a particularly attractive option breaks that budget, fine. I'm not looking to spend for the sake of spending, but neither am I optimizing for price: I don't want to find myself wishing I'd splurged on something useful, and I don't want to look at purchased resources sitting idle. Beyond that, total freedom, save one constraint: I already had my case.

The case

“Wovon man nicht sprechen kann, darüber muss man schweigen (The world is all that is the case).” ―Ludwig Wittgenstein, Tractatus Logico-Philosophicus (1921), Chapter VII

I'd purchased a CaseLabs Magnum T10 in 2013 (the Magnum T10 has been discontinued; the linked Magnum TH10A is similar), and was determined to finally make fitting use of it. The T10 is a gorgeous, brilliantly engineered monster of a double-wide reconfigurable case, weighing in at 24 pounds of aluminum. At 15 inches wide, 25.06 inches tall, and 20.06 inches deep (381mm x 637mm x 510mm), it'll easily fit EATX, SSI-CEB or even XL-ATX motherboards, dozens of hard drives, dual PSUs, and the powerful radiators/fans necessary to quietly cool it all. With casters attached, it could transport a small child in comfort and style.

Multisocket and custom liquid-cooled solutions are thus in play. Look upon my works, ye Mighty, and despair!

I can't praise CaseLabs enough. They're extremely expensive, but operating at a different level than anyone else. Even top-of-the-line offerings from Cooler Master, Fractal Design, and Ling Li seem glaringly deficient once you've worked with one of these masterpieces.

Cost: $490

The processor

“Yog-Sothoth knows the gate. Yog-Sothoth is the gate. Yog-Sothoth is the key and guardian of the gate. Past, present, future, all are one in Yog-Sothoth.” ― H. P. Lovecraft, The Dunwich Horror (1929)