Schwarzgerät III

My 2020 rebuild, Schwarzgerät II, was a beast of a machine. Like the Hubble or LHC, however, I had to give it more powah. The 2022 upgrade, Schwarzgerät III, does just that despite scrotumtightening supply chain madness. This rebuild focused on cooling, power, and aesthetics.

“But out here, down here among the people, the truer currencies come into being.”―Gravity's Rainbow (1973)

I hesitated to call this the third iteration of Schwarzgerät, as there was no real compute upgrade (I did go from 128GB of DDR4-2400 to 256GB of DDR4-3000, but that hardly counts). The CPU, motherboard, and GPU are unchanged from 2020's upgrade. Nonetheless, the complete rebuild of the cooling system (and attendant costs, both in parts and labor) and radically changed appearance seemed to justify it.

This is my first machine with internal lighting, and also my first to use custom-designed parts (both mechanical and electric). I learned OpenSCAD and improved my 3d printing techniques during this build, and also extended my knowledge of electronics, cooling, and fluids. I also developed a better understanding of power distribution. In that regard―and also with regards to the final product―I consider the build a complete success.

I bought the Corsair iCUE Commander Core Pro after having been informed that it had a Linux driver. Unfortunately, this driver only provides control of the fans. I extended it, along with OpenRGB, to fully support the device. These patches will of course make their ways upstream. I must say that it's incredibly satisfying to use a computer for which you wrote code and designed parts. In the future, I'd like to try fabricating my own chassis, and perhaps even my own PSU.

I'm not sure where else I can go with this machine. There doesn't appear to be much useful work I can do beyond what I've aleady done. Some thoughts:

• Case modding. I could add a window to the PSU door, and improve on the piping in the back. I'm very hesitant to go cutting apart the irreplaceable CaseLabs Magnum T10, though; it's not like I can go buy another one.
• Grow down. If I could find (or more likely fabricate) a pedestal for the machine, I could add extensive radiator support, or a second motherboard. I don't really need either, though.
• Mobility. Work towards the Rolling War Machine by augmenting the existing accelerometer with sensing and movement capabilities. Kind of a big (and expensive) box (and small condo) to be tearing around on its own initiative.
• Hard tubing. Regarded a superior look by many in the watercooling community, but I don't really think so<—in a big case like this, I dig the more organic look of soft tubing. It's also infinitely less annoying to work with.

Bill of materials

We're approaching the $10,000 mark before correcting for inflation, with hard drives alone representing close to $5,000. Materials in this build were acquired over a period going back to 2011 (the LSI Fusion SAS card is, I'm pretty certain, the component longest in my possession). This most recent iteration represents less than $2,000 of components, most of that being $1,150 for the 256GB of RAM (I did manage to sell my old RAM for $200, but we can't deduct that, unless we included its original cost).

Chassis

As always, I rode into battle atop my beloved CaseLabs Magnum T10. CaseLabs went ignominiously out of business in August 2018, and spare parts a la carte are now effectively unavailable. Nonetheless, it remains a truly legendary artifact, perhaps the single greatest case ever built. This build makes more complete use of it than I ever have before.

• Caselabs Magnum T10 chassis with 78mm ventilated top plus...
  • 3x Caselabs MAC-101 HDA+fan cages
  • Caselabs MAC-113 120mm fan mount
• StarTech HSB4SATSASBA 4-bay 3U HDD cage
• Icy Dock MB324SP-B 4-bay 1U SSD cage
• Self-designed and -printed case for Arduino MEGA 2560
• Self-designed and -printed case for RHElectronics Geiger counter
• Self-designed and -printed covering case for EKWB Quantum Kinetic FLT 240 mounting kit
• Self-designed and -printed cable shroud for bottom of Gigabyte Aorus Master TRX40
• Self-designed and -printed false floor for bottom of Caselabs Magnum T10 PSU side
• DEMCiflex magnetic dust filter pack for CaseLabs Magnum TH10

Cooling

An entirely custom water loop with redundant D5 pumps (either can drive the entire loop, though of course with less flow). I can partially drain and fill the loop without touching anything through the externally-mounted Kinetic FLT. Full draining and optimal filling proceed via the 5.25"-mounted Monsoon, sitting at the bottom of the case; this requires removing the USB bay installed above it.

There are 15 120mm fans, 4 140mm fans, 1 80mm fan (in the 4x3.5 bay), and 1 40mm fan (in the 4x2.5 bay). There's also a 55mm chipset fan in the lower-right corner of the motherboard, and a 30mm fan under the IO shield (now uselessly) attempting to cool the VRMs.

• EKWB Quantum Kinetic FLT 240 D5 pump + reservoir with mounting brackets. Installed halfway up the case's back, outside. Pump is an EK Laing PWM D5.
• Monsoon MMRS Series II D5 pump housing + reservoir with 2x Silver Bullet biocide G1/4 plugs. Installed at the front bottom of the case, in the lowest two 5.25" bays.
• EK Laing Vario D5 pump installed into the Monsoon.
• Bitspower BP-MBWP-CT G1/4-10K temperature sensor. Installed in Quantum FLT's central front plug, running to motherboard's first external temp sensor.
• XS-PC G1/4-10K temperature sensor. Installed in Monsoon's upper left plug, running to Corsair iCUE Commander Core XT's first external temp sensor.
• DiyHZ aluminum shell flowmeter and temperature sensor. LCD screen displays both values, and a 3-pin connector carries away flow information.
• EKWB Aorus Master TRX40 monoblock.
• EKWB EK Vector 20x0 waterblock.
• Hardware Labs GTR340 radiator, mounted to top PSU side.
• Hardware Labs Black Ice Nemesis GTS340 radiator, mounted to top motherboard side.
• 2x Hardware Labs XFLOW 240 crossflow radiators, mounted to bottoms.
• 4x Noctua NF-A12x25 chromax.black 140mm fans, connected to Corsair Commander Core, mounted in top
• Noctua NF-A8 chromax.black 80mm fan, replacing original fan in IcyDock drive bay
• 2x Noctua chromax.black NF-A12 PWM fans on GTS360
• Noctua iPPC-2000 PWM fan on GTS360
• Noctua iPPC-2000 PWM fan mounted in front Flex-Bay
• 2x Noctua chromax.black NF-A12 PWM fans on mobo-side 240 XFLOW
• 2x Noctua redux NF-P12 PWM fans on PSU-side 240 XFLOW
• 2x EK Vardar PWM fans on GTR360
• Noctua NF-A15 fan on GTR360
• 3x Noctua NF-A15 fans on drive cages
• Noctua iPPC-3000 PWM fan on top drive cage area
• Fancasee 1-to-4 PWM splitter
• Silverstone 8-way PWM splitter, SATA power

Compute

• AMD Ryzen Threadripper 3970X dontriaconta-core (32 physical, 64 logical) Zen 2 7nm FinFET CPU. Base clock 3.7GHz, turbo 4.5GHz. Overclocked to 4.1GHz.
• Gigabyte Aorus Master TRX40 Revision 1.0. Removed factory heating solution, replaced with EKWB monoblock.
• 8x Kingston Fury Renegade RGB DDR4-3000 32GB DIMMs for 256GB total RAM.
• EVGA GeForce RTX 2070 SUPER Black Gaming 6GB GDDR6 with NVIDIA TU104 GPU. Installed in topmost PCIe 4.0 16x slot, though this is only a 3.0 card.
• ELEGOO MEGA 2560 Revision 3, connected to NXZT internal USB hub, mounted to back of PSU chamber

Power

Power ended up being a tremendous pain in the ass.

• EVGA Supernova Titanium 850 G2. Can provide 850W of 12V power, but only 100W of 5V (this would be important later. read on...)
• BitFenix 3-way Molex expander
• 2x PerformancePCs PCIe-to-Molex converters
• BitFenix Molex-to-4xSATA converter
• 2x 4-way SATA expanders

Storage

I love the CableDeconn bunched SATA data cables; they're definitely the only way to fly, assuming lack of SATA backplanes. We end up with 4x 3.5 drives in the bay, 10x 3.5 drives in CaseLabs cages in the PSU side, 3x M.2 devices in the motherboard PCIe 4.0 slots, and 2x M.2 devices in the HyperX card. This leaves room for 2 more M.2s in the card, and 4x 2.5 devices in the smaller bay. The bottom 2 slots in the bottom hard drive cage are blocked by the PSU-side radiator; indeed, I had to take a hacksaw to said cage to get it into the machine.

See my analysis of how to best make use of 14 drives. I went with a striped raid2z for my 14 Exos drives (also known as a RAID60 in the Old English), yielding 180TB usable from 252TB total. I suffer data loss if I lose any combination of 4 drives, and can lose data if I lose certain combinations of 3 drives (any combination where all three lost drives are in the same raid2z), but no rebuild ever involves more than seven drives.

• 14x Seagate Exos X18 28TB 7200 rpm SATA III drives
• Asus HyperX 4x M.2 PCIe 3.0 x16
• LSI Fusion PCIe 2.0 x8 2x SAS
• Joylifeboard ASM1166 PCIe 3.0 x1 6x SATA
• 2x Samsung 970 EVO Plus 2TB NVMe M.2 in raidz1
• 2x Western Digital Black SN750 1TB NVMe M.2 in raid1z
• 1x Intel Optane 16GB M.2
• 2x CableDeconn SAS-to-4xSATA cables for use with LSI Fusion
• 2x CableDeconn 4x SATA cables for use with motherboard
• CableDeconn 6x SATA cable for use with ASM1166

Interfaces

• NXZT internal USB 2.0 hub, magnetically attached to underside of PSU
• RHElectronics Geiger counter, wired via 3 pins to 2560 MEGA, mounted to back of PSU chamber
• Corsair ICUE Commander Core XT RGB/fan controller, mounted in top, connected to NXZT internal USB hub. I extended OpenRGB and the Linux kernel to drive this device.
• Monsoon CCFL 12V inverter, mounted to top, powered via SATA power connector attached to 12V Molex attached to video power line
• GY-521 board for MPU 6050 accelerometer + gyro, wired via 8 pins to 2560 MEGA, mounted to back of PSU chamber

Lighting

• 4x Corsair ARGB LED lines, attached in series to Corsair Commander Core, attached via adhesive around top.
• 2x 12V RGB LED lines, backlighting top radiators, attached to motherboard's top RGB header via 1-to-2 RGB splitter.
• 2x green PerformancePCs CCFL rods, attached to Monsoon inverter, mounted to back inner corner of each chamber.
• ARGB lines on EVGA Quantum FLT and Aorus Master monoblock, attached to motherboard's top and bottom ARGB headers respectively.
• RGB tops on Rage DIMMs are unmanaged, and self-synchronize via infrared.