Schwarzgerät III

(65 intermediate revisions by the same user not shown)

Line 4:

My 2020 rebuild, [[Schwarzgerät_II|Schwarzgerät II]], was a beast of a machine. Like the Hubble or LHC, however, I had to give it more powah. The 2022 upgrade, Schwarzgerät III, does just that despite scrotumtightening supply chain madness. This rebuild focused on cooling, power, and aesthetics. I require that the machine be no louder than ambient noise (I live in the middle of [[Atlanta|Midtown Atlanta]]) when weakly loaded—I ought not hear the machine unless seriously engaged in active computation. At the same time, I want to overclock my 3970X as far as she can (reasonably) go.

'''(further [[Schwarzgerät_III_upgrade|upgraded]] 2022-04, hopefully the last bit for a minute)'''

<blockquote>“But out here, down here among the people, the truer currencies come into being.”―''Gravity's Rainbow'' (1973)</blockquote>

Line 17:

Line 19:

==Bill of materials==

We're approaching the $10,000 mark before correcting for inflation, with hard drives alone representing close to $5,000. Materials in this build were acquired over a period going back to 2011 (the LSI Fusion SAS ~~card~~ is, I'm pretty certain, the component longest in my possession). This most recent iteration represents less than $2,000 of components, most of that being $1,150 for the 256GB of RAM (I did manage to sell my old RAM for $200, but we can't deduct that, unless we included its original cost).

We're approaching the $10,000 mark before correcting for inflation, with hard drives alone representing close to $5,000. Materials in this build were acquired over a period going back to 2011 (one of the LSI Fusion SAS cards is, I'm pretty certain, the component longest in my possession). This most recent iteration represents less than $2,000 of components, most of that being $1,150 for the 256GB of RAM (I did manage to sell my old RAM for $200, but we can't deduct that, unless we included its original cost).

[[File:Caselabs-t10a-parts.jpg|right|400px|thumb|Parts for the Caselabs T10]]

===Chassis===

[[File:Fanmount.jpg|thumb|200px|4x140mm fan struts, designed in OpenSCAD and printed on my [[Voxel]].]]

Line 31:

Line 33:

* Self-designed and -printed case for RHElectronics Geiger counter ([https://github.com/dankamongmen/openscad-models/blob/master/geiger.scad source])

* Self-designed and -printed covering case for EKWB Quantum Kinetic FLT 240 mounting kit ([https://github.com/dankamongmen/openscad-models/blob/master/reservoir-part.scad source])

* Self-designed and -printed cable ~~shroud for bottom of Gigabyte Aorus Master TRX40~~

* Self-designed and -printed PCI brackets with cable channels ([https://github.com/dankamongmen/openscad-models/blob/master/bracketpci.scad source])

* Self-~~designed and -printed false floor for bottom of Caselabs Magnum T10 PSU side~~

* Self-designed and -printed 4x140mm fan mount for roof ([https://github.com/dankamongmen/openscad-models/blob/master/140-interlocking.scad source])

* DEMCiflex magnetic dust filter pack for CaseLabs Magnum TH10

Line 48:

Line 49:

====Bespoke loop====

[[File:Bottom.jpg|200px|thumb|The bottom complexes ~~remind me of turbines~~.]]

[[File:Bottom.jpg|200px|thumb|The bottom complexes suggest turbogenerators.]]

* [https://www.ekwb.com/shop/ek-quantum-kinetic-flt-240-d5-pwm-d-rgb-plexi EKWB Quantum Kinetic FLT 240] D5 pump + reservoir with mounting brackets. Installed halfway up the case's back, outside.

** EK Laing PWM D5 pump installed into the Quantum Kinetic.

** [https://shop.bitspower.com/index.php?route=product/product&product_id=1963 Bitspower BP-MBWP-CT] ~~G1/4~~-10K temperature sensor. Installed in Quantum FLT's central front plug, running to motherboard's first external temp sensor.

** [https://shop.bitspower.com/index.php?route=product/product&product_id=1963 Bitspower BP-MBWP-CT] G¼-10K temperature sensor. Installed in Quantum FLT's central front plug, running to motherboard's first external temp sensor.

* [http://monsooncooling.com/reservoir.php Monsoon MMRS Series II] D5 pump housing + reservoir with 2x Silver Bullet biocide ~~G1/4~~ plugs. Installed at the front bottom of the case, in the lowest two 5.25" bays.

* [http://monsooncooling.com/reservoir.php Monsoon MMRS Series II] D5 pump housing + reservoir with 2x Silver Bullet biocide G¼ plugs. Installed at the front bottom of the case, in the lowest two 5.25" bays.

** EK Laing Vario D5 pump installed into the Monsoon.

** [https://www.xs-pc.com/temperature-sensors/g14-plug-with-10k-sensor-black-chrome XS-PC] ~~G1/4~~-10K temperature sensor. Installed in Monsoon's upper left plug, running to Corsair iCUE Commander Core XT's first external temp sensor.

** [https://www.xs-pc.com/temperature-sensors/g14-plug-with-10k-sensor-black-chrome XS-PC] G¼-10K temperature sensor. Installed in Monsoon's upper left plug, running to Corsair iCUE Commander Core XT's first external temp sensor.

* [https://www.amazon.com/DIYhz-displaydigital-Thermometer-Temperature-Indicator/dp/B09QG3241X DiyHZ aluminum shell] flowmeter and temperature sensor. LCD screen displays both values, and a 3-pin connector carries away flow information.

* [https://www.ekwb.com/shop/ek-quantum-momentum-trx40-aorus-master-d-rgb-plexi EKWB Aorus Master TRX40] DRGB monoblock (nickel+plexi).

* [https://www.ekwb.com/shop/ek-quantum-vector-rtx-re-d-rgb-nickel-plexi EKWB EK-Quantum Vector RTX RE] DRGB waterblock (nickel+plexi).

* [http://hardwarelabs.com/nemesis/gtr/gtr-360/ Hardware Labs Black Ice Nemesis GTR360] 16 FPI 54.7mm radiator, mounted to top PSU side.

* [http://hardwarelabs.com/nemesis/gtr/gtr-360/ Hardware Labs Black Ice Nemesis GTR360] 16 FPI 54.7mm radiator, mounted to top PSU side (393x133mm).

* [https://hardwarelabs.com/blackice/gts/gts-360/ Hardware Labs Black Ice Nemesis GTS360] 30 FPI 29.6mm radiator, mounted to top motherboard side.

* [https://hardwarelabs.com/blackice/gts/gts-360/ Hardware Labs Black Ice Nemesis GTS360] 30 FPI 29.6mm radiator, mounted to top motherboard side (397x133mm).

* 2x [http://hardwarelabs.com/nemesis/gtsxflow/240gts-xflow/ Hardware Labs Black Ice Nemesis GTS240 XFLOW] 16 FPI 29.6mm crossflow radiators, mounted to bottoms.

* 2x [http://hardwarelabs.com/nemesis/gtsxflow/240gts-xflow/ Hardware Labs Black Ice Nemesis GTS240 XFLOW] 16 FPI 29.6mm crossflow radiators, mounted to bottoms (292x133mm).

* Fancasee 1-to-4 PWM splitter, power from header

* Silverstone 8-way PWM splitter, SATA power

* [[SilverStone_fan_hub|Silverstone 8-way]] PWM splitter, SATA power

* 12.5mm ID EKWB [https://www.ekwb.com/shop/ek-tube-zmt-matte-black-19-4-12-5mm ZMT], black

Most calculations will require knowing the total amount of coolant in our system. Fluid volumes of liquid cooling components are irritatingly difficult to establish (once the hardware is installed, anyway). With an inner diameter of 12.5mm, the ZMT has a cross-section of 122.72mm² (0.~~015m²~~). I got the EKWB fluid volumes directly from their support channel. G¼ has a diameter of 13.157mm, but that doesn't tell you anything about the inner diameter, which is not standardized =\.

Most calculations will require knowing the total amount of coolant in our system. Fluid volumes of liquid cooling components are irritatingly difficult to establish (once the hardware is installed, anyway). With an inner diameter of 12.5mm, the ZMT has a cross-section of 122.72mm² (1.2272cm²). I got the EKWB fluid volumes directly from their support channel. G¼ has a diameter of 13.157mm, but that doesn't tell you anything about the inner diameter, which is not standardized =\. I used a graduated cylinder to measure the Monsoon and the radiators. Attachments can be considered 1--2mL each (a drain structure including a ball valve attachment, a T attachment, a 90° adapter and several extenders contained a total of 10mL).

{| class="wikitable"

! Component !! Volume (mL) !! Climb (m) !! θ (rad)

|-

| HWL XFLOW 240 || ? || 0 || 0

| HWL XFLOW 240 || 90 || 0 || 0

|-

| Tubing || ? || 0 || 0

|-

| HWL GTR360 || ? || 0 || 0

| HWL GTR360 || 320 || 0 || 0

|-

| Tubing || ? || 0 || 0

|-

| HWL GTS360 || ? || 0 || 0

| HWL GTS360 || 130 || 0 || 0

|-

| Tubing || 44 || ||

|-

| Aorus Master ~~Monoblock~~ || 45 || || π/2

| Aorus Master monoblock || 45 || || π/2

|-

| Tubing || 11 || ||

|-

| EK-Vector waterblock || 50 || || π/2

Line 94:

Line 95:

| Tubing || 28 || ||

|-

| ~~Flowmeter~~ || ? || || π/4

| DiyHZ flowmeter || 4 || || π/4

|-

| HWL XFLOW 240 || ? || 0 || 0

| HWL XFLOW 240 || 90 || 0 || 0

|-

| Tubing || 11 || 0 || 0

|-

| Monsoon Series Two D5 || ? || 0 || 0

| Monsoon Series Two D5 || 300 || 0 || 0

|-

| Tubing || ? || 0 || 0

Line 108:

Line 109:

Note that water expands (becomes less dense) as it heats (down to 4℃, where it begins to do the opposite), meaning more pressure on its container. The coefficient of volume expansion β(1/℃) for water is 0.00021 at 20℃. This would suggest that a 96℃ rise would result in about a 2% expansion in the total volume, but this is incorrect, due to a changing β over temperature. By the time you get to 60℃ (most tubing isn't rated beyond this temperature), you've got a β closer to 0.0005. Here's the data for pure water at one atmosphere:

~~[[File:WaterB.png|400px|right|Thermal expansion of water as a function of temperature]]~~

{|class="wikitable"

! ℃ !! Density kg/m³ !! Specific weight kN/m³ !! Co. of thermal expansion β 10⁻⁶/K

Line 142:

* 2x EK Vardar PWM on GTR360

* Noctua NF-P12 PWM on GTR360

{| class="wikitable"

! Fan !! Location !! Watts !! db(A) !! CFM !! Pressure

|-

| NF-F12 (2) || GTS360 || 0.6 (1.2) || 22.4 (25.4) || 93.4 (186.8) || 2.61 (5.22)

|-

| NF-F12 iPPC-2000 || GTS360 || 1.2 || 29.7 || 121.8 || 3.94

|-

| NF-A12x25 (2) || GTS240 || 1.68 (3.36) || 22.6 (25.6) || 102.1 (204.2) || 2.34 (4.68)

|-

| NF-P12 redux-1700 (2) || GTS240 || 1.08 (2.16) || 25.1 (28.1) || 120.2 (240.4) || 2.83 (5.66)

|-

| EK Vardar (2) || GTR360 || 2.16 (4.32) || 33.5 (36.5) || 77 (154) || 3.16 (6.32)

|-

| NF-P12 || GTR360 || 0.6 || 19.8 || 120.2 || 2.83

|-

|}

====Blowing in====

Line 149:

Line 166:

* 2x Noctua NF-S12A PWM on lower two drive cages

* Noctua NF-S12A FLX on top drive cage

All values are maxima:

{| class="wikitable"

! Fan !! Location !! Watts !! db(A) !! CFM !! Pressure

|-

| NF-S12A (3) || Drive cages || 1.44 (4.32) || 17.8 (22.6) || 107.5 (322.5) || 1.19 (3.57)

|-

| NF-F12 iPPC-2000 || Flex-Bay || 1.2 || 29.7 || 121.8 || 3.94

|-

| NF-F12 iPPC-3000 || Bottom cage || 3.6 || 43.5 || 186.7 || 7.63

|-

| NF-A8 || 3.5" bay || 0.96 || 17.7 || 55.5 || 2.37

|-

|}

===Compute===

Line 159:

Line 190:

* [https://www.evga.com/products/specs/gpu.aspx?pn=1207a8ec-269d-4e11-91a9-e01226652c9f EVGA GeForce RTX 2070 SUPER] Black Gaming 6GB GDDR6 with NVIDIA TU104 GPU. Installed in topmost PCIe 4.0 16x slot, though this is only a 3.0 card.

* [https://www.elegoo.com/products/elegoo-mega-2560-r3-board ELEGOO MEGA 2560] Revision 3, connected to NXZT internal USB hub, mounted to back of PSU chamber

* [https://ark.intel.com/content/www/us/en/ark/products/60021/intel-ethernet-controller-x540bt2.html Intel X540-BT2 2-port 10Gbps] 10GBASE-T PCI 3.0 card

As I detailed regarding [[Schwarzgerat II]], the 3990X is an amazing achievement in chip design and fabrication, but I believe it to be severely starved for many tasks by its memory bandwidth; with its four memory channels populated, the ThreadRipper 3990X can hit about 90GB/s from fast DDR4; its ~~Epyc~~ brother can pull down ~190 through its eight channels. For my tasks, it's rare enough that I can drive all my 32 cores; with the 3990X, I'd be paying twice as much to hit full utilization less often, and be unable to bring full bandwidth to bear when I did.

As I detailed regarding [[Schwarzgerat II]], the 3990X is an amazing achievement in chip design and fabrication, but I believe it to be severely starved for many tasks by its memory bandwidth; with its four memory channels populated, the ThreadRipper 3990X can hit about 90GB/s from fast DDR4; its EPYC brother can pull down ~190 through its eight channels. For my tasks, it's rare enough that I can drive all my 32 cores; with the 3990X, I'd be paying twice as much to hit full utilization less often, and be unable to bring full bandwidth to bear when I did.

I absolutely 🖤 my 3970X, though. Bitch screams. Anyone overclocking on Linux should be aware of [https://www.linux.org/docs/man8/turbostat.html turbostat].

I absolutely 🖤 my 3970X, though. Bitch screams. Anyone overclocking on Linux should be aware of [https://www.linux.org/docs/man8/turbostat.html turbostat]. The 3970X supports 88 lanes of PCIe 4.0, of which the TRX40 chipset consumes 24, leaving 64 for expansion devices. I've got 16 (GPU) + 16 (Hyper X) + 2x8 (LSI cards) + 12 (M.2 onboards) for 60 total, coming in just under saturation.

===Power===

Line 175:

Line 205:

* 2x 4-way SATA expanders

* 2x [https://www.amazon.com/BANKEE-Converter-Voltage-Regulator-Transformer/dp/B08NZSYZRF Bankee 12V->5V/15A] buck transformers

To make a lengthy story short (it's told in much more detail below), I needed more SATA power cables than my PSU supported. The buck transformers are used to convert the 12V on PCIe cables to 5V+12V as needed by SATA drives.

===Storage===

[[File:Hacksawed.jpg|thumb|200px|Cut down that drive cage!]]

I love the [http://www.cabledeconn.com/Mini-SAS-Series-/SATA-Esata-list-138-160.html CableDeconn bunched SATA] data cables; they're definitely the only way to fly, assuming lack of SATA backplanes. We end up with 4x 3.5 drives in the bay, 10x 3.5 drives in CaseLabs cages in the PSU side, 3x M.2 devices in the motherboard PCIe 4.0 slots, and 2x M.2 devices in the HyperX card. This leaves room for 2 more M.2s in the card, and 4x 2.5 devices in the smaller bay. The bottom 2 slots in the bottom hard drive cage are blocked by the PSU-side radiator; indeed, I had to take a hacksaw to said cage to get it into the machine.

See my analysis of [[13_drives|how to best make use of 14 drives]]. I went with a striped raidz2 for my 14 Exos drives (also known as a RAID60 in the Old English), yielding 180TB usable from 252TB total. I suffer data loss if I lose any combination of 4 drives, and can lose data if I lose certain combinations of 3 drives (any combination where all three lost drives are in the same raid2z), but no rebuild ever involves more than seven drives.

~~All filesystems are ZFS, and all storage enjoys some redundancy (save the 16GB Optane, which is just for persistent memory/DAX experiments).~~

* 14x Seagate Exos X18 18TB 7200 rpm SATA III drives in striped raidz2

* Asus HyperX 4x M.2 PCIe 3.0 x16

Line 192:

Line 218:

* 4x CableDeconn SAS-to-4xSATA cables for use with LSI Fusions

* 2x CableDeconn 4x SATA cables for use with motherboard

I love the [http://www.cabledeconn.com/Mini-SAS-Series-/SATA-Esata-list-138-160.html CableDeconn bunched SATA] data cables; they're definitely the only way to fly (assuming lack of SATA backplanes, which good luck in a workstation form factor). We end up with 4x 3.5 drives in the bay, 10x 3.5 drives in CaseLabs cages in the PSU side, 3x M.2 devices in the motherboard PCIe 4.0 slots, and 2x M.2 devices in the HyperX card. This leaves room for 2 more M.2s in the card, and 4x 2.5 devices in the smaller bay. The bottom 2 slots in the bottom hard drive cage are blocked by the PSU-side radiator; indeed, I had to take a hacksaw to said cage to get it into the machine.

How to arrange 14 disks so as to create a single volume? I don't bother with solutions involving hot spares:

{| class="wikitable"

! Name !! Setup !! Availability !! MaxTol !! MinFail !! Max Rebuild

|-

| raid3z

| 11 data, 3 parity

| 78.6% (11)

| 3

| 4

| 100% + fail

|-

| raid2z

| 12 data, 2 parity

| 85.7% (12)

| 2

| 3

| 85.7% + fail

|-

| striped raid3z

| 2x(4 data, 3 parity)

| 57.1% (8)

| 3 + 3

| 4

| 50% + fail

|-

| striped raid2z

| 2x(5 data, 2 parity)

| 71.4% (10)

| 2 + 2

| 3

| 50% + fail

|-

| striped raidz

| 2x(6 data, 1 parity)

| 85.7% (12)

| 1 + 1

| 2

| 50% + fail

|-

| mirrored raid3z

| 2x(4 data, 3 parity)

| 28.6% (4)

| 7 + 3

| 8

| 50% + fail

|-

| mirrored raid2z

| 2x(5 data, 2 parity)

| 35.7% (5)

| 7 + 2

| 6

| 50% + fail

|-

| mirrored raidz

| 2x(6 data, 1 parity)

| 42.9% (6)

| 7 + 1

| 4

| 50% + fail

|-

| raidz3 of stripes

| 7x(2) 4 data, 3 parity

| 57.1% (8)

| 2 + 2 + 2

| 4

| 100% + fail

|-

| raidz2 of stripes

| 7x(2) 5 data, 2 parity

| 71.4% (10)

| 2 + 2

| 3

| 85.7% + fail

|}

I went with a striped raidz2 for my 14 Exos drives (also known as a RAID60 in the Old English), yielding 180TB usable from 252TB total (71.4%). I suffer data loss if I lose any combination of 5 drives, and can lose data if I lose certain combinations of 3 drives (any combination where all three lost drives are in the same raid2z), but no rebuild ever involves more than seven drives. All filesystems are ZFS, and all storage enjoys some redundancy (save the 16GB Optane, which is just for persistent memory/DAX experiments). I could have split the volume and had two raidz2s with no interdependence, but eh.

===Interfaces===

Line 197:

Line 303:

* NXZT internal USB 2.0 hub, magnetically attached to underside of PSU, connected to motherboard USB 2.0 header

* [https://rhelectronics.net/store/radiation-detector-geiger-counter-diy-kit-second-edition.html RHElectronics] Geiger counter, wired via 3 pins to 2560 MEGA, mounted to back of PSU chamber

* [https://www.corsair.com/us/en/Categories/Products/Accessories-%7C-Parts/CORSAIR-iCUE-COMMANDER-CORE-XT-Smart-RGB-Lighting-and-Fan-Speed-Controller Corsair ICUE Commander Core XT] RGB/fan controller, mounted in top, connected to NXZT internal USB hub~~. I extended OpenRGB and the Linux kernel to drive this device.~~

* [https://www.corsair.com/us/en/Categories/Products/Accessories-%7C-Parts/CORSAIR-iCUE-COMMANDER-CORE-XT-Smart-RGB-Lighting-and-Fan-Speed-Controller Corsair ICUE Commander Core XT] RGB/fan controller, mounted in top, connected to NXZT internal USB hub

* Monsoon CCFL 12V inverter, mounted to top, powered via SATA power connector attached to 12V Molex attached to video power line

* GY-521 board for [http://invensense.com/mems/gyro/documents/PS-MPU-6000A-00v3.4.pdf MPU 6050] accelerometer + gyro, wired via 8 pins to 2560 MEGA, mounted to back of PSU chamber

Line 204:

Line 310:

===Lighting===

* 4x Corsair ARGB LED lines, connected in series to Corsair Commander Core, attached via adhesive around ~~top~~.

* 4x Corsair 410mm ARGB LED lines, connected in series to Corsair Commander Core, attached via adhesive around bottom cooling complex of the motherboard side.

* 2x 12V RGB LED lines, backlighting top radiators, attached to motherboard's top RGB header via 1-to-2 RGB splitter.

* 2x green PerformancePCs CCFL rods, attached to Monsoon inverter, mounted to back inner corner of each chamber.

* 5x green PerformancePCs CCFL rods, attached to Monsoon DC->AC inverter, mounted to back inner corner of each chamber.

* ARGB lines on EVGA Quantum FLT and Aorus Master monoblock, attached to motherboard's top and bottom ARGB headers respectively.

* RGB tops on Rage DIMMs are unmanaged, and self-synchronize via infrared.

Line 212:

Line 318:

* Piratedog [https://piratedog.tech/collections/rgb-splitters/products/corsair-rgb-lighting-channel-splitter Corsair RGB] 1-to-2 splitter

* EZDIY-Fab [https://www.amazon.com/gp/product/B09HGP9YMK/ref=ppx_yo_dt_b_asin_title_o03_s00?ie=UTF8&psc=1 ARGB LED] strips

I currently have no window on my PSU side, so there's no point putting lighting in there. I do have a short strip of 12V LEDs just in case I need to work in there without light (though presumably with the machine on). I'd like to get these on a switch or something. Ideally both these and the CCFLs would be on a switch with a battery backup. I'm sad that I bought into the Corsair ecosystem. It's almost entirely closed and proprietary, even the connectors. Props to [https://piratedog.tech/ PirateDog Tech] for working around this.

==Distributing power==

[[File:Tobsun.jpg|200px|thumb|A 12V->5V buck transformer~~. By upping the output amperage~~, ~~it is~~ more efficient than a voltage regulator.]]

[[File:Tobsun.jpg|200px|thumb|A 12V->5V buck transformer, more efficient than a mere voltage regulator.]]

I began to run into some serious power issues on this build, originating in the Exos X18 drives (of which, you might remember, there are 14). It will be worth your time to consult the [https://www.seagate.com/www-content/product-content/enterprise-hdd-fam/exos-x18/_shared/en-us/docs/100865854a.pdf Exos 18 manual]. Remember, 12V is for the motor, and 5V is for the logic.

Line 237:

Line 345:

[schwarzgerat](0) $ sudo pwrstat -status

Model Name................... CP1500PFCLCD

~~Firmware Number.............. 000000000000~~

Rating Voltage............... 120 V

Rating Power................. 900 Watt

Line 248:

Line 355:

Load......................... 333 Watt(37 %)

Line Interaction............. None

~~Test Result.................. Unknown~~

Last Power Event............. Blackout at 2020/03/08 19:16:10 for 24 sec.

[schwarzgerat](0) $

Line 261:

Line 367:

<li>Molex (12V only): 2 (pumps)</li>

<li>Molex (5V only): 2 (flowmeter, USB hub)</li>

<li>SATA (12V+5V): 14 (10 disks, 2 for StarTech bay, 1 for IcyDock bay~~, Corsair~~)</li>

<li>SATA (12V+5V): 13 (10 disks, 2 for StarTech bay, 1 for IcyDock bay)</li>

<li>SATA (12V only): 2 (inverter, fan controller)</li>

<li>SATA (5V only): 2 (USB front bays)</li>

<li>SATA (5V only): 3 (USB front bays, Corsair)</li>

<li>3-pin ARGB: 4 (ARGB headers, 5A each)</li>

Line 270:

Line 376:

</ul>

I got these numbers from product documentation or, in the case of the EKWB LEDs, from the manufacturer's support channel.

I got these numbers from product documentation or, in the case of the EKWB LEDs, from the manufacturer's support channel. Corsair LED strip information came from [https://forum.corsair.com/forums/topic/155937-corsair-rgb-current-usage/#comment-917706 PirateDog Tech]. The Corsair iCUE Commander XT is rated for a maximum of 4.5A total.

{| class="wikitable"

! Item !! 5V watts !! 12V watts !! Source

Line 311:

Line 417:

|-

| Monsoon inverter || 0 || ? || SATA

|-

| Monsoon LEDs || 0 || ? || Molex

|-

| Exos18 (x14) (spinning) || 4.6 (64.4) || 7.68 (107.52)|| SATA

Line 319:

Line 427:

|-

| EKWB Aorus TRX40 Monoblock LEDs || 1.05 || 0 || ARGB header

|-

| DiyHZ light strips || ? || 0 || ARGB header

|-

| Corsair Commander Pro XT (logic) || ? || 0 || SATA

|-

| Corsair LED strips (x4) || ? (?) || 0 || Corsair (SATA)

| Corsair LED strips (x4) || 1.835 (7.34) || 0 || Corsair (SATA)

|-

| DiyHZ flowmeter || ? || 0 || Molex

Line 337:

Line 447:

|-

|}

The Silverstone supports draws up to 4.5A (54W). The Corsair supports a 12V fan draw of 4.5A (54W) and a 5V RGB draw of 4.5A (22.5W).

The PSU claims support of up to 100W as 5V (20A), and up to 850W as 12V (71A). The 5V line is dominated by hard drives, which will consume 65W when in active use (and 70W when spinning up). Beyond them, there are very few 5V draws; the ARGB light strips are going through the motherboard's ARGB headers, the internal USB hub and downstream devices are getting 5V from Molex, and the Corsair is pulling over SATA. Even if the motherboard ARGB headers are fed by the ATX12V's 5V lines, we're talking 20W max, and probably less than that.

We're all clear on 12V until overclocking comes into play. The GPU and CPU will never be maxed out while the drives are spinning up (provided that we don't spin down the motors after startup), so the drives will be consuming 107W instead of 339W. The stock CPU+GPU are 495W, 46W for pumps, and some unknown but substantial amount is consumed by the motherboard; let's estimate a very generous 100W. That's 748W total. With 750W left after 100W are devoted to 5W, we're cutting it close, but that ought be a rare load. In practice, the machine tends to take about 300W in everyday use, up to about 600W under computational load; to really push the envelope, I'd need force heavy computational and I/O load. I'll see what happens next time I build a kernel while my ZFS is rebuilding.

===The drive problem===

Line 372:

Line 488:

We have six remaining items requiring 5V power: the flowmeter, the Corsair, the internal USB hub, and the three front panel USB bays. Of these, three natively want Molex 4-pin, and the other 3 want SATA. Combined with our 14 drives, that's 20 power drains.

I'm initially ~~solving~~ this problem using PUIS (Power-Up In Standby), a feature of the SATA specification. Enabling this feature on a disk will prevent it from spinning up until it receives a particular command, which can be issued by the OS (so long as you don't need your system firmware to recognize the disk, which will be unreadable until this command is sent). I intended to solve this ~~later~~ with 12V relays (or better yet, an [https://www.crowdsupply.com/sequent-microsystems/8-mosfet 8-MOSFET]) controlled by the Arduino, but instead ~~I think I'm going to use a~~ 12V->5V buck ~~transformer~~, splice the 12V line of my remaining PCIe cables (see below), and ~~make~~ myself an underpowered 4-pin Molex from it. Each PCIe cable and its 66W of 12V ~~ought be~~ good for three drives~~, if I'm right about the whole feasibility of that plan~~.

I initially solved this problem using PUIS (Power-Up In Standby), a feature of the SATA specification. Enabling this feature on a disk will prevent it from spinning up until it receives a particular command, which can be issued by the OS (so long as you don't need your system firmware to recognize the disk, which will be unreadable until this command is sent). This had the downside of requiring me to keep my disks spinning once the machine had started, since I'd otherwise run into the problem anew should they all spin back up at the same time.

I at first intended to solve this with 12V relays (or better yet, an [https://www.crowdsupply.com/sequent-microsystems/8-mosfet 8-MOSFET]) controlled by the Arduino, but instead used two 12V->5V buck transformers, splice the 12V line of my remaining PCIe cables (see below), and made myself underpowered 4-pin Molex cables from them. Each PCIe cable and its 66W of 12V is good for three drives when taken through three Molex-to-SATA adapters.

===Power sources===

Line 384:

Line 502:

| Perif (Molex) || 0 || 55 || 132 || Can bridge to SATA or PCIe. Wiring might not be safe for the full pin capacity.

|-

| PCIe x4|| 9 || 0 || 66 || Can bridge to (underpowered) 12V-only (2-pin) Molex. Can probably bridge (with buck transformer) to underpowered Molex.

| PCIe x4|| 0 || 0 || 75 || Can bridge to (underpowered) 12V-only (2-pin) Molex. Can probably bridge (with buck transformer) to underpowered Molex.

|-

|}

Line 414:

Line 532:

==Regrets and stupidities==

[[File:Stupidgpu.jpg|200px|thumb|There are at least two reasons why this GPU's waterblock configuration is suboptimal.]]

* I ought have wrapped all the tubing in anti-kink coils, which I already had, way earlier.

* I ought have wrapped all the tubing in anti-kink coils (which I already had!) way earlier.

* I ought have bought a second LSI SAS card from the outset, rather than going through two POS chinesium 1x cards, both of which continuously reset attached disks (I'm guessing due to my fairly lengthy cables? who knows).

* I ought have paid more attention to which fans I was putting on which radiators.

Line 442:

Line 560:

<li>Replace the 45-degree adapter connecting the flowmeter to the radiator with a 90-degree adapter. This will lower the flowmeter, putting its output near its original location, eliminating any tubing length gains.</li>

</ol>

The reservoir will be raised no more than the height of the fan being added to the radiator; it will likely move from the bottom bay to the bay immediately above that. Essentially, we're raising the entire floor of our cooling system by one fan's height. I'll be using Phanteks T30-120s, so that's 30mm. We could theoretically move everything higher with a shroud around these fans, but it would make working on the machine a goddamn nightmare.

The reservoir will be raised no more than the height of the fan being added to the radiator; it will likely move from the bottom bay to the bay immediately above that. Essentially, we're raising the entire floor of our cooling system by one fan's height. I'll be using [[Phanteks|Phanteks T30-120s]], so that's 30mm. We could theoretically move everything higher with a shroud around these fans, but it would make working on the machine a goddamn nightmare.

When I do this, I'll likely raise the Vario pump's setting from 2 to 3. I might even replace the Vario pump with a proper PWM one. It would also be a fine time to [https://kampidh.blogspot.com/2019/11/down-rabbit-hole-d5-pwm-pump-flaw-and.html fix my other D5] so it reports tach properly.

==Ongoing issues==

* My flow rate is lower than I would like, assuming this flowmeter to be reliable. Maxing out the PWM pump hits desirable flow rates, but with more noise than I want. I think there might be some flow reduction in the tubing between the flowmeter and Quantum Kinetic; I'm considering replacing that with hardline. For all I know, there's still a big air pocket in some radiator.

* <s>My flow rate is lower than I would like, assuming this flowmeter to be reliable. Maxing out the PWM pump hits desirable flow rates, but with more noise than I want. I think there might be some flow reduction in the tubing between the flowmeter and Quantum Kinetic; I'm considering replacing that with hardline. For all I know, there's still a big air pocket in some radiator.</s> there was indeed a big air pocket

* <s>My onboard Ethernet stopped working, and I'm still not sure why.</s> the ethernet cable wasn't plugged into the outlet all the way, lol, you dumb idiot

* <s>I'm getting a kernel oops in the igb driver on startup, not sure why.</s>

* I have to unplug my keyboard and plug it back in on each boot =/ (this was also happening before). No, this is not fixed by changing the state of XHCI Handoff in my firmware.

* <s>I have to unplug my keyboard and plug it back in on each boot =/ (this was also happening before). No, this is not fixed by changing the state of XHCI Handoff in my firmware.</s> <tt>powertop</tt> was being invoked on each boot, turning on autosuspend.

* I'm not sure this 4x140mm structure of fans in the roof is really doing me any good, and need to test with and without them. What really happened here is I saw [[Noctua]] had a chromax.black 140mm and creamed my jeans and was like "gotta order that!" Then I realized I had no 140mm mounts, and rather than do something sensible, I built up an autistic rage and was like OH FUCK YOU GOD, YOU TRY TO FUCK ME, NAH, I'LL JUST BUY THREE MORE FANS AND THREE-DEE PRINT MYSELF A BARUD-DÛR AND IT'LL INTERLOCK BECAUSE I TOO AM A MASTER OF REALITY, YOU OLD SHITTER, THEN I'LL GO BACK IN TIME AND KILL MY PARENTS BEFORE I CAN BE BORN AND UNDO ALL YOUR WORKS BITCH and then it was like, well, better use these fans I guess.

* <s>I'm not sure this 4x140mm structure of fans in the roof is really doing me any good, and need to test with and without them. What really happened here is I saw [[Noctua]] had a chromax.black 140mm and creamed my jeans and was like "gotta order that!" Then I realized I had no 140mm mounts, and rather than do something sensible, I built up an autistic rage and was like OH FUCK YOU GOD, YOU TRY TO FUCK ME, NAH, I'LL JUST BUY THREE MORE FANS AND THREE-DEE PRINT MYSELF A BARUD-DÛR AND IT'LL INTERLOCK BECAUSE I TOO AM A MASTER OF REALITY, YOU OLD SHITTER, THEN I'LL GO BACK IN TIME AND KILL MY PARENTS BEFORE I CAN BE BORN AND UNDO ALL YOUR WORKS BITCH and then it was like, well, better use these fans I guess.</s> they were indeed doing me no good, and have been replaced with a [[inaMORAta|MoRa-3]].

* Meanwhile, I don't think there's enough inflow. Need to test with the door open.

* <s>Meanwhile, I don't think there's enough inflow. Need to test with the door open.</s> remedied.

* Apparently Type-C USB Power Delivery runs at 20 and 48 volts. No idea if I'm supplying this correctly, or if it's even in play.

==Results (pictures!)==

The machine is essentially silent outside of intense computational tasks and heavy disk I/O (the Exos drives are pretty solid across the board, except they're sometimes loud). It is rock stable despite the mastodontic 3970X being overclocked slightly above 4.3GHz, and can spend arbitrarily much time in the Turbo state (I am not currently making use of [https://en.wikichip.org/wiki/amd/pbo AMD PBO]). The processor temperature never exceeds ~~80℃~~; the GPU processor stays comfortably under 40℃. Coolant temperature stays thus far within ~5℃ of ambient.

The machine is essentially silent outside of intense computational tasks and heavy disk I/O (the Exos drives are pretty solid across the board, except they're sometimes loud). It is rock stable despite the mastodontic 3970X being overclocked slightly above 4.3GHz, and can spend arbitrarily much time in the Turbo state (I am not currently making use of [https://en.wikichip.org/wiki/amd/pbo AMD PBO]). The processor temperature never exceeds 60℃; the GPU processor stays comfortably under 40℃. Coolant temperature stays thus far within ~8℃ of ambient.

I can pull about 2Gbps off my RAID60 on a good day, an insane amount of spinning disk bandwidth, especially for archival media. My primary RAID1 is fast enough that I've never noticed blocking on it.

Line 471:

Line 589:

[[File:Ohyeah.jpg|center]]

That'll do, ~~machine~~. That'll do.

That'll do, pig. That'll do.

==Future directions==

Line 483:

Line 601:

<li>LoRa. [[LoRa]] is a long-range, low-bandwidth radio protocol. I could bring an antenna out, and use the Arduino together with a LoRa chip.</li>

<li>Battery for the CCFL. It would be nice to have some light when I'm working inside the machine. If I could provide selectable battery-based backup for these rods, that would be useful.</li>

<li>PID control for fans/pumps. The Proportional-Integral-Derivative controller is a simple feedback mechanism that I suspect would work well with fans and pumps. I don't care how many RPM my fans are spinning at; what I care about is how warm my coolant and components are (and noise). I'd like to set up target ΔTs (as a function of ambient temp) and a target noise ceiling, and use an inline sensor, an ambient sensor, and an acoustic sensor in combination to manage my loop's active components.</li>

<li>PID control for fans/pumps. The Proportional-Integral-Derivative controller is a simple feedback mechanism that I suspect would work well with fans and pumps. I don't care how many RPM my fans are spinning at; what I care about is how warm my coolant and components are (and noise). I'd like to set up target ΔTs (as a function of ambient temp) and a target noise ceiling, and use an inline sensor, an ambient sensor, and an acoustic sensor in combination to manage my loop's active components. update: see my [[Counterforce]] project, which does all this and much more!</li>

</ul>

Line 492:

Line 610:

* [https://www.overclockers.com/water-cooling-flow-rate-and-heat-transfer/ Water Cooling Flow Rate and Heat Transfer] on overclockers.com

* Intel's [https://www.intel.com/content/dam/www/public/us/en/documents/guides/power-supply-design-guide-june.pdf ATX Power Supply] Design Guide

[[CATEGORY: Hardware]]

Schwarzgerät III: Difference between revisions

Dank (talk | contribs)

@@ Line 4: / Line 4: @@
 My 2020 rebuild, [[Schwarzgerät_II|Schwarzgerät II]], was a beast of a machine. Like the Hubble or LHC, however, I had to give it more powah. The 2022 upgrade, Schwarzgerät III, does just that despite scrotumtightening supply chain madness. This rebuild focused on cooling, power, and aesthetics. I require that the machine be no louder than ambient noise (I live in the middle of [[Atlanta|Midtown Atlanta]]) when weakly loaded—I ought not hear the machine unless seriously engaged in active computation. At the same time, I want to overclock my 3970X as far as she can (reasonably) go.
+'''(further [[Schwarzgerät_III_upgrade|upgraded]] 2022-04, hopefully the last bit for a minute)'''
 <blockquote>“But out here, down here among the people, the truer currencies come into being.”―<i>''Gravity's Rainbow''</i> (1973)</blockquote>
@@ Line 17: / Line 19: @@
 ==Bill of materials==
-We're approaching the $10,000 mark before correcting for inflation, with hard drives alone representing close to $5,000. Materials in this build were acquired over a period going back to 2011 (the LSI Fusion SAS card is, I'm pretty certain, the component longest in my possession). This most recent iteration represents less than $2,000 of components, most of that being $1,150 for the 256GB of RAM (I did manage to sell my old RAM for $200, but we can't deduct that, unless we included its original cost).
+We're approaching the $10,000 mark before correcting for inflation, with hard drives alone representing close to $5,000. Materials in this build were acquired over a period going back to 2011 (one of the LSI Fusion SAS cards is, I'm pretty certain, the component longest in my possession). This most recent iteration represents less than $2,000 of components, most of that being $1,150 for the 256GB of RAM (I did manage to sell my old RAM for $200, but we can't deduct that, unless we included its original cost).
+[[File:Caselabs-t10a-parts.jpg|right|400px|thumb|Parts for the Caselabs T10]]
 ===Chassis===
 [[File:Fanmount.jpg|thumb|200px|4x140mm fan struts, designed in OpenSCAD and printed on my [[Voxel]].]]
@@ Line 31: / Line 33: @@
 * Self-designed and -printed case for RHElectronics Geiger counter ([https://github.com/dankamongmen/openscad-models/blob/master/geiger.scad source])
 * Self-designed and -printed covering case for EKWB Quantum Kinetic FLT 240 mounting kit ([https://github.com/dankamongmen/openscad-models/blob/master/reservoir-part.scad source])
-* Self-designed and -printed cable shroud for bottom of Gigabyte Aorus Master TRX40
+* Self-designed and -printed PCI brackets with cable channels ([https://github.com/dankamongmen/openscad-models/blob/master/bracketpci.scad source])
-* Self-designed and -printed false floor for bottom of Caselabs Magnum T10 PSU side
 * Self-designed and -printed 4x140mm fan mount for roof ([https://github.com/dankamongmen/openscad-models/blob/master/140-interlocking.scad source])
 * DEMCiflex magnetic dust filter pack for CaseLabs Magnum TH10
@@ Line 48: / Line 49: @@
 ====Bespoke loop====
-[[File:Bottom.jpg|200px|thumb|The bottom complexes remind me of turbines.]]
+[[File:Bottom.jpg|200px|thumb|The bottom complexes suggest turbogenerators.]]
 * [https://www.ekwb.com/shop/ek-quantum-kinetic-flt-240-d5-pwm-d-rgb-plexi EKWB Quantum Kinetic FLT 240] D5 pump + reservoir with mounting brackets. Installed halfway up the case's back, outside.
 ** EK Laing PWM D5 pump installed into the Quantum Kinetic.
-** [https://shop.bitspower.com/index.php?route=product/product&product_id=1963 Bitspower BP-MBWP-CT] G1/4-10K temperature sensor. Installed in Quantum FLT's central front plug, running to motherboard's first external temp sensor.
+** [https://shop.bitspower.com/index.php?route=product/product&product_id=1963 Bitspower BP-MBWP-CT] G¼-10K temperature sensor. Installed in Quantum FLT's central front plug, running to motherboard's first external temp sensor.
-* [http://monsooncooling.com/reservoir.php Monsoon MMRS Series II] D5 pump housing + reservoir with 2x Silver Bullet biocide G1/4 plugs. Installed at the front bottom of the case, in the lowest two 5.25" bays.
+* [http://monsooncooling.com/reservoir.php Monsoon MMRS Series II] D5 pump housing + reservoir with 2x Silver Bullet biocide G¼ plugs. Installed at the front bottom of the case, in the lowest two 5.25" bays.
 ** EK Laing Vario D5 pump installed into the Monsoon.
-** [https://www.xs-pc.com/temperature-sensors/g14-plug-with-10k-sensor-black-chrome XS-PC] G1/4-10K temperature sensor. Installed in Monsoon's upper left plug, running to Corsair iCUE Commander Core XT's first external temp sensor.
+** [https://www.xs-pc.com/temperature-sensors/g14-plug-with-10k-sensor-black-chrome XS-PC] G¼-10K temperature sensor. Installed in Monsoon's upper left plug, running to Corsair iCUE Commander Core XT's first external temp sensor.
 * [https://www.amazon.com/DIYhz-displaydigital-Thermometer-Temperature-Indicator/dp/B09QG3241X DiyHZ aluminum shell] flowmeter and temperature sensor. LCD screen displays both values, and a 3-pin connector carries away flow information.
 * [https://www.ekwb.com/shop/ek-quantum-momentum-trx40-aorus-master-d-rgb-plexi EKWB Aorus Master TRX40] DRGB monoblock (nickel+plexi).
 * [https://www.ekwb.com/shop/ek-quantum-vector-rtx-re-d-rgb-nickel-plexi EKWB EK-Quantum Vector RTX RE] DRGB waterblock (nickel+plexi).
-* [http://hardwarelabs.com/nemesis/gtr/gtr-360/ Hardware Labs Black Ice Nemesis GTR360] 16 FPI 54.7mm radiator, mounted to top PSU side.
+* [http://hardwarelabs.com/nemesis/gtr/gtr-360/ Hardware Labs Black Ice Nemesis GTR360] 16 FPI 54.7mm radiator, mounted to top PSU side (393x133mm).
-* [https://hardwarelabs.com/blackice/gts/gts-360/ Hardware Labs Black Ice Nemesis GTS360] 30 FPI 29.6mm  radiator, mounted to top motherboard side.
+* [https://hardwarelabs.com/blackice/gts/gts-360/ Hardware Labs Black Ice Nemesis GTS360] 30 FPI 29.6mm  radiator, mounted to top motherboard side (397x133mm).
-* 2x [http://hardwarelabs.com/nemesis/gtsxflow/240gts-xflow/ Hardware Labs Black Ice Nemesis GTS240 XFLOW] 16 FPI 29.6mm crossflow radiators, mounted to bottoms.
+* 2x [http://hardwarelabs.com/nemesis/gtsxflow/240gts-xflow/ Hardware Labs Black Ice Nemesis GTS240 XFLOW] 16 FPI 29.6mm crossflow radiators, mounted to bottoms (292x133mm).
 * Fancasee 1-to-4 PWM splitter, power from header
-* Silverstone 8-way PWM splitter, SATA power
+* [[SilverStone_fan_hub|Silverstone 8-way]] PWM splitter, SATA power
 * 12.5mm ID EKWB [https://www.ekwb.com/shop/ek-tube-zmt-matte-black-19-4-12-5mm ZMT], black
-Most calculations will require knowing the total amount of coolant in our system. Fluid volumes of liquid cooling components are irritatingly difficult to establish (once the hardware is installed, anyway). With an inner diameter of 12.5mm, the ZMT has a cross-section of 122.72mm² (0.015m²). I got the EKWB fluid volumes directly from their support channel. G¼ has a diameter of 13.157mm, but that doesn't tell you anything about the inner diameter, which is not standardized =\.
+Most calculations will require knowing the total amount of coolant in our system. Fluid volumes of liquid cooling components are irritatingly difficult to establish (once the hardware is installed, anyway). With an inner diameter of 12.5mm, the ZMT has a cross-section of 122.72mm² (1.2272cm²). I got the EKWB fluid volumes directly from their support channel. G¼ has a diameter of 13.157mm, but that doesn't tell you anything about the inner diameter, which is not standardized =\. I used a graduated cylinder to measure the Monsoon and the radiators. Attachments can be considered 1--2mL each (a drain structure including a ball valve attachment, a T attachment, a 90° adapter and several extenders contained a total of 10mL).
 {| class="wikitable"
 ! Component !! Volume (mL) !! Climb (m) !! θ (rad)
 |-
-| HWL XFLOW 240 || ? || 0 || 0
+| HWL XFLOW 240 || 90 || 0 || 0
 |-
 | Tubing || ? || 0 || 0
 |-
-| HWL GTR360 || ? || 0 || 0
+| HWL GTR360 || 320 || 0 || 0
 |-
 | Tubing || ? || 0 || 0
 |-
-| HWL GTS360 || ? || 0 || 0
+| HWL GTS360 || 130 || 0 || 0
 |-
 | Tubing || 44 || ||
 |-
-| Aorus Master Monoblock || 45 || || π/2
+| Aorus Master monoblock || 45 || || π/2
 |-
 | Tubing || 11 || ||
 |-
 | EK-Vector waterblock || 50 || || π/2
@@ Line 94: / Line 95: @@
 | Tubing || 28 || ||
 |-
-| Flowmeter || ? || || π/4
+| DiyHZ flowmeter || 4 || || π/4
 |-
-| HWL XFLOW 240 || ? || 0 || 0
+| HWL XFLOW 240 || 90 || 0 || 0
 |-
 | Tubing || 11 || 0 || 0
 |-
-| Monsoon Series Two D5 || ? || 0 || 0
+| Monsoon Series Two D5 || 300 || 0 || 0
 |-
 | Tubing || ? || 0 || 0
@@ Line 108: / Line 109: @@
 Note that water expands (becomes less dense) as it heats (down to 4℃, where it begins to do the opposite), meaning more pressure on its container. The coefficient of volume expansion β(1/℃) for water is 0.00021 at 20℃. This would suggest that a 96℃ rise would result in about a 2% expansion in the total volume, but this is incorrect, due to a changing β over temperature. By the time you get to 60℃ (most tubing isn't rated beyond this temperature), you've got a β closer to 0.0005. Here's the data for pure water at one atmosphere:
-[[File:WaterB.png|400px|right|Thermal expansion of water as a function of temperature]]
 {|class="wikitable"
 ! ℃ !! Density kg/m³ !! Specific weight kN/m³ !! Co. of thermal expansion β 10⁻⁶/K
@@ Line 142: / Line 142: @@
 * 2x EK Vardar PWM on GTR360
 * Noctua NF-P12 PWM on GTR360
+{| class="wikitable"
+! Fan !! Location !! Watts !! db(A) !! CFM !! Pressure
+|-
+| NF-F12 (2) || GTS360 || 0.6 (1.2) || 22.4 (25.4) || 93.4 (186.8) || 2.61 (5.22)
+|-
+| NF-F12 iPPC-2000 || GTS360 || 1.2 || 29.7 || 121.8 || 3.94
+|-
+| NF-A12x25 (2) || GTS240 || 1.68 (3.36) || 22.6 (25.6) || 102.1 (204.2) || 2.34 (4.68)
+|-
+| NF-P12 redux-1700 (2) || GTS240 || 1.08 (2.16) || 25.1 (28.1) || 120.2 (240.4) || 2.83 (5.66)
+|-
+| EK Vardar (2) || GTR360 || 2.16 (4.32) || 33.5 (36.5) || 77 (154) || 3.16 (6.32)
+|-
+| NF-P12 || GTR360 || 0.6 || 19.8 || 120.2 || 2.83
+|-
+|}
 ====Blowing in====
@@ Line 149: / Line 166: @@
 * 2x Noctua NF-S12A PWM on lower two drive cages
 * Noctua NF-S12A FLX on top drive cage
+All values are maxima:
+{| class="wikitable"
+! Fan !! Location !! Watts !! db(A) !! CFM !! Pressure
+|-
+| NF-S12A (3) || Drive cages || 1.44 (4.32) || 17.8 (22.6) || 107.5 (322.5) || 1.19 (3.57)
+|-
+| NF-F12 iPPC-2000 || Flex-Bay || 1.2 || 29.7 || 121.8 || 3.94
+|-
+| NF-F12 iPPC-3000 || Bottom cage || 3.6 || 43.5 || 186.7 || 7.63
+|-
+| NF-A8 || 3.5" bay || 0.96 || 17.7 || 55.5 || 2.37
+|-
+|}
 ===Compute===
@@ Line 159: / Line 190: @@
 * [https://www.evga.com/products/specs/gpu.aspx?pn=1207a8ec-269d-4e11-91a9-e01226652c9f EVGA GeForce RTX 2070 SUPER] Black Gaming 6GB GDDR6 with NVIDIA TU104 GPU. Installed in topmost PCIe 4.0 16x slot, though this is only a 3.0 card.
 * [https://www.elegoo.com/products/elegoo-mega-2560-r3-board ELEGOO MEGA 2560] Revision 3, connected to NXZT internal USB hub, mounted to back of PSU chamber
-* [https://ark.intel.com/content/www/us/en/ark/products/60021/intel-ethernet-controller-x540bt2.html Intel X540-BT2 2-port 10Gbps] 10GBASE-T PCI 3.0 card
-As I detailed regarding [[Schwarzgerat II]], the 3990X is an amazing achievement in chip design and fabrication, but I believe it to be severely starved for many tasks by its memory bandwidth; with its four memory channels populated, the ThreadRipper 3990X can hit about 90GB/s from fast DDR4; its Epyc brother can pull down ~190 through its eight channels. For my tasks, it's rare enough that I can drive all my 32 cores; with the 3990X, I'd be paying twice as much to hit full utilization less often, and be unable to bring full bandwidth to bear when I did.
+As I detailed regarding [[Schwarzgerat II]], the 3990X is an amazing achievement in chip design and fabrication, but I believe it to be severely starved for many tasks by its memory bandwidth; with its four memory channels populated, the ThreadRipper 3990X can hit about 90GB/s from fast DDR4; its EPYC brother can pull down ~190 through its eight channels. For my tasks, it's rare enough that I can drive all my 32 cores; with the 3990X, I'd be paying twice as much to hit full utilization less often, and be unable to bring full bandwidth to bear when I did.
-I absolutely 🖤 my 3970X, though. Bitch screams. Anyone overclocking on Linux should be aware of [https://www.linux.org/docs/man8/turbostat.html turbostat].
+I absolutely 🖤 my 3970X, though. Bitch screams. Anyone overclocking on Linux should be aware of [https://www.linux.org/docs/man8/turbostat.html turbostat]. The 3970X supports 88 lanes of PCIe 4.0, of which the TRX40 chipset consumes 24, leaving 64 for expansion devices. I've got 16 (GPU) + 16 (Hyper X) + 2x8 (LSI cards) + 12 (M.2 onboards) for 60 total, coming in just under saturation.
 ===Power===
@@ Line 175: / Line 205: @@
 * 2x 4-way SATA expanders
 * 2x [https://www.amazon.com/BANKEE-Converter-Voltage-Regulator-Transformer/dp/B08NZSYZRF Bankee 12V->5V/15A] buck transformers
+To make a lengthy story short (it's told in much more detail below), I needed more SATA power cables than my PSU supported. The buck transformers are used to convert the 12V on PCIe cables to 5V+12V as needed by SATA drives.
 ===Storage===
 [[File:Hacksawed.jpg|thumb|200px|Cut down that drive cage!]]
-I love the [http://www.cabledeconn.com/Mini-SAS-Series-/SATA-Esata-list-138-160.html CableDeconn bunched SATA] data cables; they're definitely the only way to fly, assuming lack of SATA backplanes. We end up with 4x 3.5 drives in the bay, 10x 3.5 drives in CaseLabs cages in the PSU side, 3x M.2 devices in the motherboard PCIe 4.0 slots, and 2x M.2 devices in the HyperX card. This leaves room for 2 more M.2s in the card, and 4x 2.5 devices in the smaller bay. The bottom 2 slots in the bottom hard drive cage are blocked by the PSU-side radiator; indeed, I had to take a hacksaw to said cage to get it into the machine.
-See my analysis of [[13_drives|how to best make use of 14 drives]]. I went with a striped raidz2 for my 14 Exos drives (also known as a RAID60 in the Old English), yielding 180TB usable from 252TB total. I suffer data loss if I lose any combination of 4 drives, and can lose data if I lose certain combinations of 3 drives (any combination where all three lost drives are in the same raid2z), but no rebuild ever involves more than seven drives.
-All filesystems are ZFS, and all storage enjoys some redundancy (save the 16GB Optane, which is just for persistent memory/DAX experiments).
 * 14x Seagate Exos X18 18TB 7200 rpm SATA III drives in striped raidz2
 * Asus HyperX 4x M.2 PCIe 3.0 x16
@@ Line 192: / Line 218: @@
 * 4x CableDeconn SAS-to-4xSATA cables for use with LSI Fusions
 * 2x CableDeconn 4x SATA cables for use with motherboard
+I love the [http://www.cabledeconn.com/Mini-SAS-Series-/SATA-Esata-list-138-160.html CableDeconn bunched SATA] data cables; they're definitely the only way to fly (assuming lack of SATA backplanes, which good luck in a workstation form factor). We end up with 4x 3.5 drives in the bay, 10x 3.5 drives in CaseLabs cages in the PSU side, 3x M.2 devices in the motherboard PCIe 4.0 slots, and 2x M.2 devices in the HyperX card. This leaves room for 2 more M.2s in the card, and 4x 2.5 devices in the smaller bay. The bottom 2 slots in the bottom hard drive cage are blocked by the PSU-side radiator; indeed, I had to take a hacksaw to said cage to get it into the machine.
+How to arrange 14 disks so as to create a single volume? I don't bother with solutions involving hot spares:
+{| class="wikitable"
+! Name !! Setup !! Availability !! MaxTol !! MinFail !! Max Rebuild
+|-
+| raid3z
+| 11 data, 3 parity
+| 78.6% (11)
+| 3
+| 4
+| 100% + fail
+|-
+| raid2z
+| 12 data, 2 parity
+| 85.7% (12)
+| 2
+| 3
+| 85.7% + fail
+|-
+| striped raid3z
+| 2x(4 data, 3 parity)
+| 57.1% (8)
+| 3 + 3
+| 4
+| 50% + fail
+|-
+| striped raid2z
+| 2x(5 data, 2 parity)
+| 71.4% (10)
+| 2 + 2
+| 3
+| 50% + fail
+|-
+| striped raidz
+| 2x(6 data, 1 parity)
+| 85.7% (12)
+| 1 + 1
+| 2
+| 50% + fail
+|-
+| mirrored raid3z
+| 2x(4 data, 3 parity)
+| 28.6% (4)
+| 7 + 3
+| 8
+| 50% + fail
+|-
+| mirrored raid2z
+| 2x(5 data, 2 parity)
+| 35.7% (5)
+| 7 + 2
+| 6
+| 50% + fail
+|-
+| mirrored raidz
+| 2x(6 data, 1 parity)
+| 42.9% (6)
+| 7 + 1
+| 4
+| 50% + fail
+|-
+| raidz3 of stripes
+| 7x(2) 4 data, 3 parity
+| 57.1% (8)
+| 2 + 2 + 2
+| 4
+| 100% + fail
+|-
+| raidz2 of stripes
+| 7x(2) 5 data, 2 parity
+| 71.4% (10)
+| 2 + 2
+| 3
+| 85.7% + fail
+|}
+I went with a striped raidz2 for my 14 Exos drives (also known as a RAID60 in the Old English), yielding 180TB usable from 252TB total (71.4%). I suffer data loss if I lose any combination of 5 drives, and can lose data if I lose certain combinations of 3 drives (any combination where all three lost drives are in the same raid2z), but no rebuild ever involves more than seven drives. All filesystems are ZFS, and all storage enjoys some redundancy (save the 16GB Optane, which is just for persistent memory/DAX experiments). I could have split the volume and had two raidz2s with no interdependence, but eh.
 ===Interfaces===
@@ Line 197: / Line 303: @@
 * NXZT internal USB 2.0 hub, magnetically attached to underside of PSU, connected to motherboard USB 2.0 header
 * [https://rhelectronics.net/store/radiation-detector-geiger-counter-diy-kit-second-edition.html RHElectronics] Geiger counter, wired via 3 pins to 2560 MEGA, mounted to back of PSU chamber
-* [https://www.corsair.com/us/en/Categories/Products/Accessories-%7C-Parts/CORSAIR-iCUE-COMMANDER-CORE-XT-Smart-RGB-Lighting-and-Fan-Speed-Controller Corsair ICUE Commander Core XT] RGB/fan controller, mounted in top, connected to NXZT internal USB hub. I extended OpenRGB and the Linux kernel to drive this device.
+* [https://www.corsair.com/us/en/Categories/Products/Accessories-%7C-Parts/CORSAIR-iCUE-COMMANDER-CORE-XT-Smart-RGB-Lighting-and-Fan-Speed-Controller Corsair ICUE Commander Core XT] RGB/fan controller, mounted in top, connected to NXZT internal USB hub
 * Monsoon CCFL 12V inverter, mounted to top, powered via SATA power connector attached to 12V Molex attached to video power line
 * GY-521 board for [http://invensense.com/mems/gyro/documents/PS-MPU-6000A-00v3.4.pdf MPU 6050] accelerometer + gyro, wired via 8 pins to 2560 MEGA, mounted to back of PSU chamber
@@ Line 204: / Line 310: @@
 ===Lighting===
-* 4x Corsair ARGB LED lines, connected in series to Corsair Commander Core, attached via adhesive around top.
+* 4x Corsair 410mm ARGB LED lines, connected in series to Corsair Commander Core, attached via adhesive around bottom cooling complex of the motherboard side.
 * 2x 12V RGB LED lines, backlighting top radiators, attached to motherboard's top RGB header via 1-to-2 RGB splitter.
-* 2x green PerformancePCs CCFL rods, attached to Monsoon inverter, mounted to back inner corner of each chamber.
+* 5x green PerformancePCs CCFL rods, attached to Monsoon DC->AC inverter, mounted to back inner corner of each chamber.
 * ARGB lines on EVGA Quantum FLT and Aorus Master monoblock, attached to motherboard's top and bottom ARGB headers respectively.
 * RGB tops on Rage DIMMs are unmanaged, and self-synchronize via infrared.
@@ Line 212: / Line 318: @@
 * Piratedog [https://piratedog.tech/collections/rgb-splitters/products/corsair-rgb-lighting-channel-splitter Corsair RGB] 1-to-2 splitter
 * EZDIY-Fab [https://www.amazon.com/gp/product/B09HGP9YMK/ref=ppx_yo_dt_b_asin_title_o03_s00?ie=UTF8&psc=1 ARGB LED] strips
+I currently have no window on my PSU side, so there's no point putting lighting in there. I do have a short strip of 12V LEDs just in case I need to work in there without light (though presumably with the machine on). I'd like to get these on a switch or something. Ideally both these and the CCFLs would be on a switch with a battery backup. I'm sad that I bought into the Corsair ecosystem. It's almost entirely closed and proprietary, even the connectors. Props to [https://piratedog.tech/ PirateDog Tech] for working around this.
 ==Distributing power==
-[[File:Tobsun.jpg|200px|thumb|A 12V->5V buck transformer. By upping the output amperage, it is more efficient than a voltage regulator.]]
+[[File:Tobsun.jpg|200px|thumb|A 12V->5V buck transformer, more efficient than a mere voltage regulator.]]
 I began to run into some serious power issues on this build, originating in the Exos X18 drives (of which, you might remember, there are 14). It will be worth your time to consult the [https://www.seagate.com/www-content/product-content/enterprise-hdd-fam/exos-x18/_shared/en-us/docs/100865854a.pdf Exos 18 manual]. Remember, 12V is for the motor, and 5V is for the logic.
@@ Line 237: / Line 345: @@
 [schwarzgerat](0) $ sudo pwrstat -status
 Model Name................... CP1500PFCLCD
-Firmware Number.............. 000000000000
 Rating Voltage............... 120 V
 Rating Power................. 900 Watt
@@ Line 248: / Line 355: @@
 Load......................... 333 Watt(37 %)
 Line Interaction............. None
-Test Result.................. Unknown
 Last Power Event............. Blackout at 2020/03/08 19:16:10 for 24 sec.
 [schwarzgerat](0) $
@@ Line 261: / Line 367: @@
 <li>Molex (12V only): 2 (pumps)</li>
 <li>Molex (5V only): 2 (flowmeter, USB hub)</li>
-<li>SATA (12V+5V): 14 (10 disks, 2 for StarTech bay, 1 for IcyDock bay, Corsair)</li>
+<li>SATA (12V+5V): 13 (10 disks, 2 for StarTech bay, 1 for IcyDock bay)</li>
 <li>SATA (12V only): 2 (inverter, fan controller)</li>
-<li>SATA (5V only): 2 (USB front bays)</li>
+<li>SATA (5V only): 3 (USB front bays, Corsair)</li>
 <li>PCIe: 1 (GPU)</li>
 <li>3-pin ARGB: 4 (ARGB headers, 5A each)</li>
@@ Line 270: / Line 376: @@
 </ul>
-I got these numbers from product documentation or, in the case of the EKWB LEDs, from the manufacturer's support channel.
+I got these numbers from product documentation or, in the case of the EKWB LEDs, from the manufacturer's support channel. Corsair LED strip information came from [https://forum.corsair.com/forums/topic/155937-corsair-rgb-current-usage/#comment-917706 PirateDog Tech]. The Corsair iCUE Commander XT is rated for a maximum of 4.5A total.
 {| class="wikitable"
 ! Item !! 5V watts !! 12V watts !! Source
@@ Line 311: / Line 417: @@
 |-
 | Monsoon inverter || 0 || ? || SATA
+|-
+| Monsoon LEDs || 0 || ? || Molex
 |-
 | Exos18 (x14) (spinning) || 4.6 (64.4) || 7.68 (107.52)|| SATA
@@ Line 319: / Line 427: @@
 |-
 | EKWB Aorus TRX40 Monoblock LEDs || 1.05 || 0 || ARGB header
+|-
+| DiyHZ light strips || ? || 0 || ARGB header
 |-
 | Corsair Commander Pro XT (logic) || ? || 0 || SATA
 |-
-| Corsair LED strips (x4) || ? (?) || 0 || Corsair (SATA)
+| Corsair LED strips (x4) || 1.835 (7.34) || 0 || Corsair (SATA)
 |-
 | DiyHZ flowmeter || ? || 0 || Molex
@@ Line 337: / Line 447: @@
 |-
 |}
+The Silverstone supports draws up to 4.5A (54W). The Corsair supports a 12V fan draw of 4.5A (54W) and a 5V RGB draw of 4.5A (22.5W).
+The PSU claims support of up to 100W as 5V (20A), and up to 850W as 12V (71A). The 5V line is dominated by hard drives, which will consume 65W when in active use (and 70W when spinning up). Beyond them, there are very few 5V draws; the ARGB light strips are going through the motherboard's ARGB headers, the internal USB hub and downstream devices are getting 5V from Molex, and the Corsair is pulling over SATA. Even if the motherboard ARGB headers are fed by the ATX12V's 5V lines, we're talking 20W max, and probably less than that.
+We're all clear on 12V until overclocking comes into play. The GPU and CPU will never be maxed out while the drives are spinning up (provided that we don't spin down the motors after startup), so the drives will be consuming 107W instead of 339W. The stock CPU+GPU are 495W, 46W for pumps, and some unknown but substantial amount is consumed by the motherboard; let's estimate a very generous 100W. That's 748W total. With 750W left after 100W are devoted to 5W, we're cutting it close, but that ought be a rare load. In practice, the machine tends to take about 300W in everyday use, up to about 600W under computational load; to really push the envelope, I'd need force heavy computational <i>and</i> I/O load. I'll see what happens next time I build a kernel while my ZFS is rebuilding.
 ===The drive problem===
@@ Line 372: / Line 488: @@
 We have six remaining items requiring 5V power: the flowmeter, the Corsair, the internal USB hub, and the three front panel USB bays. Of these, three natively want Molex 4-pin, and the other 3 want SATA. Combined with our 14 drives, that's 20 power drains.
-I'm initially solving this problem using PUIS (Power-Up In Standby), a feature of the SATA specification. Enabling this feature on a disk will prevent it from spinning up until it receives a particular command, which can be issued by the OS (so long as you don't need your system firmware to recognize the disk, which will be unreadable until this command is sent). I intended to solve this later with 12V relays (or better yet, an [https://www.crowdsupply.com/sequent-microsystems/8-mosfet 8-MOSFET]) controlled by the Arduino, but instead I think I'm going to use a 12V->5V buck transformer, splice the 12V line of my remaining PCIe cables (see below), and make myself an underpowered 4-pin Molex from it. Each PCIe cable and its 66W of 12V ought be good for three drives, if I'm right about the whole feasibility of that plan.
+I initially solved this problem using PUIS (Power-Up In Standby), a feature of the SATA specification. Enabling this feature on a disk will prevent it from spinning up until it receives a particular command, which can be issued by the OS (so long as you don't need your system firmware to recognize the disk, which will be unreadable until this command is sent). This had the downside of requiring me to keep my disks spinning once the machine had started, since I'd otherwise run into the problem anew should they all spin back up at the same time.
+I at first intended to solve this with 12V relays (or better yet, an [https://www.crowdsupply.com/sequent-microsystems/8-mosfet 8-MOSFET]) controlled by the Arduino, but instead used two 12V->5V buck transformers, splice the 12V line of my remaining PCIe cables (see below), and made myself underpowered 4-pin Molex cables from them. Each PCIe cable and its 66W of 12V is good for three drives when taken through three Molex-to-SATA adapters.
 ===Power sources===
@@ Line 384: / Line 502: @@
 | Perif (Molex) || 0 || 55 || 132 || Can bridge to SATA or PCIe. Wiring might not be safe for the full pin capacity.
 |-
-| PCIe x4|| 9 || 0 || 66 || Can bridge to (underpowered) 12V-only (2-pin) Molex. Can probably bridge (with buck transformer) to underpowered Molex.
+| PCIe x4|| 0 || 0 || 75 || Can bridge to (underpowered) 12V-only (2-pin) Molex. Can probably bridge (with buck transformer) to underpowered Molex.
 |-
 |}
@@ Line 414: / Line 532: @@
 ==Regrets and stupidities==
 [[File:Stupidgpu.jpg|200px|thumb|There are at least two reasons why this GPU's waterblock configuration is suboptimal.]]
-* I ought have wrapped all the tubing in anti-kink coils, which I already had, way earlier.
+* I ought have wrapped all the tubing in anti-kink coils (which I already had!) way earlier.
 * I ought have bought a second LSI SAS card from the outset, rather than going through two POS chinesium 1x cards, both of which continuously reset attached disks (I'm guessing due to my fairly lengthy cables? who knows).
 * I ought have paid more attention to which fans I was putting on which radiators.
@@ Line 442: / Line 560: @@
 <li>Replace the 45-degree adapter connecting the flowmeter to the radiator with a 90-degree adapter. This will lower the flowmeter, putting its output near its original location, eliminating any tubing length gains.</li>
 </ol>
-The reservoir will be raised no more than the height of the fan being added to the radiator; it will likely move from the bottom bay to the bay immediately above that. Essentially, we're raising the entire floor of our cooling system by one fan's height. I'll be using Phanteks T30-120s, so that's 30mm. We could theoretically move everything higher with a shroud around these fans, but it would make working on the machine a goddamn nightmare.
+The reservoir will be raised no more than the height of the fan being added to the radiator; it will likely move from the bottom bay to the bay immediately above that. Essentially, we're raising the entire floor of our cooling system by one fan's height. I'll be using [[Phanteks|Phanteks T30-120s]], so that's 30mm. We could theoretically move everything higher with a shroud around these fans, but it would make working on the machine a goddamn nightmare.
 When I do this, I'll likely raise the Vario pump's setting from 2 to 3. I might even replace the Vario pump with a proper PWM one. It would also be a fine time to [https://kampidh.blogspot.com/2019/11/down-rabbit-hole-d5-pwm-pump-flaw-and.html fix my other D5] so it reports tach properly.
 ==Ongoing issues==
-* My flow rate is lower than I would like, assuming this flowmeter to be reliable. Maxing out the PWM pump hits desirable flow rates, but with more noise than I want. I think there might be some flow reduction in the tubing between the flowmeter and Quantum Kinetic; I'm considering replacing that with hardline. For all I know, there's still a big air pocket in some radiator.
+* <s>My flow rate is lower than I would like, assuming this flowmeter to be reliable. Maxing out the PWM pump hits desirable flow rates, but with more noise than I want. I think there might be some flow reduction in the tubing between the flowmeter and Quantum Kinetic; I'm considering replacing that with hardline. For all I know, there's still a big air pocket in some radiator.</s> <b>there was indeed a big air pocket</b>
 * <s>My onboard Ethernet stopped working, and I'm still not sure why.</s> <b>the ethernet cable wasn't plugged into the outlet all the way, lol, you dumb idiot</b>
 * <s>I'm getting a kernel oops in the igb driver on startup, not sure why.</s>
-* I have to unplug my keyboard and plug it back in on each boot =/ (this was also happening before). No, this is not fixed by changing the state of XHCI Handoff in my firmware.
+* <s>I have to unplug my keyboard and plug it back in on each boot =/ (this was also happening before). No, this is not fixed by changing the state of XHCI Handoff in my firmware.</s> <tt>powertop</tt> was being invoked on each boot, turning on autosuspend.
-* I'm not sure this 4x140mm structure of fans in the roof is really doing me any good, and need to test with and without them. What really happened here is I saw [[Noctua]] had a chromax.black 140mm and creamed my jeans and was like "gotta order that!" Then I realized I had no 140mm mounts, and rather than do something sensible, I built up an autistic rage and was like OH FUCK YOU GOD, YOU TRY TO FUCK ME, NAH, I'LL JUST BUY THREE MORE FANS AND THREE-DEE PRINT MYSELF A BARUD-DÛR AND IT'LL INTERLOCK BECAUSE I TOO AM A MASTER OF REALITY, YOU OLD SHITTER, THEN I'LL GO BACK IN TIME AND KILL MY PARENTS BEFORE I CAN BE BORN AND UNDO ALL YOUR WORKS BITCH and then it was like, well, better use these fans I guess.
+* <s>I'm not sure this 4x140mm structure of fans in the roof is really doing me any good, and need to test with and without them. What really happened here is I saw [[Noctua]] had a chromax.black 140mm and creamed my jeans and was like "gotta order that!" Then I realized I had no 140mm mounts, and rather than do something sensible, I built up an autistic rage and was like OH FUCK YOU GOD, YOU TRY TO FUCK ME, NAH, I'LL JUST BUY THREE MORE FANS AND THREE-DEE PRINT MYSELF A BARUD-DÛR AND IT'LL INTERLOCK BECAUSE I TOO AM A MASTER OF REALITY, YOU OLD SHITTER, THEN I'LL GO BACK IN TIME AND KILL MY PARENTS BEFORE I CAN BE BORN AND UNDO ALL YOUR WORKS BITCH and then it was like, well, better use these fans I guess.</s> they were indeed doing me no good, and have been replaced with a [[inaMORAta|MoRa-3]].
-* Meanwhile, I don't think there's enough inflow. Need to test with the door open.
+* <s>Meanwhile, I don't think there's enough inflow. Need to test with the door open.</s> remedied.
 * Apparently Type-C USB Power Delivery runs at 20 and 48 volts. No idea if I'm supplying this correctly, or if it's even in play.
 ==Results (pictures!)==
-The machine is essentially silent outside of intense computational tasks and heavy disk I/O (the Exos drives are pretty solid across the board, except they're sometimes <i>loud</i>). It is rock stable despite the mastodontic 3970X being overclocked slightly above 4.3GHz, and can spend arbitrarily much time in the Turbo state (I am not currently making use of [https://en.wikichip.org/wiki/amd/pbo AMD PBO]). The processor temperature never exceeds 80℃; the GPU processor stays comfortably under 40℃. Coolant temperature stays thus far within ~5℃ of ambient.
+The machine is essentially silent outside of intense computational tasks and heavy disk I/O (the Exos drives are pretty solid across the board, except they're sometimes <i>loud</i>). It is rock stable despite the mastodontic 3970X being overclocked slightly above 4.3GHz, and can spend arbitrarily much time in the Turbo state (I am not currently making use of [https://en.wikichip.org/wiki/amd/pbo AMD PBO]). The processor temperature never exceeds 60℃; the GPU processor stays comfortably under 40℃. Coolant temperature stays thus far within ~8℃ of ambient.
 I can pull about 2Gbps off my RAID60 on a good day, an insane amount of spinning disk bandwidth, especially for archival media. My primary RAID1 is fast enough that I've never noticed blocking on it.
@@ Line 471: / Line 589: @@
 [[File:Ohyeah.jpg|center]]
-That'll do, machine. That'll do.
+That'll do, pig. That'll do.
 ==Future directions==
@@ Line 483: / Line 601: @@
 <li><b>LoRa.</b> [[LoRa]] is a long-range, low-bandwidth radio protocol. I could bring an antenna out, and use the Arduino together with a LoRa chip.</li>
 <li><b>Battery for the CCFL.</b> It would be nice to have some light when I'm working inside the machine. If I could provide selectable battery-based backup for these rods, that would be useful.</li>
-<li><b>PID control for fans/pumps.</b> The Proportional-Integral-Derivative controller is a simple feedback mechanism that I suspect would work well with fans and pumps. I don't care how many RPM my fans are spinning at; what I care about is how warm my coolant and components are (and noise). I'd like to set up target ΔTs (as a function of ambient temp) and a target noise ceiling, and use an inline sensor, an ambient sensor, and an acoustic sensor in combination to manage my loop's active components.</li>
+<li><b>PID control for fans/pumps.</b> The Proportional-Integral-Derivative controller is a simple feedback mechanism that I suspect would work well with fans and pumps. I don't care how many RPM my fans are spinning at; what I care about is how warm my coolant and components are (and noise). I'd like to set up target ΔTs (as a function of ambient temp) and a target noise ceiling, and use an inline sensor, an ambient sensor, and an acoustic sensor in combination to manage my loop's active components. <b>update: see my [[Counterforce]] project, which does all this and much more!</b></li>
 </ul>
@@ Line 492: / Line 610: @@
 * [https://www.overclockers.com/water-cooling-flow-rate-and-heat-transfer/ Water Cooling Flow Rate and Heat Transfer] on overclockers.com
 * Intel's [https://www.intel.com/content/dam/www/public/us/en/documents/guides/power-supply-design-guide-june.pdf ATX Power Supply] Design Guide
+[[CATEGORY: Hardware]]