Reflections on watercooling: Difference between revisions

No edit summary
 
(8 intermediate revisions by the same user not shown)
Line 1: Line 1:
I've been watercooling my primary personal machine since [[Schwarzgerät_II|2020]]. It started off simple—a waterblock on my Threadripper 3970X, an EKWB reservoir+pump combo, and a Hardware Labs GTR360 radiator. A total of about $480 (not including fans), less than my motherboard cost. It is no longer so simple: the same machine in its [[Schwarzgerät_III_upgrade|current incarnation]] boasts a monoblock, a GPU block, a custom GPU backplane, four radiators, two reservoirs, four D5 pumps, two temperature sensors, two discrete inline sensors, and about twenty attachments of all kinds. There's <i>at least</i> $1000 worth of watercooling crap in there, and  another $500 of fans. I guess you could say things are getting serious.
'''[[Dankblog|dankblog!]] 2022-04-17, 0836 EDT, at [[Viewpoint|the danktower]]'''
 
I've been watercooling my primary personal machine since [[Schwarzgerät_II|2020]]. It started off simple—a waterblock on my Threadripper 3970X, an EKWB reservoir+pump combo, and a Hardware Labs GTR360 radiator. A total of about $480 (not including fans), less than my motherboard cost. It is no longer so simple: the same machine in its [[Schwarzgerät_III_upgrade|current incarnation]] boasts a monoblock, a GPU block, a custom GPU backplane, four radiators, two reservoirs, four D5 pumps, two temperature sensors, two discrete inline sensors, and about twenty attachments of all kinds (note that this is an almost laughably overpowered setup). There's <i>at least</i> $1000 worth of watercooling crap in there, and  another $500 of fans. I guess you could say things are getting serious.


I've put together some thoughts having watercooled for over two years.
I've put together some thoughts having watercooled for over two years.


==Generic cooling notes==
==Generic cooling notes==
It might be useful to read [[The_Power,_pt_1#Let's_do_the_do|section six]] from [[The_Power,_pt_1|this essay]] of mine, but you can probably get by without it. It is very useful to have a working knowledge of the [https://en.wikipedia.org/wiki/Laws_of_thermodynamics Laws of Thermodynamics] and [https://en.wikipedia.org/wiki/Thermal_conduction Fourier's Law].
It might be useful to read [[The_Power,_pt_1#Let's_do_the_do|section six]] from [[The_Power,_pt_1|this essay]] of mine, but you can probably get by without it. It is very useful to have a working knowledge of the [https://en.wikipedia.org/wiki/Laws_of_thermodynamics Laws of Thermodynamics] and [https://en.wikipedia.org/wiki/Thermal_conduction Fourier's Law] ([https://en.wikipedia.org/wiki/Pascal%27s_law Pascal's Law] and some basic hydrostatics wouldn't be bad, either).


* We don't use Fahrenheit, but it's worth knowing that each ℃ is 1.8℉, with an offset of 32. We don't really use Kelvins either, but Kelvins are equivalent to Celsius with an offset of -273.15. Rankines can fuck right off. Liquids boil at lower temperatures at higher altitudes (boiling occurs when internal vapor pressure equals atmospheric pressure). Water expands both above and below 4℃. A quick table:
* We don't use Fahrenheit, but it's worth knowing that each ℃ is 1.8℉, with an offset of 32. We don't really use Kelvins either, but Kelvins are equivalent to Celsius with an offset of -273.15. Rankines can fuck right off. Liquids boil at lower temperatures at higher altitudes (boiling occurs when internal vapor pressure equals atmospheric pressure). Water expands both above and below 4℃. A quick table:
<center>
{| class="wikitable"
{| class="wikitable"
! ℉ !! ℃ !! Notes
! ℉ !! ℃ !! Notes
Line 25: Line 28:
|-
|-
|}
|}
</center>
* Neither air- nor water-based cooling can get your temperatures below ambient (this can be achieved with phase cooling, thermoelectric aka Peltier, and cyrogenics). If it's 30 degrees in your room, your coolant and component temps are going to be at least 30 degrees. <b>It is useless to report temperatures without reporting ambient temps.</b> The effectiveness of your cooling system is measured by how close it gets to ambient, ΔT. Likewise, your heat-generating components are going to be hotter than your coolant.
* Neither air- nor water-based cooling can get your temperatures below ambient (this can be achieved with phase cooling, thermoelectric aka Peltier, and cyrogenics). If it's 30 degrees in your room, your coolant and component temps are going to be at least 30 degrees. <b>It is useless to report temperatures without reporting ambient temps.</b> The effectiveness of your cooling system is measured by how close it gets to ambient, ΔT. Likewise, your heat-generating components are going to be hotter than your coolant.
* <b>The goal is to move heat from the heat-generating components, and ultimately from the case.</b> If you can't remove as much heat as you generate, temperatures will rise with time. If you can remove all the heat you generate, temperatures will fall towards ambient. Effectively moving heat requires (a) thermal conductivity, (b) contact area and (c) a temperature gradient. The thermal conductivity of air is about two orders of magnitude less than water, which is about two orders of magnitude less than aluminum or copper.
* <b>The goal is to move heat from the heat-generating components, and ultimately from the case.</b> If you can't remove as much heat as you generate, temperatures will rise with time. If you can remove all the heat you generate, temperatures will fall towards ambient. Effectively moving heat requires (a) thermal conductivity, (b) contact area and (c) a temperature gradient. The thermal conductivity of air is about two orders of magnitude less than water, which is about two orders of magnitude less than aluminum or copper.
Line 39: Line 43:
TIM is thermally conductive viscous material applied between heat contacts. There's some between your dies and the IHS, and then some between the IHS and your waterblock/heatsink. Its purpose is to smooth out the conducting surface, which is dotted with microscopic pits that would otherwise preclude contact. TIM is one or two orders of magnitude less thermally conductive than metals making direct contact, but much more conductive than the air which would otherwise fill these regions. Thermal conductance is measured in W/Km (watts per kelvin-meter).
TIM is thermally conductive viscous material applied between heat contacts. There's some between your dies and the IHS, and then some between the IHS and your waterblock/heatsink. Its purpose is to smooth out the conducting surface, which is dotted with microscopic pits that would otherwise preclude contact. TIM is one or two orders of magnitude less thermally conductive than metals making direct contact, but much more conductive than the air which would otherwise fill these regions. Thermal conductance is measured in W/Km (watts per kelvin-meter).
* I don't delid my processors, but it's an intriguing idea. Delidding is the removal of the processor's integrated heat spreader. The IHS is the metal cover atop your processor. The actual heat-generating die elements tend to be only a small portion of this total area (the remainder is mostly devoted to pins). Delidding is a bet that you have better thermal interface material than what was used between the die(s) and the IHS. This was absolutely true in the past, but modern processors often use an indium soldering solution that is just about as good, IMHO, as anything you're going to do.
* I don't delid my processors, but it's an intriguing idea. Delidding is the removal of the processor's integrated heat spreader. The IHS is the metal cover atop your processor. The actual heat-generating die elements tend to be only a small portion of this total area (the remainder is mostly devoted to pins). Delidding is a bet that you have better thermal interface material than what was used between the die(s) and the IHS. This was absolutely true in the past, but modern processors often use an indium soldering solution that is just about as good, IMHO, as anything you're going to do.
* Liquid metal is conductive thermal paste. It is more thermally conductive than non-metal pastes, which is Good. It will short out any electronics it touches, which is Bad. I don't mess with it. The place to use liquid metal IMHO is between the die and IHS when delidding. In this case it's contained, and you're usually replacing a crap TIM. Liquid metal's thermal conductance is no better than a good indium solder, so there's just no point with a good die-to-IHS TIM. Atop the IHS, it's unlikely that liquid metal will beat quality non-conducting paste by more than one or two degrees.
* Liquid metal is electrically conductive thermal paste. It is more thermally conductive than non-metal pastes, which is Good. It will short out any electronics it touches, which is Bad. I don't mess with it. The place to use liquid metal IMHO is between the die and IHS when delidding. In this case it's contained, and you're usually replacing a crap TIM. Liquid metal's thermal conductance is no better than a good indium solder, so there's just no point with a good die-to-IHS TIM. Atop the IHS, it's unlikely that liquid metal will beat quality non-conducting paste by more than one or two degrees.
* No matter your cooling strategy, you'll be fucked without properly-applied high-quality thermal paste. It is easy to apply too little or too much. Application of the waterblock/heatsink will spread the paste for you; you want a very thin layer across the entirety of the integrated heat spreader. A thick layer will absolutely cause problems. Thermal paste ought be replaced every few years, theoretically.
* No matter your cooling strategy, you'll be fucked without properly-applied high-quality thermal paste. It is easy to apply too little or too much. Application of the waterblock/heatsink will spread the paste for you; you want a very thin layer across the entirety of the integrated heat spreader. A thick layer will absolutely cause problems. Thermal paste ought be replaced every few years, theoretically.


Line 58: Line 62:
I skipped over AIOs (all-in-ones, aka "closed loops") entirely, opting for a custom loop out of the gate. An AIO will consist of a waterblock with integrated pump, a radiator, tubing to connect the two, and fans on said radiator. AIOs are <i>significantly</i> cheaper, easier to install, simpler to work with, and don't end up requiring a bunch of small expensive attachments. You buy it, install it, and you're done. They have limited lifetimes due to coolant loss, though I wouldn't be at all surprised if someone brought out a refillable AIO. They're comparable to the best air coolers with regards to cooling capability, though they'll likely run quieter.
I skipped over AIOs (all-in-ones, aka "closed loops") entirely, opting for a custom loop out of the gate. An AIO will consist of a waterblock with integrated pump, a radiator, tubing to connect the two, and fans on said radiator. AIOs are <i>significantly</i> cheaper, easier to install, simpler to work with, and don't end up requiring a bunch of small expensive attachments. You buy it, install it, and you're done. They have limited lifetimes due to coolant loss, though I wouldn't be at all surprised if someone brought out a refillable AIO. They're comparable to the best air coolers with regards to cooling capability, though they'll likely run quieter.
* <b>This is an <i>expensive</i> hobby.</b> I'm lucky enough to have essentially unlimited funds, as I'm a software engineer of more than twenty years with no dependents. If you can't comfortably make a $100 purchase without thinking about it, you probably shouldn't be spending money on watercooling.
* <b>This is an <i>expensive</i> hobby.</b> I'm lucky enough to have essentially unlimited funds, as I'm a software engineer of more than twenty years with no dependents. If you can't comfortably make a $100 purchase without thinking about it, you probably shouldn't be spending money on watercooling.
* <b>It's less risky than you think.</b> I rejected watercooling for many years because the entire premise seemed ludicrous: one might as well fill your machine with powerful magnets, or fragmentation grenades, or alligator snapping turtles. Water doesn't belong in machines. It's why we have them in a case rather than just sitting around in the front yard. So first off, your watercooling components needn't come anywhere near your storage, which is where the irreplaceable stuff lives. My M.2 drives are all sealed away, and my spinning disks are located far away from any water. Any leak is almost certainly going to be your fault, and will be detectable before turning on any other components: this is the <i>leak test</i>. Hook up your power supply and watercooling components, but <i>leave everything else unpowered</i>. Fill and run your loop, ideally for several hours at maximum levels. Put paper towels underneath all loop components. If the paper towels are dry, go ahead and turn everything else on, hurrah. If they're wet, you have a problem. <b>Getting water on components <i>while they're powered down</i> is no big deal.</b>
* <b>It's less risky than you think.</b> I rejected watercooling for many years because the entire premise seemed ludicrous: one might as well fill one's box with powerful magnets, or fragmentation grenades, or alligator snapping turtles. Water doesn't belong in machines. It's why we have them in a case rather than just sitting around in the front yard. So first off, your watercooling components needn't come anywhere near your storage, which is where the irreplaceable stuff lives. My M.2 drives are all sealed away, and my spinning disks are located far away from any water. Any leak is almost certainly going to be your fault, and will be detectable before turning on any other components: this is the <i>leak test</i>. Hook up your power supply and watercooling components, but <i>leave everything else unpowered</i>. Fill and run your loop, ideally for several hours at maximum levels. Put paper towels underneath all loop components. If the paper towels are dry, go ahead and turn everything else on, hurrah. If they're wet, you have a problem. <b>Getting water on components <i>while they're powered down</i> is no big deal.</b>
* Avoid particulate coolants. They gunkify components over time, making them look like shit and reducing flow. Distilled water ought be treated with biocides and anticorrosive agents (easily available as coolant concentrates), which will be included in any premixed cooler.
* Avoid particulate coolants. They gunkify components over time, making them look like shit and reducing flow. Distilled water ought be treated with biocides and anticorrosive agents (easily available as coolant concentrates), which will be included in any premixed cooler.
* Order of components doesn't matter <i>assuming sufficient flow</i>. If your flow rate is very low, you can get significant localized heating, at which point heat-generating components without intermediary heat-expelling components can be suboptimal. With sufficient flow rate, your loop will reach effective equilibrium. This question comes up because if you're watercooling both a CPU and GPU (or multiple GPUs), they're generally close together, and it's unwieldy to separate them with a radiator. It's certainly not worth running distinct loops (which would require multiple pumps, which ought be able to achieve the necessary flow rate when placed in series).
* Order of components doesn't matter <i>assuming sufficient flow</i>. If your flow rate is very low, you can get significant localized heating, at which point heat-generating components without intermediary heat-expelling components can be suboptimal. With sufficient flow rate, your loop will reach effective equilibrium. This question comes up because if you're watercooling both a CPU and GPU (or multiple GPUs): they're generally close together, and it's unwieldy to separate them with a radiator. It's certainly not worth running distinct loops (which would require multiple pumps, which ought be able to achieve the necessary flow rate when placed in series).
* Increasing flow rate is pretty much only useful up to rough temperature equilibrium. Running your pumps higher will add more heat to the loop (assuming they're immersed, as they usually are), and reduce their lifespan.
* Increasing flow rate is pretty much only useful up to rough temperature equilibrium. Running your pumps higher will add more heat to the loop (assuming they're immersed, as they usually are), and reduce their lifespan.
* Tubing kinks reduce flow, as does transport (especially vertically), as do attachments (especially angled ones), as do radiators and waterblocks.
* Tubing kinks reduce flow, as does transport (especially vertically), as do attachments (especially angled ones), as do radiators and waterblocks.
Line 66: Line 70:
* SSDs are meant to run hotter than hard drives. You don't need to watercool them, nor RAM. Please don't try to watercool your hard drives.
* SSDs are meant to run hotter than hard drives. You don't need to watercool them, nor RAM. Please don't try to watercool your hard drives.


Every pump I'm aware of runs off 12V, with a maximum draw no greater than 30W. You can thus power them with any combination of Molex, SATA, or even PCIe cables, so long as you match the final form factor. I use PCIe cables with Molex converters; [https://www.performance-pcs.com/cables/power-supply-adapter-cables/pci-express-to-molex-adapter-cable-sleeved-pcie2molex-c.html PerformancePCs] sell a nice sleeved one. Almost all pumps are either D5 or DDC; the former is quieter and more powerful, but slightly larger. I roll exclusively with Liang D5s. PWM models can be controlled on the fly, while "Vario" models have a physical potentiometer which almost always ends up being a bitch and a half to get to. I recommend going with PWM. Varios will typically have a 3-pin hookup reporting tach, while PWM will have a 4-pin hookup.
Every pump I'm aware of runs off 12V, with a maximum draw no greater than 30W. You can thus power them with any combination of Molex, SATA, or even PCIe cables, so long as you match the final form factor. I use PCIe cables with Molex converters; [https://www.performance-pcs.com/cables/power-supply-adapter-cables/pci-express-to-molex-adapter-cable-sleeved-pcie2molex-c.html PerformancePCs] sell a nice sleeved one. Almost all pumps are either D5 or DDC; the former is quieter and more powerful, but slightly larger. I roll exclusively with Laing D5s. PWM models can be controlled on the fly, while "Vario" models have a physical potentiometer which almost always ends up being a bitch and a half to get to. I recommend going with PWM. Varios will typically have a 3-pin hookup reporting tach, while PWM will have a 4-pin hookup.


As for soft vs hard tubing, I use soft tubing, and don't intend to ever do otherwise. Soft tubing is trivial to cut, cheap, has plenty of give, bends without a heating gun, doesn't shatter, and is already quite enough of a pain in the ass. Old types of soft tubing were prone to plasticizer leeching, but newer types don't have them. I recommend [https://www.ekwb.com/shop/ek-tube-zmt-matte-black-19-4-12-5mm EKWB ZMT] (zero-maintenance tubing), so long as you don't mind it being opaque. Cheap anti-kinking coil can wrap your soft tubing to give it more resistance, and also provide some color. (Some) hard tubing can survive higher temperatures than soft tubing, but such high temperatures are bad for your other components, especially pumps, and ought be avoided.
As for soft vs hard tubing, I use soft tubing, and don't intend to ever do otherwise. Soft tubing is trivial to cut, cheap, has plenty of give, bends without a heating gun, doesn't shatter, and is already quite enough of a pain in the ass. Old types of soft tubing were prone to plasticizer leeching, but newer types don't have them. I recommend [https://www.ekwb.com/shop/ek-tube-zmt-matte-black-19-4-12-5mm EKWB ZMT] (zero-maintenance tubing), so long as you don't mind it being opaque. Cheap anti-kinking coil can wrap your soft tubing to give it more resistance, and also provide some color. (Some) hard tubing can survive higher temperatures than soft tubing, but such high temperatures are bad for your other components, especially pumps, and ought be avoided.