Check out my first novel, midnight's simulacra!

Reflections on watercooling

From dankwiki

I've been watercooling my primary personal machine since 2020. It started off simple—a waterblock on my Threadripper 3970X, an EKWB reservoir+pump combo, and a Hardware Labs GTR360 radiator. A total of about $480 (not including fans), less than my motherboard cost. It is no longer so simple: the same machine in its current incarnation boasts a monoblock, a GPU block, a custom GPU backplane, four radiators, two reservoirs, four D5 pumps, two temperature sensors, two discrete inline sensors, and about twenty attachments of all kinds. There's at least $1000 worth of watercooling crap in there, and another $500 of fans. I guess you could say things are getting serious.

I've put together some thoughts having watercooled for over two years.

Generic cooling notes

It might be useful to read section six from this essay of mine, but you can probably get by without it. It is very useful to have a working knowledge of the Laws of Thermodynamics and Fourier's Law.

  • We don't use Fahrenheit, but it's worth knowing that each ℃ is 1.8℉, with an offset of 32. We don't really use Kelvins either, but Kelvins are equivalent to Celsius with an offset of -273.15. Rankines can fuck right off. Liquids boil at lower temperatures at higher altitudes (boiling occurs when internal vapor pressure equals atmospheric pressure). Water expands both above and below 4℃. A quick table:
Notes
212 100 Water boils at sea level. Very undesirable in computers.
140 60 Upper limit for pump + some tubing.
90 32.2 Thermal throttling begins around here
75 23.9
60 15.6
39.2 4 Water's volume is minimized (density maximum).
32 0 STP. Water freezes at sea level, expanding by 9%.
  • Neither air- nor water-based cooling can get your temperatures below ambient (this can be achieved with phase cooling, thermoelectric aka Peltier, and cyrogenics). If it's 30 degrees in your room, your coolant and component temps are going to be at least 30 degrees. It is useless to report temperatures without reporting ambient temps. The effectiveness of your cooling system is measured by how close it gets to ambient, ΔT. Likewise, your heat-generating components are going to be hotter than your coolant.
  • The goal is to move heat from the heat-generating components, and ultimately from the case. If you can't remove as much heat as you generate, temperatures will rise with time. If you can remove all the heat you generate, temperatures will fall towards ambient. Effectively moving heat requires (a) thermal conductivity, (b) contact area and (c) a temperature gradient. The thermal conductivity of air is about two orders of magnitude less than water, which is about two orders of magnitude less than aluminum or copper.
    • A processor without a heatsink is depositing heat directly to stagnant air, which will heat up (reducing the temperature gradient) and can't take much heat anyway.
    • Adding a heatsink means the heat is deposited into a good conductor with more surface area...which in turn deposits into stagnant air. This air will heat up, and eventually the heatsink will, as well.
    • Adding a waterblock to the heatsink means that heat is deposited into water as opposed to air. That water has ~4 times the heat capacity of air, meaning it takes 4x heat to raise a unit mass of water by one degree. Furthermore, that water is ~800 times denser than air, meaning there's much more mass per unit volume. As noted above, the water is ~100 times as thermally conductive. It thus takes heat better, and the temperature gradient is lessened much less quickly. With that said, it's still depositing its heat into stagnant air, and will reach equilibrium with the heatsink given sufficient time.
    • Adding fans moves the hot air (ideally out of the case), to be replaced by ambient air. The ejected hot air will diffuse into the ambient air, which is (ideally) a large convecting volume.
    • Adding more water (via tubing and reservoirs) is like using a larger case: there is now more water to heat up. The heat moves through the water slowly, however; the water closest to the heat-producing components is still likely to overheat.
    • Adding water flow (via pumps) is like using internal fans, without expelling air from the case: we now make use of the full volume of the conductor, because it's moving around, and the heat is deposited throughout (much more quickly than it would move through the material itself). Without heat being expelled, however, we're still going to heat the entire volume given enough time.
    • Adding radiators at the surface of the case means the water is now depositing its heat to the ambient air. By using powerful fans colocated with the radiators, we can help the hot air diffuse more quickly into the ambient air.

We thus see the necessary components of an effective watercooling system: waterblocks (to conduct heat into the water), tubing (to contain the water), pumps (to move the water), radiators (to conduct heat into the ambient air), and fans (to move the air). Immersion cooling eliminates the need for the first two elements, at the cost of some madness.

Thermal interface material

TIM is thermally conductive viscous material applied between heat contacts. There's some between your dies and the IHS, and then some between the IHS and your waterblock/heatsink. Its purpose is to smooth out the conducting surface, which is dotted with microscopic pits that would otherwise preclude contact. TIM is one or two orders of magnitude less thermally conductive than metals making direct contact, but much more conductive than the air which would otherwise fill these regions. Thermal conductance is measured in W/Km (watts per kelvin-meter).

  • I don't delid my processors, but it's an intriguing idea. Delidding is the removal of the processor's integrated heat spreader. The IHS is the metal cover atop your processor. The actual heat-generating die elements tend to be only a small portion of this total area (the remainder is mostly devoted to pins). Delidding is a bet that you have better thermal interface material than what was used between the die(s) and the IHS. This was absolutely true in the past, but modern processors often use an indium soldering solution that is just about as good, IMHO, as anything you're going to do.
  • Liquid metal is conductive thermal paste. It is more thermally conductive than non-metal pastes, which is Good. It will short out any electronics it touches, which is Bad. I don't mess with it. The place to use liquid metal IMHO is between the die and IHS when delidding. In this case it's contained, and you're usually replacing a crap TIM. Liquid metal's thermal conductance is no better than a good indium solder, so there's just no point with a good die-to-IHS TIM. Atop the IHS, it's unlikely that liquid metal will beat quality non-conducting paste by more than one or two degrees.
  • No matter your cooling strategy, you'll be fucked without properly-applied high-quality thermal paste. It is easy to apply too little or too much. Application of the waterblock/heatsink will spread the paste for you; you want a very thin layer across the entirety of the integrated heat spreader. A thick layer will absolutely cause problems. Thermal paste ought be replaced every few years, theoretically.

It's not unwise to keep track of your ambient, coolant, and component temperatures over time. If you see the latter rising while the former is constant, your thermal material might be breaking down, or you might have dust in the machine, or you might have a contaminated waterblock.

Air cooling

Modern tower CPU coolers are pretty damn good, and can handle most processors without getting very loud or allowing the chip to get too hot. A top-of-the-line bequiet! or Noctua tower cooler can be had for $100, and will serve most people just fine. Get one, slap on some case fans, and you're done. GPU air coolers are not generally as capable (just look at the power draw of modern GPUs vs CPUs); if you intend to make heavy use of a powerful GPU, you'll be making a lot of noise. Air coolers are not generally well-suited for overclocking, but it depends entirely on how much power the die is drawing.

Your computer can almost certainly be effectively cooled with a tower cooler and some case fans.

Water cooling

Watercooling, then, has four advantages over air cooling:

  • Fans are the primary producers of noise within most computers. With a good watercooling setup, fans can be run at lower speeds for the same temperatures, reducing noise. It might alternatively be possible to use fewer fans (there are no CPU or GPU fans, for instance, when these components are watercooled). With that said, I have many more fans in my current waterbuild than I've ever used in a computer, but I'm also paying much more attention to temperatures and noise. By controlling fans based off coolant temperatures rather than component temperatures, there are far fewer distracting transitions between noise regimes.
  • Water can typically move more total heat than air, allowing higher performance envelopes on heat-generating components. Even without overclocking, it's easy to trigger thermal throttling on an air-cooled CPU or GPU. With overclocking in play, especially if higher voltages are used, watercooling can become necessary for proper function. Recall that the dynamic CMOS power equation is linear with regards to frequency and quadratic with regards to voltage.
  • Aesthetics. If you are friends with the kind of people who appreciate a classy computer, they're sure to be wowed by a good watercooling setup, before they go behind your back and talk about how dumb you are to be watercooling.
  • It sucks money out of your wallet, preventing you from spending it on cocaine.

I skipped over AIOs (all-in-ones, aka "closed loops") entirely, opting for a custom loop out of the gate. An AIO will consist of a waterblock with integrated pump, a radiator, tubing to connect the two, and fans on said radiator. AIOs are significantly cheaper, easier to install, simpler to work with, and don't end up requiring a bunch of small expensive attachments. You buy it, install it, and you're done. They have limited lifetimes due to coolant loss, though I wouldn't be at all surprised if someone brought out a refillable AIO. They're comparable to the best air coolers with regards to cooling capability, though they'll likely run quieter.

  • This is an expensive hobby. I'm lucky enough to have essentially unlimited funds, as I'm a software engineer of more than twenty years with no dependents. If you can't comfortably make a $100 purchase without thinking about it, you probably shouldn't be spending money on watercooling.
  • It's less risky than you think. I rejected watercooling for many years because the entire premise seemed ludicrous: one might as well fill your machine with powerful magnets, or fragmentation grenades, or alligator snapping turtles. Water doesn't belong in machines. It's why we have them in a case rather than just sitting around in the front yard. So first off, your watercooling components needn't come anywhere near your storage, which is where the irreplaceable stuff lives. My M.2 drives are all sealed away, and my spinning disks are located far away from any water. Any leak is almost certainly going to be your fault, and will be detectable before turning on any other components: this is the leak test. Hook up your power supply and watercooling components, but leave everything else unpowered. Fill and run your loop, ideally for several hours at maximum levels. Put paper towels underneath all loop components. If the paper towels are dry, go ahead and turn everything else on, hurrah. If they're wet, you have a problem. Getting water on components while they're powered down is no big deal.
  • Avoid particulate coolants. They gunkify components over time, making them look like shit and reducing flow. Distilled water ought be treated with biocides and anticorrosive agents (easily available as coolant concentrates), which will be included in any premixed cooler.
  • Order of components doesn't matter assuming sufficient flow. If your flow rate is very low, you can get significant localized heating, at which point heat-generating components without intermediary heat-expelling components can be suboptimal. With sufficient flow rate, your loop will reach effective equilibrium. This question comes up because if you're watercooling both a CPU and GPU (or multiple GPUs), they're generally close together, and it's unwieldy to separate them with a radiator. It's certainly not worth running distinct loops (which would require multiple pumps, which ought be able to achieve the necessary flow rate when placed in series).
  • Increasing flow rate is pretty much only useful up to rough temperature equilibrium. Running your pumps higher will add more heat to the loop (assuming they're immersed, as they usually are), and reduce their lifespan.
  • Tubing kinks reduce flow, as does transport (especially vertically), as do attachments (especially angled ones), as do radiators and waterblocks.
  • Don't mix metals. Doing so invites galvanic corrosion, which can only be slowed down by anticorrosives. This is most easily achieved by eschewing aluminum entirely. Don't worry about bronze vis-à-vis copper.