The Power: Difference between revisions
m Dank moved page The Power, pt 1 to The Power |
|||
| (36 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
<b>nb: if this reads like a script, that's because it was originally going to be a video in my [[DANKTECH]] series. maybe it still will, one day.</b> | |||
<b>nb: | |||
hey there, my hax0rs and hax0rettes! it's been a good long time since we last | hey there, my hax0rs and hax0rettes! it's been a good long time since we last | ||
| Line 15: | Line 13: | ||
with a critical ear and watch with a scrupulous eye, but, you know, you really | with a critical ear and watch with a scrupulous eye, but, you know, you really | ||
ought be doing that all the time. if i have a general theme tonight, i suppose | ought be doing that all the time. if i have a general theme tonight, i suppose | ||
that theme is: <b>power</b>. | that theme is: <b>power</b>. in the world of huey lewis and the news, love can be measured in watts, the SI unit of power. love applied over time would furthermore be work, and measured in joules. but i'm an engineer, so this evening we'll be quantifying the power of love, but mainly the love electrons feel towards cations. | ||
in the world of huey lewis and the news, love can be measured in watts, the SI unit of | |||
power. love applied over time would furthermore be work, and measured in joules. but | |||
i'm an engineer, so this evening we'll be quantifying the power of love, but mainly | |||
the love electrons feel towards cations. | |||
| Line 157: | Line 148: | ||
So the highest voltage we can get from the PSU is 12V, so that's what the CPU | So the highest voltage we can get from the PSU is 12V, so that's what the CPU | ||
wants. And thus ATXV12 was born | wants. And thus in February 2000 ATXV12 was born, adding a 4-pin Molex Minifit | ||
Jr 39-01-2040 carrying an extra 3.3, 5, and 12V pin (plus an extra ground). Now | Jr 39-01-2040 carrying an extra 3.3, 5, and 12V pin (plus an extra ground). Now | ||
we already had 3x 3.3V pins and 4x 5V pins, so our capacities there rose 33% | we already had 3x 3.3V pins and 4x 5V pins, so our capacities there rose 33% | ||
| Line 522: | Line 513: | ||
you're surely aware that the cpu is built up of transistors. my amd 3970x has | you're surely aware that the cpu is built up of transistors. my amd 3970x has | ||
15.2 gigatransistors on TSMC's 7nm process. nvidia's a100 has 54 billion. | 15.2 gigatransistors on TSMC's 7nm process and another eight on 12nm. nvidia's a100 has 54 billion. | ||
graphcore's colossus MK2 boasts 59.6 billion. samsung built a 1TB eUFS VNAND | graphcore's colossus MK2 boasts 59.6 billion. samsung built a 1TB eUFS VNAND | ||
flash with 2 teratransistors. but the reigning dance hall champion, at least | flash with 2 teratransistors. but the reigning dance hall champion, at least | ||
| Line 540: | Line 531: | ||
[ E = k_B T ln2 ] | [ E = k_B T ln2 ] | ||
and that's Landauer's principle, and you really ought go read john | and that's Landauer's principle, and you really ought go read john archibald | ||
wheeler's "it from bit", which is fantastic. | wheeler's <i>"it from bit"</i>, which is fantastic. | ||
k_B is the boltzmann constant 1.38 * 10^-23 J/K. so at 30C, we've got 303.15K, | k_B is the boltzmann constant 1.38 * 10^-23 J/K. so at 30C, we've got 303.15K, | ||
| Line 751: | Line 742: | ||
polarity. when you plug a conductor in, a path is made, and now electrons can | polarity. when you plug a conductor in, a path is made, and now electrons can | ||
shuffle along. nuclei don't move, but the valence electrons do, each | shuffle along. nuclei don't move, but the valence electrons do, each | ||
representing one fundamental charge. an amp is about 6. | representing one fundamental charge. an amp is about 6.24x10¹⁸ elementary charges. | ||
to conduct more amps, you need a bigger conductor; the number of charges that | to conduct more amps, you need a bigger conductor; the number of charges that | ||
can move is directly proportional to the cross-section of the wire, and thus | can move is directly proportional to the cross-section of the wire, and thus | ||
| Line 851: | Line 842: | ||
itself. the charge is propagated along as a change to the electric field, | itself. the charge is propagated along as a change to the electric field, | ||
exploding forward far more quickly than the electrons themselves (indeed, faster | exploding forward far more quickly than the electrons themselves (indeed, faster | ||
than any massive particle will move | than any massive particle will or can move, as we know | ||
from special relativity). a final speed we want to look at is thus the signal | from special relativity). a final speed we want to look at is thus the signal | ||
propagation velocity, the speed with which this change to the field ripples out. | propagation velocity, the speed with which this change to the field ripples out. | ||
| Line 923: | Line 914: | ||
would raise the temperature, and that denatures proteins. most enzymatic | would raise the temperature, and that denatures proteins. most enzymatic | ||
proteins base their tertiary structure on hydrogen and van der waals bonds, | proteins base their tertiary structure on hydrogen and van der waals bonds, | ||
and we're talking weak hydrogen bonds here a few kilojoules per mole. | and we're talking weak hydrogen bonds here: a few kilojoules per mole. | ||
well, a square meter receives about 1.4 kilojoules every second in full | well, a square meter receives about 1.4 kilojoules every second in full | ||
sunlight. a nutritional calorie is 4 kilojoules. and that's per mole. | sunlight. a nutritional calorie is 4 kilojoules. and that's per mole. | ||
| Line 991: | Line 982: | ||
energy budget, but it's fascinating. as one last note, if you've ever | energy budget, but it's fascinating. as one last note, if you've ever | ||
wondered why carbs and proteins are considered 4kcals a gram, and fats are | wondered why carbs and proteins are considered 4kcals a gram, and fats are | ||
9 kcals, know that it's the Atwater system, it's | 9 kcals, know that it's the Atwater system, it's digestibility-scaled (proteins | ||
have more raw energy, but we piss away 20% of the protein we take in), and | have more raw energy, but we piss away 20% of the protein we take in), and | ||
fats lack oxygen, supplying only energy-rich CH bonds (oxygen being the | fats lack oxygen, supplying only energy-rich CH bonds (oxygen being the | ||
| Line 1,030: | Line 1,021: | ||
of longitudinal and traverse waves, and the kinetic energy of displaced | of longitudinal and traverse waves, and the kinetic energy of displaced | ||
particles. pneumatics, hydraulics, telodynamics, gears, these all exist. | particles. pneumatics, hydraulics, telodynamics, gears, these all exist. | ||
london moved 7000 | fin de siècle london moved 7000 horsepower over 180 miles of pipes carrying water at | ||
800PSI. rather than electric lines from Georgia Power, we could have Municipal | 800PSI. rather than electric lines from Georgia Power, we could have Municipal | ||
Belts and Yaks, and somewhere in Georgia's blighted south there'd be tens of | Belts and Yaks, and somewhere in Georgia's blighted south there'd be tens of | ||
| Line 1,128: | Line 1,119: | ||
course. steel-core aluminum wires generally. let's look at some of the numerics here. | course. steel-core aluminum wires generally. let's look at some of the numerics here. | ||
georgia has nuclear plants in Waynesboro and Baxley, about 150 and 180 | georgia has nuclear plants in Waynesboro and Baxley, about 150 and 180 | ||
miles respectively from power-hungry Atlanta. | miles respectively from power-hungry Atlanta. their turbines generate 26kV, | ||
which is taken up to 500kV at its generator step up units. this is three-phase | which is taken up to 500kV at its generator step up units. this is three-phase | ||
alternating current. | alternating current. | ||
| Line 1,190: | Line 1,181: | ||
==Let's do the do== | ==Let's do the do== | ||
alright, with this information from the previous five sections, we can pretty | alright, with this information from the previous five sections, we can pretty | ||
much | much conclude that our workstation is going to need to accept 120VAC/60Hz | ||
alternating current sourced from a utility. some people would have just | alternating current sourced from a utility. some people would have just taken this on faith, but like i said, i like to get to the bottom of things. | ||
this, but like i said, i like to get to the bottom of things. | |||
the remaining questions are how to distribute power within the machine, and | the remaining questions are how to distribute power within the machine, and | ||
| Line 1,331: | Line 1,321: | ||
what is the goal of cooling? first and foremost, the active components must maintain temperatures within their operating ranges. electric devices have a threshold temperature and a maximum temperature. peak (advertised) performance is typical up to the threshold temperature. the device derates beyond that, and should not be operated above the maximum temperature. modern processors have thermal protection built in to keep temperatures from getting too high (my AMD 3970X oughtn't run over 95℃). older processors, especially AMD chips from the beginning of the century, would begin literally smoldering, reaching temperatures of hundreds of degrees, if run without cooling for even a few seconds. for processors, the resistance (and thus heat generation) is very dense; the first task of any cooling system is to remove heat from these units. modern processors typically advertise "boost" speeds; these speeds can only be reached below temperature thresholds. indeed, even the "base" clocks can only be reached with a decent cooling solution, lest the thermal protection lead to "throttling". remember that in the presence of overclocking, more heat and power are in play than at design clocks. | what is the goal of cooling? first and foremost, the active components must maintain temperatures within their operating ranges. electric devices have a threshold temperature and a maximum temperature. peak (advertised) performance is typical up to the threshold temperature. the device derates beyond that, and should not be operated above the maximum temperature. modern processors have thermal protection built in to keep temperatures from getting too high (my AMD 3970X oughtn't run over 95℃). older processors, especially AMD chips from the beginning of the century, would begin literally smoldering, reaching temperatures of hundreds of degrees, if run without cooling for even a few seconds. for processors, the resistance (and thus heat generation) is very dense; the first task of any cooling system is to remove heat from these units. modern processors typically advertise "boost" speeds; these speeds can only be reached below temperature thresholds. indeed, even the "base" clocks can only be reached with a decent cooling solution, lest the thermal protection lead to "throttling". remember that in the presence of overclocking, more heat and power are in play than at design clocks. | ||
secondly, the temperature inside the machine must be kept below thresholds. higher case temperatures will degrade performance and reduce lifetimes for peripherals, and retard the ability of the cooling system to remove heat from the processors. finally, it is desirable to do this as efficiently as possible, since the cooling itself consumes power and generates noise. leaving aside immersion cooling, this means we need to get the hot air out of the system. | secondly, the temperature inside the machine must be kept below thresholds. higher case temperatures will degrade performance and reduce lifetimes for peripherals, and retard the ability of the cooling system to remove heat from the processors. finally, it is desirable to do this as efficiently as possible, since the cooling itself consumes power and generates noise. leaving aside immersion cooling, this means we need to get the hot air out of the system. the ambient temperature is critical: higher ambient temperatures will mean hotter temperatures throughout, and (as we'll see below) lower heat flux (exchange with the environment). "standard ambient temperature" is 22℃. at the moment of startup, all components within a machine are equal to the ambient temperature; <i>whether using air or liquid cooling, they will never go below</i> (it is possible to reach subambient temperatures with extreme cooling methods i don't plan to cover in any detail). <b>it is completely meaningless to compare uncorrected operating temperatures at different ambient temperatures.</b> in addition, the sensors in your machine all have some degree of systemic error; it is not generally useful to compare absolute temperatures across machines or even across sensors. | ||
fun fact: system firmware boots the processor without any power-saving features, as you probably knew, and at a fixed, higher-than-default voltage (to help ensure proper booting), which i'm guessing you didn't. temperatures there will usually be noticeably higher than when idling in the operating system environment. | |||
[[File:Amd3970xdelidded.png|left|thumb|gold-plated IHS contacts, classy.]] | [[File:Amd3970xdelidded.png|left|thumb|gold-plated IHS contacts, classy.]] | ||
my AMD Threadripper 3970X (Zen 2 microarchitecture, but 3rd Generation Threadripper, in a maddening disconnect) is 4 "core complex dies" (CCDs), each containing two "core complexes" (CCXen). a core complex contains four cores and their caches (Zen 3 dispenses with the CCX concept; CCDs remain octacore, but unite the L3 cache). each CCD is 74mm² containing 3.9 gigatransistors on TSMC's 7nm process. "Infinity Fabric" connects them to a central "i/o die" (IOD) is 416mm² containing 8.34 gigatransistors on TSMC's 12nm process (this is apparently borrowed from 2nd Generation EPYC). the TRX40 chipset uses a 14nm process from Global Foundries and dissipates 15W itself, talking to the CPU over 16 PCIe 4.0 lanes. the package is 4410.9mm². the five dies sum to 712mm², 16.14% of the package area. at 280W, this is an average of 0.393W/mm² or 39.33W/cm² | my AMD Threadripper 3970X (Zen 2 microarchitecture, but 3rd Generation Threadripper, in a maddening disconnect) is 4 "core complex dies" (CCDs), each containing two "core complexes" (CCXen). a core complex contains four cores and their caches (Zen 3 dispenses with the CCX concept; CCDs remain octacore, but unite the L3 cache). each CCD is 74mm² containing 3.9 gigatransistors on TSMC's 7nm process. "Infinity Fabric" connects them to a central "i/o die" (IOD) is 416mm² containing 8.34 gigatransistors on TSMC's 12nm process (this is apparently borrowed from 2nd Generation EPYC). the TRX40 chipset uses a 14nm process from Global Foundries and dissipates 15W itself, talking to the CPU over 16 PCIe 4.0 lanes. the package is 4410.9mm². the five dies sum to 712mm², 16.14% of the package area. at 280W, this is an average of 0.393W/mm² or 39.33W/cm². | ||
this topology changes the heat profile of the package. we're used to seeing a single central heat element, but here we'll have five heat generators in a quincunx configuration. the first step in cooling is the heat spreader, part of the retail CPU package. the integrated heat spreader's gold-plated base is soldered directly to the dies using an indium solder, a top-of-the-line solution. delidding this processor (removing the IHS to apply superior thermal interface material) is not likely to be helpful, and the few experiments i've seen confirm that intuition. this is a large package, and one's well-advised to get a cooler that covers the entire IHS. thankfully, there now exist numerous solutions for the sizable sTRX40. | this topology changes the heat profile of the package. we're used to seeing a single central heat element, but here we'll have five heat generators in a quincunx configuration. the first step in cooling is the heat spreader, part of the retail CPU package. the integrated heat spreader's gold-plated base is soldered directly to the dies using an indium solder, a top-of-the-line solution. delidding this processor (removing the IHS to apply superior thermal interface material) is not likely to be helpful, and the few experiments i've seen confirm that intuition. this is a large package, and one's well-advised to get a cooler that covers the entire IHS. thankfully, there now exist numerous solutions for the sizable sTRX40. | ||
| Line 1,342: | Line 1,334: | ||
Fourier's law, the heat analogue of Ohm's law, states that heat transfer is proportional to the temperature gradient and proportional to the area normal to said gradient through which heat flows: q = -k∇T, ∂Q/∂t = -k∯∇T * dS. k is the material's thermal conductivity in W/mK, ∇T is the temperature gradient in K/m, q is the local heat flux density in W/m², and dS is an orthonormal surface area element in m². at 0℃, the thermal conductivity of copper is 401 W/mK; this falls to about 390 as the temperature approaches 100℃. its nickel plating is closer to 100W/mK. there will be some small thermal resistance due to the thermal interface material (it is for this reason that applying even slightly too much TIM is counterproductive. if enough TIM is used, it effectively becomes the conductor, which is very bad). the best non-conductive TIMs have conductivity of about 8.5–10W/mK; conductive "liquid metal" TIMs can reach 40, still well below true metal conductors. | Fourier's law, the heat analogue of Ohm's law, states that heat transfer is proportional to the temperature gradient and proportional to the area normal to said gradient through which heat flows: q = -k∇T, ∂Q/∂t = -k∯∇T * dS. k is the material's thermal conductivity in W/mK, ∇T is the temperature gradient in K/m, q is the local heat flux density in W/m², and dS is an orthonormal surface area element in m². at 0℃, the thermal conductivity of copper is 401 W/mK; this falls to about 390 as the temperature approaches 100℃. its nickel plating is closer to 100W/mK. there will be some small thermal resistance due to the thermal interface material (it is for this reason that applying even slightly too much TIM is counterproductive. if enough TIM is used, it effectively becomes the conductor, which is very bad). the best non-conductive TIMs have conductivity of about 8.5–10W/mK; conductive "liquid metal" TIMs can reach 40, still well below true metal conductors. | ||
< | atop our IHS goes more TIM (we want TIM wherever we have two rough surfaces meeting for thermal transfer), and then our real cooling story begins. we've safely conducted heat away from its source, but without further removing that heat, our IHS will eventually serve as a blanket. eventually, we want the heat out of the system entirely, mixing with and effectively being hidden by the ambient air (even this doesn't necessarily complete the solution, as anyone who's locked a few servers away in a poorly-ventilated closet can tell you). the usual solution is a large heatsink with numerous fins, maybe some heat pipes, and localized fans, combined with ingress and egress fans on the case. this results in heat dissipating from the processor complexes (and VRMs, and memory, and chipset, etc.) into the case's air, and that air being ejected, with ambient air replacing it. | ||
[[File:Ohyeah.jpg|right|thumb|[ferris bueller music intensifies]]] | |||
let's analyze this. my Caselabs Magnum T10A is 15x25.06x26.06in, or 381x637x662mm. empty, that's 0.161m³. at 25℃ and one atmosphere, dry air has a density of 1.1839kg/m³ (it gets less dense as temperature rises), suggesting 0.19kg of air in the case. its specific heat is 1.005 kJ/kg/K, which rises with temperature. assuming no replacement of air, 190.95J ought raise the temperature of the case's interior one degree. at 280W TDP, that amount of power requires 0.7s. i guess we'd better move that air! my [[Schwarzgerät_III#Cooling|custom loop]] contains about 1L of distilled water, weighing about 1kg. water's specific heat at 25℃ is 4.187kJ/kg/K. this fourfold increase in specific heat and 700+-fold increase in density work in our favor: it ought require about 4187J to raise the loop's temperature by a degree, about 22x as much power, requiring 15s at 280W. remember the all-important temperature delta: heat moves more effectively when there is a larger difference, and the liquid fosters such. it is furthermore a larger sink for sudden heat spikes. even with these advantages, however, it's no good if we can't get the liquid cooled; adding four degrees per minute is completely unsustainable. try running a watercooled system without any fans going, and you'll verify this quite quickly. furthermore, the heat doesn't propagate quickly enough through the liquid to be removed on its own; it's necessary to drive the liquid with a pump, and this can likewise be verified quickly and easily. indeed, without pumping the liquid, results are likely to be far worse than with air cooling. | |||
[ show machine running without fans, then without pumps ] | |||
another advantage is that our heat is removed from the loop at the case's boundaries (heat leakage from the tubes themselves is negligible), where we install our radiators. the hot air is thus ejected outside the system immediately, rather than transiting the interior. this keeps other components cooler, and is likely to remove the heat more quickly overall. finally, using the surface of the machine rather than the surface of the heatspreader as our release means massively more area. my machine boasts 1200mm² of radiator area, meaning greater volumes into which heat can diffuse. the upshot is lower temperatures per watt (opening up overclocking headroom and avoiding any thermal throttling), and fewer acoustic decibels per watt (meaning complete silence in which to hear the loudest fucking keyboard i can buy, CLACK CLACK CLACK). | |||
another concern with this build was power. while the Supernova can supply all its rated 850W as 12V DC, it can do only 100W as native 5V output. furthermore, SATA cables carry only 22.5W of 5V power and 54W of 12V power. i have 14 Seagate Exos 18TB hard drives, requiring peak inrush currents of 1.01 5V amps and 2.02 12V amps. that's 70.7W of 5V and 339.36W of 12V total, with each SATA power cable capable of handling only two drives within its spec (in actuality, the inrush current is only needed for a fraction of a second, and i've had no problem hanging four off a single cable...but this presumably tempts fate). with numerous other 5V draws competing, it's quite possible that i could exceed the PSU's 100W of 5V. what to do, what to do? | |||
well, again, you can consult [[Schwarzgerät_III|my writeup]] to get all the gory details. suffice to say that several 66W/12V PCIe extension cables were going unused, and were pressed into service for hard drives. PCIe 8-pin goes to Molex 2-pin 12V. splice the 12V line as it emerges from the Molex, and throw a buck converter on that fucker, bringing out a 5V level. feed it back into the Molex connector as the other two 5V pins. boom, we've got ~60W of combined 5+12V available from where there was once only 12V, <i>and it doesn't count against the PSU's 5V total</i>. and that, my droogs, is a hack like grandma used to make. | |||
==Final words== | |||
We've gone back to the big bang and seen how fourteen billion years and three generations of stars nucleosynthesized the elements we use to build and power our machines and ourselves. We've looked at the results of three and a half billion years of terrestrial evolution, probably not the only life in the universe, but definitely the only life we know. We talked about five hundred million years of pressure and desiccation, and how they gave rise to the fossil fuels that powered the Industrial Revolution and now present a bill in the form of climate change. | |||
We, as individuals or as a species, don't get billions of years, but only a brief existence, an absurd blink of the eye, our identities and senses of selves arising from a short-lived ensemble of low-entropy neurons, decaying like all things back into the lukewarm slurry of thermodynamic equilibrium. It is our duty in this interim to learn what we can, to improve our species, and to live bravely despite heat death's ineluctable execration. To <i>differentiate</i>. To put energies to use, to work and to extract work. Yet all remain destined for a night that is eternal and without name. One by one they will step down into the darkness before the footlamps. | |||
Until that time, we shall overclock. Thanks for reading. Hack on. | |||
<i>“I leave Sisyphus at the foot of the mountain. One always finds one's burden again. But Sisyphus teaches the higher fidelity that negates the gods and raises rocks. He too concludes that all is well. This universe henceforth without a master seems to him neither sterile nor futile. Each atom of that stone, each mineral flake of that night-filled mountain, in itself, forms a world. The struggle itself toward the heights is enough to fill a man's heart. One must imagine Sisyphus happy.”</i>—Albert Camus | |||
[[File:Therock.jpg|center|frame|<i>The Rock</i>, Peter Blume (1948)]] | |||
==Going deeper== | ==Going deeper== | ||