Liquid Cooling for AI Data Centers: Why Air Cooling Hit Its Limit

For most of the data center's history, cooling was a solved problem. Servers pushed warm air out of their backs, air handlers pulled it away, and chillers carried the heat outside. Better fans and hot-aisle/cold-aisle layouts stretched that design for decades.

AI broke it. A single rack of Nvidia's GB200 NVL72 system pulls 120 to 130 kW, while air cooling was built around racks of 8 to 12 kW (see Sources, [2]). That is roughly a tenfold jump in the heat concentrated in one rack footprint — and it is why the industry is now running water, and in some cases special fluids, directly to the chips.

This article explains what changed, why liquid moves heat so much better than air, how the main liquid-cooling designs differ, and what the shift means for the economics of a data center.

The problem starts with the chip

Almost all the electricity a server draws ends up as heat, so the power rating of an accelerator is also, in practice, a heat rating.

That rating has climbed fast. IEEE Spectrum traces Nvidia's data-center GPUs from 300 watts for the V100 in 2017, to up to 400 W for the A100 in 2020, up to 700 W for the H100 in 2022, and up to 1,200 W for Blackwell GPUs revealed in 2024 (see Sources, [1]). The same article quotes Drew Matter of Mikros Technologies saying chip roadmaps are looking at more than 2,000 watts per chip within the next year or two (see Sources, [1]).

Source: IEEE Spectrum (see Sources, [1]).

One hot chip is not the problem by itself. The problem is that AI systems pack dozens of them tightly together, because the chips need to exchange data over very short, very fast links. Spreading them out to make cooling easier would slow the system down.

From 8 kW to 120 kW per rack

Rack power density is the clearest way to see the change.

Network World puts typical racks at 2 to 3 kW in the 1980s, rising to 5 to 8 kW through the 2000s (see Sources, [2]). It cites an October 2024 McKinsey report finding that average rack density more than doubled in two years, from 8 kW to 17 kW, with 30 kW projected by 2027 — and notes that AI training racks are already well ahead of that average (see Sources, [2]).

Josh Claman, CEO of the cooling startup Accelsius, described the same shift to IEEE Spectrum: the average rack was around 8 kW, and for AI it is growing to 100 kW per rack (see Sources, [1]).

The GB200 NVL72 rack at 120 to 130 kW sits at the far end of that curve (see Sources, [2]). If you want to see what those densities mean in heat and electricity, our rack density and cooling calculator turns a rack count and kW per rack into estimated IT heat output, facility power and annual electricity cost.

Why air runs out — and why water doesn't

Air is a poor carrier of heat. To pull more heat out of a rack with air, you have to push more air through it, faster. At some point the fans, the airflow paths and the room itself cannot keep up.

Water is a far better medium. IEEE Spectrum explains that water has roughly four times the specific heat of air and is about 800 times as dense, so it can absorb around 3,200 times as much heat as a comparable volume of air (see Sources, [1]). Its thermal conductivity is also 23.5 times that of air, which means heat moves into it far more readily (see Sources, [1]).

That is the whole case for liquid cooling in two numbers: per unit of volume, liquid carries vastly more heat, and it takes that heat in faster.

Network World summarises JLL research that places approximate thresholds on each approach (see Sources, [2]):

Source: Network World, summarising JLL research (see Sources, [2]).

The four main types of liquid cooling

"Liquid cooling" covers several quite different designs. IEEE Spectrum groups them into four (see Sources, [1]).

1. Single-phase direct-to-chip

This is the most mature approach. Metal blocks called cold plates, with channels inside, sit directly on top of the hottest chips. A water-glycol mixture is pumped through them, carrying heat away at the source (see Sources, [1]).

It is also the easiest to fit into existing buildings. Network World notes that direct-to-chip cooling deploys rack by rack and works with existing chilled-water infrastructure, rather than requiring a facility rebuild (see Sources, [2]).

2. Two-phase direct-to-chip

Here the cold plate carries a fluid designed to boil on contact with the hot chip. Turning liquid into vapour absorbs extra energy — latent heat — without raising the fluid's temperature, so each unit of coolant removes more heat (see Sources, [1]). The trade-off is more specialised fluids and equipment.

3. Single-phase immersion

Instead of racks, the data hall is arranged around tanks roughly the size of a refrigerator. Whole servers are submerged in a non-conductive dielectric fluid, usually an oil, which absorbs heat from every component (see Sources, [1]).

4. Two-phase immersion

The most aggressive option: servers sit in a fluid that boils at a low temperature. The vapour condenses on a cooled coil inside the tank and drips back down. Lucas Beran of Accelsius called it "probably the most moon-shot technology" in data-center liquid cooling (see Sources, [1]). Critics point to the difficulty of servicing equipment in tanks of specialised, expensive fluid that can evaporate (see Sources, [1]).

In practice, AI data centers are hybrid

Liquid cooling does not usually replace air entirely.

In Nvidia's GB200 design, the compute processors and network switch chips are liquid-cooled, while storage, power distribution and ancillary networking are still air-cooled; Network World describes real-world GB200 deployments as hybrid by design (see Sources, [2]). Karin Overstreet of Nortek Data Center Cooling gave IEEE Spectrum a typical split: liquid does about 80 percent of the cooling for a data hall, and existing air cooling handles about 20 percent (see Sources, [1]).

That hybrid reality matters for operators. A facility does not need to be rebuilt from scratch to host AI racks, but it does need piping, coolant distribution, and a way to reject a much larger amount of heat from a much smaller area.

How fast the shift is happening

The market numbers suggest this is no longer an experiment.

Network World cites a January 2026 Dell'Oro Group report finding the liquid-cooling market nearly doubled in 2025, approaching $3 billion, with a forecast of $7 billion by 2029 (see Sources, [2]). It also cites a November 2025 S&P Global 451 Research survey: only 45% of data centers now run purely on air cooling, down from 48% in 2024, and 59% plan to implement liquid cooling within five years (see Sources, [2]).

The largest operators are already there. Network World reports that Microsoft moved all new data-center designs to closed-loop, zero-water-evaporation liquid cooling in August 2024, saving more than 125 million liters of water per facility per year, and that Meta committed $800 million to a liquid-cooled AI data center in Indiana (see Sources, [2]).

The economics: energy, water and cost

Cooling is one of the biggest costs in running a data center. Network World cites the U.S. Department of Energy estimate that cooling accounts for up to 40% of data-center energy use (see Sources, [2]). Anything that cools more efficiently goes straight to the operating budget — and, in a power-constrained market, frees up electricity for revenue-earning IT equipment. (For how power and capacity turn into revenue, see how data centers make money.)

Liquid cooling can also change what equipment a facility needs. At CES in January 2026, Nvidia said its Vera Rubin platform supports liquid cooling at 45 °C — warm enough, Network World explains, for data centers to reject heat through dry coolers using outside air rather than energy-hungry mechanical chillers (see Sources, [2]).

The upfront picture is more complicated, because the right choice depends on climate, density and the existing building. One published comparison gives a sense of scale. A case study by Chemours researchers, reported by IEEE Spectrum, estimated 10-year total cost of ownership for a site in Ashburn, Virginia (see Sources, [1]):

Source: Chemours case study, as reported by IEEE Spectrum (see Sources, [1]).

What it means

Liquid cooling is being adopted for a simple reason: the heat coming off AI hardware has outgrown what moving air can remove from a single rack. The chips went from hundreds of watts to more than a thousand, and the racks from single-digit kilowatts to more than a hundred.

For data-center operators, that turns cooling from background infrastructure into a strategic decision. It shapes which buildings can host AI hardware, how much of a facility's limited power reaches revenue-earning servers, and how much water a site consumes. For the suppliers of cold plates, coolant distribution units, fluids and heat-rejection equipment, it has become one of the fastest-growing parts of the AI build-out.

To explore the numbers yourself, try the rack density and cooling calculator or the AI data center power and cost calculator. For the power side of the same problem, read why the grid is becoming AI's bottleneck.

This article is for educational and informational purposes only. It is not engineering, financial or investment advice.