For many years, enterprise data centers were designed around racks of roughly 3 to 8 kW. Air cooling from perimeter units handled that comfortably. Training and inference clusters built on GPUs and other accelerators have changed the picture: a single rack of current accelerator hardware can draw tens of kilowatts, and the largest liquid-cooled designs exceed 100 kW per rack.
At these densities, the questions shift from "how much cooling capacity do we have?" to "how do we get heat out of each rack at all?". This guide walks through what changes, which technologies match which density ranges, and the decisions that must be made before equipment arrives.
Why density changes everything
Air-cooled servers need a certain volume of air per kilowatt, set by the temperature rise across the server. At a typical rise of about 10 to 15 K, each kilowatt needs roughly 200 to 300 m³/h (about 120 to 180 cfm) of air. A 40 kW rack therefore needs several thousand cubic metres per hour delivered to one cabinet face, and exhausted from the back without recirculating. Getting that air through perforated tiles, around cable bundles and into the right rack becomes impractical.
Liquid has a far higher volumetric heat capacity than air. A small water flow at the rack does the work of a very large airflow. That is why liquid cooling becomes necessary as densities rise.
Matching technology to density
The density bands below are indicative. Actual limits depend on room design, containment quality, server airflow and supply temperature.
| Rack density | Typical cooling approach |
|---|---|
| Up to about 10 kW | Room air cooling with good airflow management and containment |
| About 10 to 20 kW | Contained aisles, in-row coolers or close-coupled air cooling |
| About 20 to 40 kW | Rear-door heat exchangers, in-row cooling with containment, or partial liquid cooling |
| About 40 kW and above | Direct-to-chip liquid cooling with residual air cooling |
| Very high, uniform loads | Direct-to-chip or immersion cooling |
For the technologies in more depth, see in-row and rear-door heat exchanger cooling, direct-to-chip liquid cooling and immersion cooling.
The hybrid reality
Most AI deployments are hybrid. Even when accelerators and CPUs are on cold plates, the rest of the server, network switches, storage and power equipment remain air cooled. A rack that is 80 percent liquid cooled at 100 kW still rejects about 20 kW to air, which is itself a high-density air load.
A practical hybrid design addresses both:
- A liquid loop with CDUs sized to the cold plate share of the load
- Air-side cooling sized to the residual load, often using rear-door heat exchangers or contained in-row coolers
- Controls that coordinate both, so a failure on one side is detected and handled
Facility decisions to make early
Supply temperatures
Accelerator vendors publish liquid supply temperature limits. Some allow warm water, while others specify cooler supply. The answer determines whether the plant needs chillers or can rely mostly on dry coolers and economizers. Choose the plant temperature strategy to accommodate likely hardware generations over the facility's life, not just the first deployment.
Power and cooling go together
Power density and cooling density are the same number from the building's point of view: almost every watt delivered to the rack becomes heat. Electrical upgrades, busway capacity, UPS and generator sizing should be planned alongside the cooling design.
Structural loading
Liquid-cooled racks are heavy. Filled racks with manifolds and dense hardware can exceed the rated load of older raised floors. Check floor ratings and point loads early.
Transient behaviour
AI training jobs can ramp power quickly and synchronously across hundreds of racks. Cooling controls must respond to fast load changes without overshoot. Cold plates have little thermal mass, so a loss of coolant flow can cause throttling within seconds. Pumps on CDUs need UPS backing, and chilled water systems may need thermal storage for ride-through. See continuous cooling and thermal ride-through.
Water and leak management
Bringing water to each rack introduces leak risk. Mitigations include secondary loops with low volume, dripless connectors, leak detection with alarms and, where justified, automatic isolation valves.
Planning procedure
- Define the IT roadmap: hardware platforms, rack power, liquid share, supply temperature and flow requirements.
- Assess existing capacity: electrical, cooling plant, distribution, structural and space.
- Choose the cooling architecture for each zone: air, close-coupled air, liquid or hybrid.
- Size the residual air cooling, not just the liquid cooling.
- Set plant temperatures and heat rejection strategy, including free cooling hours.
- Design redundancy and ride-through for both loops.
- Model the room with CFD for the air side and hydraulic analysis for the liquid side.
- Commission under simulated load with load banks before production hardware arrives.
Energy and efficiency
Higher densities can actually improve efficiency. Liquid cooling allows warmer supply water, which increases economizer hours and lowers chiller lift. Fewer air handlers and lower fan energy reduce facility overhead. Removing server fans in immersion systems reduces IT power too, which slightly complicates PUE comparisons because the IT denominator changes. Interpreting PUE therefore needs care when comparing liquid-cooled and air-cooled facilities.
Frequently asked questions
Can I put AI racks in my existing room?
Usually in limited numbers, with targeted upgrades such as rear doors, in-row coolers or a dedicated liquid-cooled pod. Large clusters typically need new distribution, plant capacity and power.
Do I need chillers for AI cooling?
Not necessarily. If hardware supports warm supply water, dry coolers can carry most of the load in Canada. Chillers may still be needed for the residual air load, for hardware requiring cool water, or for peak summer conditions.
How much redundancy is needed?
That depends on business requirements. Training clusters may tolerate some interruption, while inference platforms serving customers may need concurrent maintainability. Set the requirement explicitly in the owner's project requirements.
Key takeaways
- AI and GPU racks commonly exceed what room air cooling can handle; liquid becomes necessary as density rises.
- Most deployments are hybrid, so the residual air-cooled load still needs careful design.
- Supply water temperature drives plant design and free cooling potential; plan for future hardware.
- Power, cooling, structure and ride-through must be planned together.
- Commission both air and liquid systems under realistic load before go-live.
Talk to an engineer
Have a project or a problem this guide touches on? Tell us about the building and we’ll be in touch within one hour.