The law of physics states that what rises must fall, and the same applies to the temperatures in data centers. As computer densities in racks and cabinets continue to increase, so do the temperatures.
This is due to the new generations of CPUs and GPUs producing higher thermal power densities. With numerous active components in a rack, traditional cooling methods such as in-room, in-row, or rack-based containment systems may no longer be effective as the rack densities surpass 20kW.
To maintain the temperatures, data centers need cooling systems that can work around the clock, every day of the year. Air-cooled systems have reached their limits, and liquid cooling technology is becoming the norm to tackle high-density racks that generate up to 30kW or more of power.
When choosing a cooling system, there are a few factors to consider, including the size of the computer room, the maximum kW load per cabinet, the number of cooling units, the room’s location, ceiling and access floor heights, and future expansion needs.
There’s no one-size-fits-all solution for data center cooling, but liquid cooling systems are gaining popularity.
To better understand why let’s take a look at the different systems available.
Air Cooling
Managing air is a key aspect in the thermal management of air cooling systems. Older air cooling systems like CRACs or CRAHs were designed to cool the entire room, with a capacity to cool each rack to 15kW. Air distribution includes options such as underfloor supply, open room returns, supply and returns, and vertical heat collars directing hot air to plenum return areas.
In-row air cooling systems are positioned close to servers and racks and implement hot/cold aisle containment techniques. Cool air is pushed in at the front and hot air is expelled out the back of the racks and cabinets. By being close to the source of heat, the hot air return path to the CRAC or CRAH is reduced, reducing the possibility of mixing hot and cold air streams.
Rack-based systems have cooling units dedicated to specific racks, which can be mounted directly onto or within the IT racks. With even shorter airflow paths than in-row-based systems, airflow is unaffected by objects within the room or room constraints like doorways. However, rack-based cooling can be costly as it requires more cooling units and, therefore, more power consumption.
Liquid Cooling
Liquid cooling is a popular method for handling high-density facilities as it has a much greater heat capacity than air per volume. Other advantages include lower energy consumption, reduced operational expenses, making less noise compared to air cooling systems, and taking up less space in the data center.
Several liquid cooling technologies have been developed to cater to the cooling needs of high-density racks and cabinets. These include direct-to-chip cooling, rack/server cooling, and immersion cooling. These techniques work by pumping cold liquid close to or directly onto hardware to perform the heat exchange, and the fluid constantly circulates to remove heat as fast as it’s generated.
Direct-to-chip cooling
Direct-to-chip cooling involves bringing a liquid coolant through tubes directly to electronics, such as CPUs or GPUs, where heat is absorbed. The fluid doesn’t make direct contact with electronics but absorbs the heat by converting it into vapor, which then carries the heat out of the IT equipment through evaporation.
Rack liquid cooling
With rack liquid cooling, the rear door of the rack is replaced with a liquid heat exchanger. Passive designs feature fans that blow heated air through a liquid-filled coil mounted in place of the rear door of the rack. The coil absorbs heat before the air is released into the data center. Active heat exchangers, on the other hand, come with fans that pull air through the coils and remove heat from high-density racks.
Immersion cooling
Immersion cooling, both single-phase and two-phase, is becoming increasingly popular in the data center space. This method involves submerging hardware, such as servers or processors, in special dielectric liquids that conduct heat but not electricity. In single-phase immersion cooling, electronic components are immersed in dielectric liquid in a sealed but accessible enclosure.
Pumps are often used to move the heated fluid to a heat exchanger where it’s cooled and cycled back into the enclosure. In two-phase immersion cooling, fluid is boiled and condensed, which exponentially increases heat transfer efficiency. Components are directly immersed in dielectric liquid in a sealed enclosure, producing a vapor that rises from the liquid. The vapor condenses on a heat exchanger within the tank, returning the fluid to its cooling point.
Choosing the Right Way to Cool
When it comes to cooling your data center, you have the option of using liquid cooling or sticking with traditional air cooling systems. While liquid cooling offers improved cooling capacity, it also comes with a higher initial cost for hardware, specialized liquids, safety equipment, and the training required to effectively use these systems. As next-gen processors and GPUs continue to advance, integrating them into existing air-cooled servers may pose a challenge. IT managers and data center designers will have to weigh the upfront costs of liquid cooling against the long-term operational savings it offers to determine if it’s the right choice for their needs.
Last Updated on June 8, 2023 by Josh Mahan
