How to determine when to embrace liquid cooling in data centers with different scales?

         One of the most notable events in the tech industry this year is the launch of the world's most powerful supercomputing center, "Supercluster," by Musk's company.It is worth mentioning that behind this world's newly established largest supercomputing center, there is still the presence of Dell Technologies: the supplier providing servers for the "Supercluster" is Dell Technologies

This cluster consists of 100000 liquid cooled GPUs running on a single Remote Direct Data Access (RDMA) architecture. In terms of GPU scale, this "supercomputing factory" has surpassed any supercomputer on the Top500 list.

The computing power scale of "Supercluster" is indeed impressive, but in the current era where liquid cooling technology is still developing and expensive, why should Supercluster fully invest in liquid cooling?

On the one hand, it is due to considerations of energy consumption and cost. It is understood that each GPU in the cluster consumes at least 700 watts of power, which means that powering only the GPU requires over 70 megawatts of power, not including the power consumption of other servers, networks, and cooling equipment.

The use of liquid cooling technology can significantly reduce energy consumption. According to the latest research from the National Renewable Energy Laboratory (NREL) in the United States, liquid cooling technology can achieve PUE values as low as 1.05 (power utilization efficiency), which is much higher than the typical 1.5 to 2.0 PUE values of air cooling systems, with an overall energy savings of about 40%.

 On the other hand, the adoption of liquid cooling technology in ultra large scale data centers is also a trend.

With the increase in energy consumption in data centers, energy efficiency has become crucial. In large data centers, the power density of racks generally exceeds 30 kilowatts, and the increasing thermal design power (TDP) of modern CPUs and GPUs further increases the burden on cooling systems. In this context, the advantages of liquid cooled servers become even more apparent.

So, the question is: Since liquid cooled servers are so popular, does this mean that future data centers will all turn to liquid cooling technology?

       Although liquid cooling technology is becoming a key trend in the development of data centers, it does not mean that all enterprises should immediately adopt liquid cooling solutions. At present, air cooling technology will continue to be widely used in the foreseeable future due to its convenience and cost-effectiveness, and industry practice has shown that there is a critical point in the transition from air cooling to liquid cooling.

        According to Schneider Electric's cost research, for a 10kW rack, the capital expenditure of immersion liquid cooling technology based on chassis is comparable to traditional hot aisle enclosed rack air cooling. When the power density of the rack exceeds 15-20kW, the advantages of liquid cooled servers begin to emerge, and liquid cooling technology is more suitable at this time.

       Therefore, for newly built ultra large data centers, fully adopting liquid cooled servers is undoubtedly a wise choice; For most enterprises, the scale and power density of their data centers have not yet reached the level where they must fully adopt liquid cooling technology. Whether from the perspective of business needs or cost investment, it is not realistic to switch to liquid cooling technology in a short period of time.

        Therefore, in this situation, many data centers choose to adopt a gradual transition strategy to adapt to the needs of business development and cost control.

        For example, liquid air cooling (LAC) technology uses air as the heat transfer medium, without the need to directly connect water pipes to the data center, avoiding the need for overall renovation of the data center. This method not only provides operational flexibility but also reduces costs, allowing the data center to flexibly adjust its cooling strategy according to changes in demand.

Another hybrid solution is to use a rear door heat exchanger (RDHx) to retrofit the air cooling device. It transforms the traditional air cooling system by concentrating the fans inside the server chassis at the rear door of the cabinet, thereby discharging the hot air flowing through the server. Enterprises can continue to use the air cooling system while supplementing liquid cooling through RDHx to improve the upper limit of cooling efficiency and effectiveness.

       We can see that switching from air cooling to liquid cooling is not necessarily an either or measure, the key is how to determine the cooling technology that is suitable for the development of one's own enterprise.

                                                                                   ------------ Article excerpt from Dell WeChat official account