Server racks are sagging under the weight of an imaginary force.
The exponential growth of artificial intelligence (AI) platforms is putting data center operations through their paces, particularly when it comes to cooling. AI applications, especially those involving machine learning or media generation, require big-brain processing power, leading to increased strain on data center infrastructure.
The world is relying heavily on AI and cloud storage to keep businesses operating smoothly. If anything goes wrong with electronic storage, it’s *gone*.
Fortunately, help is on the way. New cooling technology, methods, and products are in development to keep up with the worldwide sprawl of AI.
Now’s a pivotal time to review how AI data center cooling is reaching a breaking point … and the solutions coming our way.
AI may “talk” and act like it’s sentient, but that doesn’t mean it’s self-sufficient.
You, a human, are independently capable of waking up on a Wednesday morning, checking off mental tasks, and moving from room to room. Your warm-blooded nature generally keeps you at the right temperature, and in the dog days of summer, you can turn up the A/C or take a dip in a pool. And you can do all of those things without being a nuisance to the world.
AI, on the other hand? It’s more like an obnoxious, entitled, and needy neighbor (who also happens to be quite handy).
In 2022-23, data centers consumed 2% of the world’s total residential, commercial, and industrial energy. Experts believe that by 2030, the number will jump to 10%. Data center electricity demand will at least double from 2022 to 2026 alone, according to the International Energy Agency.
AI is almost entirely to blame.
In the history of electronics, there’s no precedent for AI’s energy appetite. As Time Magazine so eloquently put it:
“Generative AI does not simply float upon ephemeral intuition. Rather, it gathers strength via thousands of computers in data centers across the world, which operate constantly on full blast.”
Generative AI is the area of machine learning that allows users to create new images, text and so on. ChatGPT and other large language models (LLMs) use 10x more energy to answer a query than Google Search does.
To be able to “converse” with human users, LLMs must be able to effortlessly crunch vast amounts of data and perform complex calculations in seconds. Because of that, AI platforms use intense processing power in the form of:
At home, this would be like running a huge oven range, high-performance hair-dryer, and 3,000-watt space heater on the same circuit.
It’s not just the AI chipsets themselves that are straining data centers. Cooling systems were already one of the largest consumers of power in data centers. The addition of advanced chipsets is compounding the energy that cooling devices need to keep up with AI processing.
Why do server rooms need more cooling power than ever before?
You’ve probably already made the connection: Energy equals heat.
AI applications often ask data centers to maintain high-density server configurations to handle these extreme computational loads. A high-density server can maximize computing power and storage capacity in a small space, but this creates so much excess heat that traditional data center cooling systems can have trouble keeping up.
Of course, that’s not an option here. Excessive heat can lead to:
All of which can lead to catastrophic results. And none of which are acceptable in a world that now expects AI to process data and provide instant insight 24/7/365. Anyone involved in budgeting at a data center won’t appreciate having to replace these expensive processors, either.
A data center’s temperature should hover between 64-81°F to work at peak efficiency, according to ASHRAE guidelines. This range feels further out of reach with each passing generation of NVIDIA chip. The tech giant’s new Blackwell GPU runs at 1,200 W. That hair dryer in your bathroom? About the same.
Designing cooling architecture is a bit like a pyramid. You start at the top, identifying the most difficult area of the assembly to cool. That, without a doubt, is the AI chipset.
Data centers are key to the growth of U.S. tech infrastructure. Unfortunately, just like the growth of a city, server room growth comes with a bothersome byproduct: pollution.
Noise pollution is a real problem with computer room air conditioning – and not only inside the facility. Just ask citizens in northern Virginia, who have logged formal complaints comparing a nearby facility’s whirring fans to a leaf blower running 24/7.
As power density rises, cooling a data center requires more air handling units with more cooling potency. Inevitably, more air movers plus faster speeds equals a higher decibel level. Even a 2-minute drive away, this humming can be loud enough to upset a neighborhood.
ChatGPT and friends have forever changed how much cooling is needed for a server room. If data centers are struggling to keep up now, how will they cope with the next generation of AI chips?
Turning up the A/C won’t cut it. Not only does it make for an inefficient server cooling system, it also doesn’t address sustainability.
To soothe these sore spots, more data centers are turning to advanced cooling solutions:
The explosion of heat-intensive AI processes is a threat to space and efficiency issues inside server farms. In enterprise-level setups, some technicians are starting to use liquid coolant distribution units (CDUs). They’re not for all facility sizes, but liquid-based techniques are great at efficiently absorbing and dissipating heat in extremely dense networks.
Liquid cooling takes a highly targeted approach to carrying away thermal energy, sometimes in conjunction with fans. Liquid-based systems remove heat from components by circulating a specialized fluid close to or directly over them. The liquid is either deionized water, glycol-based fluid, or low-boiling-point dielectric fluid.
There are three main methods of liquid cooling for data centers:
Users in most markets prefer a door-mounted heat exchanger (which also includes a fan ~90% of the time), or a separate cabinet that sits next to the AI setup.
Liquid cooling isn’t a new discussion; however, it’s a much bigger one today thanks to AI and other high-performance computing tasks. Several companies are already commercializing liquid cooling solutions:
So why isn’t everyone sprinting to RFQ a liquid coolant system? There are a couple of main reasons:
Computing infrastructure may be getting denser, but that doesn’t mean designers and installers are getting more room to play around with.
Since electronic equipment is becoming smaller and sleeker, so must the devices used to cool them.
Luckily, there are a variety of high-performance fans that are small enough to fit into compact spaces and provide the advanced cooling technology necessary. (Consider a compact fan to be any product that’s 40-200 mm in diameter.)
|
Fan 🌀 Cooling |
Liquid 💧 Cooling |
|
|
Cost |
▪️Low initial cost |
▪️Higher initial cost |
|
Power |
▪️Many types & performance levels available |
▪️Can manage intense heat loads from CPUs & GPUs |
|
Size |
▪️Low profile suitable for small spaces |
▪️May not fit in smaller systems |
|
Usability |
▪️Simple design |
▪️Complex setup with pumps & reservoirs |
|
Reliability |
▪️Fewer failure points for leaks |
▪️Potential for pump failure, leaks |
A wide range of fan concepts – many of them available today – are working to address AI data center demand:
High static pressure is a major indicator of resistance against airflow in a cooling system. This resistance can come from dense server racks, long ductwork, or clogged filters.
Fans with steep power curves can efficiently push air through rack spaces with low-to-medium component density.
In data center airflow management, all cooling products should add to your solution, not to your problems.
New fans are coming out with clever aerodynamics and motor technology, minimizing the need for energy. In racks with low-to-medium congestion, these advanced designs are cooling effectively while remaining sustainable and relatively quiet.
Example: The AxiEco series is ideal for applications with high back pressure. It’s a great candidate for door-mounted heat exchangers, cabinet cooling, and in-row cooling. (More on that last one soon.)
When you need a high-force fan capable of cooling dense server racks, it’s hard to beat the power and airflow of a counterrotating fan.
Just one problem: They’re horrendous to the human ear.
Optimizing airflow with an alternative design or newer technology can go a long way toward lowering data center decibels. The latest diagonal compact fans can accomplish that and more, offering a way to futureproof a heat-management system without resorting to liquid cooling.
DiaForce combines the high airflow of axial fans (preferred for easy installation) with the high back pressure of centrifugal fans (to push through racks of dense components). The result outperforms counterrotating fans with significant energy savings, noise reduction, and psychoacoustic tonality improvements.
Smartphones. Smart TVs. Smart refrigerators. Smart fans?
Electric fans haven’t evolved a ton since their 1882 invention, but they’ve picked up the pace lately – sometimes literally – with smart technology.
These advanced commercial fans allow you to plan ahead for maintenance and avoid costly downtime. The benefits tie together nicely:
Today’s data center fans outpace the old misconceptions that fans: A) are dirty, and B) break too easily. Choosing an enclosure fan with smart technology allows your system to adapt in the moment, adding durability to key areas of the data center.
Want to knock out a server rack’s thermal issues in one shot?
Self-cooling enclosures do just what their name says. They combine a cooling system with a sealed unit to regulate its temperature. Rather than targeting specific parts of an assembly, cooling enclosures control the environment for the entire device.
Common features include:
Self-cooling enclosures, together with high-performance fans, can serve as a complete solution that’s easy to set up and modify within your existing infrastructure. It’s ideal for situations where traditional data center cooling is inadequate or impractical, but that’s not all. Self-cooling enclosures are also a solution for environments where liquid cooling might be too complex or risky, such as in remote or small-scale installations.
When heat generation in a server center is fairly consistent from rack to rack, it’s generally safe to rely on room-wide cooling. In AI data center design, heavy-duty chips need attention that’s a little more personal.
Targeted cooling is the use of anti-heat measures specifically in areas with the highest thermal loads, such as AI processing clusters. This approach prevents hotspots and ensures that all equipment operates within safe temperature thresholds.
In-row cooling, for example, is a method in which the technician puts fan units or CDUs directly among or adjacent to the server racks. This reduces how far the air or coolant must travel to the servers, minimizing the mixing of hot and cold air. As the server area grows, it’s easy to add more in-row units without overhauling the entire system.
If your data center handles a mix of high- and lower-intensity processing, localized cooling will probably be more efficient than room-based systems.
Containment systems are even more relentless in physically separating hot exhaust air from cold supply air.
Their goal is to provide a stable, controlled airflow to high-density IT equipment, while also returning dry and warm air to the cooling unit.
There are two main types of data center containment:
Both methods help maintain more consistent temperatures and increase cooling capacity. In an AI server rack, containment systems can be a cost-effective way to maximize your existing cooling setup.
Are your electronic components adding up? It’s time for your cooling capabilities to level up, too.
Today, heat flux is orders of magnitude beyond the data centers of last generation. As AI applications drive the need for more powerful and densely packed servers, the demand for energy-efficient cooling solutions has gone from “add-on” to “absolute necessity.”
Efficient server rack cooling fans and techniques not only save energy; they also support uptime in big-data analytics operations. Where traditional cooling methods fail, the new solutions listed above are picking up the slack. Maybe AI isn’t so threatening after all.
For more tips on meeting requirements for server rooms and other projects, grab our free guide to electronic parts sourcing:
(This article is in collaboration with ebm-papst, the world’s top maker of high-performance fans for data centers. The company also serves commercial refrigeration, medical technology, industrial, HVAC, auto, and other markets. Energy efficiency, service life, noise minimization, and sustainability are at the heart of ebm-papst’s innovations.)