Original title: ETCInfer: An Energy-efficient Thermal-aware Cooling-joint Scheduler for LLM Inference in AI Datacenters
Rui Lu, Rui Ge, Huanghuang Liang, Xiaobo Zhou +1 · September 2026
Running large language models uses huge amounts of power in datacenters, split between the graphics processing units that run the code and the cooling systems that chill the room. Setting the cooling air warmer cuts cooling energy, but hot rooms make chips overheat, slow down, and miss their latency targets. To solve this, researchers built ETCInfer. It chooses a temperature setpoint for the air conditioning before a job starts. Then, while the job runs, it constantly adjusts the speed of each chip and the size of the micro-batches it processes. It uses calibrated math models of heat, power, and latency to guess hidden temperatures and time-to-throttle before making changes. In testing, this system cut total job energy by up to 33.1%, reduced thermal throttling time by up to 92.9%, and kept target deadline misses below 0.7%, even at ambient temperatures as high as 48^{\circ}\mathrm{C}.
Imagine cooking a large meal in a small kitchen. Normally you blast the air conditioner to keep the room cool, but it costs a lot. If you turn the AC down a bit, the kitchen gets hotter—and you might have to cook slower to avoid burning the food. This paper finds the sweet spot: set the AC just right and adjust your cooking speed so the meal finishes on time without overheating. Here, the kitchen is the datacenter, the AC is the CRAC, and the cooking speed is the GPU frequency and batch size.
AI datacenters burn enormous energy running both GPUs and cooling systems. This scheduler coordinates them to save electricity and reduce carbon emissions without breaking service guarantees. If adopted widely, it could lower the cost and environmental impact of every ChatGPT query or AI image generation.
The abstract does not specify how the scheduler performs under workload spikes or across different hardware configurations. It also does not address the cost or complexity of deploying the coordination layer in existing datacenters, or whether the model generalizes to non-LLM inference workloads.
Ask anything about this paper. Answers come from its abstract, and say so when the abstract doesn't cover your question.
ETCInfer: An Energy-efficient Thermal-aware Cooling-joint Scheduler for LLM Inference in AI Datacenters — Rui Lu, Rui Ge, Huanghuang Liang et al.
Abstract licence: CC0
Explanation generated by Synthesis from the abstract — it is not the original paper. This is a preprint, so it hasn't necessarily been peer reviewed.