arXiv:2608.11226 Reports RL Agents Cutting Measured LLM Training Power from Single GPU to Fleet
arXiv:2608.11226 applies reinforcement learning to measured power control during LLM training. The work spans single-GPU experiments to fleet deployments and reports energy reductions without throughput loss. It fills a gap between simulation studies and production schedulers.
The paper deploys RL policies that modulate GPU power caps and batch sizing in real time during training. Experiments tracked watt-hour consumption on production hardware rather than simulation. Results showed consistent energy savings while maintaining convergence curves within 1 percent of baseline throughput.
Prior work on data-center power relied on static heuristics or reactive DVFS. This study adds online learning that adapts to workload phase changes across model scales. It connects to earlier measured traces from Microsoft and Meta clusters showing 30-40 percent headroom in average utilization during training. The RL approach targets exactly those idle intervals without requiring hardware changes.
Operational impact centers on integration with existing job schedulers. Policies must respect thermal limits and job SLAs already enforced by Kubernetes or Slurm. Next steps include closed-loop deployment on clusters exceeding 10,000 GPUs and direct comparison against Oracle or Google published energy traces.
Validation requires public release of the reward function and state features so independent teams can reproduce fleet-level gains.
Meta: Production RL power agents control >15 percent of training GPU-hours by Q4 2025.
Sources (2)
- [1]Primary Source(https://arxiv.org/abs/2608.11226)
- [2]Supporting Source(https://research.google/pubs/pub49953/)