Power Requirements for GPU Racks: From 10kW to 120kW and Beyond

TL;DR

Every generation of NVIDIA GPUs has increased power consumption. The GB300 draws approximately 1,200W per GPU. A single GB300 NVL72 rack draws 120kW — the equivalent of 80 American homes. Power delivery is now the primary constraint on GPU data center deployment.

The Power Problem

Every generation of NVIDIA GPUs has increased power consumption. The A100 drew 400W. The H100 draws 700W. The GB300 draws approximately 1,200W per GPU module. A single GB300 NVL72 rack containing 72 GPUs draws 120kW — the equivalent of 80 American homes.

Power delivery is now the primary constraint on GPU data center deployment. It determines how many racks a facility can support, how quickly a deployment can scale, and in many cases, whether a project is feasible at a given site. Understanding power requirements at the rack level, row level, and facility level is essential for anyone planning a GPU deployment.

Power Profiles by GPU Platform

Standard GPU Servers (H100, H200): 10-40kW per Rack

A single NVIDIA DGX H100 or HGX H100 server draws approximately 10.2kW at full load. In a standard 42U rack, you can install 4-5 of these 4U servers (allowing space for top-of-rack switches and cable management), bringing the total rack power to 40-50kW.

Many deployments use lower-density configurations with 2-3 GPU servers per rack supplemented by storage or CPU compute, bringing the rack total to 20-30kW. This density range is manageable with traditional power distribution: two redundant PDUs (A-feed and B-feed), each rated for the full rack load, connected to separate utility feeds.

At this density, standard 30A or 60A single-phase or three-phase power whips are sufficient. The PDUs distribute power to individual server power supplies through C13/C14 or C19/C20 connectors. This is conventional data center power infrastructure that any facility built in the last 20 years can support.

Transition Density (GH200, Dense HGX): 40-60kW per Rack

Grace Hopper Superchip configurations and densely packed HGX systems push rack power into the 40-60kW range. This density exceeds the capacity of many existing data center power distributions and requires careful planning.

At 60kW per rack, a single row of 10 racks draws 600kW — more than many smaller data centers allocate to an entire room. The facility electrical distribution must have sufficient capacity at the bus duct or power panel serving that row, and the branch circuit wiring must be rated for the continuous load.

PDU selection at this density requires careful attention to phase balancing. Three-phase PDUs are standard, and the load must be balanced across all three phases to avoid overloading any single phase conductor. Monitored PDUs that report per-phase current in real time are essential for maintaining balance as servers are added or workloads change.

Rack-Scale GPU Systems (GB200, GB300): 100-120kW per Rack

The GB200 NVL72 and GB300 NVL72 operate at approximately 120kW per rack. This density breaks conventional power distribution architecture and requires a fundamentally different approach.

At 120kW, the rack draws approximately 175 amps at 480V three-phase (or proportionally more at lower voltages). This exceeds the capacity of any single PDU commonly used in data center environments. Power delivery for these racks typically uses busway distribution systems — overhead or under-floor power rails that provide high-current tap-off points at each rack position.

Busway systems from vendors like Starline, Legrand, and Eaton provide 400A to 800A per run, with tap-off boxes at each rack position that can deliver 100-200A per rack. The busway connects to the facility transformer room through high-current feeders, with automatic transfer switches for redundancy.

The GB200 and GB300 racks also require power for the liquid cooling infrastructure (CDU pumps, control electronics, leak detection systems), which adds 5-15kW to the total power demand per rack depending on the CDU configuration.

Facility-Level Power Planning

Total IT Load Calculation

The total IT power load for a GPU deployment is calculated by multiplying the per-rack power by the number of racks, then adding overhead for network infrastructure (spine and aggregation switches, storage systems) and cooling infrastructure power. A common mistake is calculating IT load based on GPU nameplate power alone, ignoring the CPUs, memory, storage, and network components in each server that collectively add 20-30% to the GPU power draw.

For a 100-rack GB300 NVL72 deployment: IT load is approximately 12MW (100 racks × 120kW), cooling infrastructure adds approximately 3-4MW (assuming PUE of 1.25-1.3 for liquid-cooled facilities), network and support infrastructure adds approximately 0.5-1MW. Total facility power requirement: approximately 15-17MW.

Power Usage Effectiveness (PUE)

PUE measures the ratio of total facility power to IT equipment power. A PUE of 1.3 means the facility consumes 30% more power than the IT equipment alone, with the overhead going to cooling, lighting, and building systems.

Liquid-cooled GPU deployments achieve lower PUE than air-cooled deployments because the cooling system is more energy-efficient. Direct-to-chip liquid cooling eliminates server fans (saving 10-15% of IT power), enables higher cooling water temperatures (reducing chiller energy), and allows waste heat recovery in some configurations.

Target PUE for a new liquid-cooled GPU facility is 1.2-1.3. Air-cooled GPU facilities typically achieve 1.4-1.6 PUE, with the difference representing significant additional energy cost over the facility's operational life.

Redundancy

Power redundancy for GPU deployments follows standard data center tier classifications, but the financial impact of downtime is amplified. A single hour of unplanned downtime on a 1,000-GPU training cluster can waste hundreds of thousands of dollars in GPU-hours and set back training progress by days (since checkpointing overhead means the lost work exceeds the actual downtime duration).

Tier III facilities provide N+1 redundancy with concurrent maintainability — any single power component can be taken offline for maintenance without affecting IT load. This is the minimum acceptable redundancy level for GPU deployments.

Tier IV facilities provide 2N redundancy with fault tolerance — any single failure anywhere in the power path does not affect IT load. Tier IV is preferred for GPU deployments where training jobs run for weeks or months and any interruption is extremely costly.

UPS systems for GPU deployments must be sized for the high inrush current that occurs when GPU racks power on after an outage. GPU power supplies have significant capacitive loads that draw 2-3x steady-state current during startup. UPS systems not sized for this inrush will trip during the recovery sequence, causing a cascading failure.

Generator Backup

Diesel generators provide extended runtime backup when utility power fails and the UPS battery is depleted. Generator capacity must match the full facility load including cooling infrastructure. A common failure mode is generators sized for the IT load alone that cannot support the cooling system — resulting in GPU thermal shutdowns even though electrical power is available.

Generator startup time (typically 10-15 seconds for modern units) must be shorter than the UPS battery runtime at full load. For GPU deployments with large UPS battery banks, this is typically not a concern. However, for deployments that use flywheel-based UPS or small battery banks, the generator transfer window is tight and must be validated through live testing.

Power Distribution Design

High-Voltage Distribution

Modern GPU data centers distribute power at the highest practical voltage to minimize current and associated I²R losses. For North American facilities, 480V three-phase distribution from the utility transformer to the rack is standard. Some facilities use 415V three-phase (common in European-designed facilities operating in the US).

Higher distribution voltages reduce cable sizes, lower distribution losses, and enable more efficient power supplies. GPU server power supplies are increasingly designed for 277V single-phase input (one leg of 480V three-phase), which eliminates the need for step-down transformers at the rack level.

Busway vs. Cable-Based Distribution

For rack densities above 40kW, busway distribution provides significant advantages over traditional cable-based distribution. Busway systems are modular (tap-off boxes can be added or relocated), provide higher current capacity in less space, and are faster to install than pulling individual power cables to each rack position.

The initial cost of busway is higher than cable-based distribution for small deployments, but the total cost of ownership favors busway at scale due to lower installation labor, easier reconfiguration, and reduced floor space consumption.

Monitoring and Management

Every GPU rack must have real-time power monitoring at the PDU or busway tap-off level. Monitoring data includes per-phase voltage, per-phase current, power factor, total power consumption, and energy consumption over time.

This data feeds into the facility's Data Center Infrastructure Management (DCIM) system and is used for capacity planning (identifying when a row or room is approaching its power limit), load balancing (redistributing workloads to avoid overloaded circuits), billing (for colocation environments), and anomaly detection (identifying failing power supplies or unbalanced loads before they cause outages).

Power Delivery Services

Leviathan Systems integrates power distribution as part of every GPU rack deployment. Our scope includes verifying facility power capacity against deployment requirements, connecting rack power feeds to busway or PDU infrastructure, verifying phase rotation, voltage levels, and circuit protection, staged power-on with current monitoring at each stage, and documenting power measurements as part of the commissioning package.

We coordinate with facility electrical engineers and building management teams to ensure that GPU rack power requirements are understood and accommodated before any equipment arrives on site.