Platforms_
H200 Deployment Guide: What's Different from H100 on the Floor
Specifies the power and thermal deltas between H200 and H100 SXM GPUs that change rack power budgeting, liquid cooling setup, and commissioning steps for deployment crews.
Key facts
- H200 and H100 SXM modules share the same TDP rating in NVIDIA specifications.
- H200 increases HBM capacity to 141 GB HBM3e, raising memory subsystem heat relative to the 80 GB HBM3 in H100.
- Cold-plate designs qualified for H100 require re-validation of contact pressure on the taller HBM stack of H200.
- Rack-level power feeds sized for H100 populations remain adequate for H200 at identical TDP but need updated breaker coordination studies.
- ASHRAE TC 9.9 Class A2 inlet air limits continue to apply, yet H200 memory thermals tighten the allowable approach temperature at the cold plate.
- Factory-terminated MPO trunks carry only the scale-out fabric; NVLink remains copper inside the rack for both GPU generations.
- Leviathan Systems field teams record H200 rack bring-up by confirming GPU power capping before full liquid loop flow is enabled.
Power delivery changes at the busbar and PDU
H200 modules draw the same maximum power as H100 modules, so existing busbar ampacity and PDU outlet ratings sized for H100 racks do not require upsizing. The memory subsystem, however, sustains higher average power during large-context inference, shifting the load profile toward continuous draw rather than short bursts.
Crews must rerun the rack-level breaker coordination study with the new load shape before energizing. Overcurrent settings that tolerated H100 spikes can nuisance-trip on H200 if the time-current curve is not adjusted for the longer high-current intervals.
Cold-plate contact and memory stack height
The H200 HBM3e stack is taller than the H100 HBM3 stack, changing the required deflection of the cold-plate TIM. Plates already installed for H100 must be checked with a feeler gauge at each memory site; insufficient compression leaves a gap that raises junction temperature even when GPU core temperatures remain in spec.
Leviathan Systems crews replace the TIM and re-torque the retention hardware to the OEM cold-plate specification whenever a rack is converted from H100 to H200. Reusing the original torque value without measuring stack height is the most common source of post-commissioning thermal alarms.
Liquid loop flow and temperature setpoints
Facility supply temperature setpoints calibrated for H100 remain acceptable for H200, but the secondary loop flow rate must be verified at each manifold. Higher memory power increases the heat load per node, so flow that was marginal on H100 can produce unacceptable approach temperatures on H200.
Operators increase secondary pump VFD speed in increments while monitoring GPU memory temperature sensors until the hottest site stays below the NVIDIA throttling threshold. This step is performed before any workload is started.
Rack power budgeting and circuit coordination
Nameplate totals for an H200 rack are numerically identical to an H100 rack at the same node count. The difference appears in the diversity factor applied during the coordination study; sustained memory traffic reduces the allowable diversity, requiring the upstream breaker settings to be reviewed in some switchgear configurations.
Field teams therefore request the updated one-line diagram from the electrical engineer before breaker adjustments are made. Skipping this step has caused arc-flash label changes and delayed energization on multiple projects.
Common failure modes observed in the field
The dominant failure is cold-plate lift-off at the memory sites after the rack is rolled into the row and the quick-connects are mated. Vibration during transport relaxes the retention screws enough to open a gap that only appears under load.
A second recurring issue is power capping left at the H100 default value. The GPU accepts the cap, but memory bandwidth throttling occurs earlier than expected, producing silent performance loss that is discovered only during acceptance benchmarking. Both conditions are caught by running the OEM memory-bandwidth diagnostic immediately after the first liquid loop stabilization.
Commissioning sequence specific to H200
After mechanical installation, apply facility power with GPUs held at minimum power state. Confirm all cold-plate inlet and outlet temperatures are stable before raising the power cap in increments.
At each step, log GPU memory temperature and secondary loop delta-T. Only when the full TDP cap produces stable readings within OEM limits is the rack released for workload. This staged approach prevents thermal runaway that occurs when full power is applied before flow is verified.
Standards referenced: ASHRAE TC 9.9 · NVIDIA SXM GPU mechanical and thermal specifications · TIA-942
Frequently asked_
Can the same cold plates used for H100 be reused on H200?
Only after measuring stack height and replacing the TIM. The taller HBM3e package changes the required compression. Reusing the original plates without inspection produces memory hotspots that appear after burn-in. Always follow the OEM torque sequence after any height measurement.
Does the rack PDU need to be changed when swapping H100 for H200?
No change to PDU hardware is required at the same node count because TDP remains the same. The coordination study must still be updated to reflect the altered load duration curve. This prevents nuisance trips during sustained memory-heavy workloads. Electrical engineers should review the one-line before any settings are touched.
What is the first thermal check performed on an H200 rack?
Secondary loop flow verification at each manifold while the GPUs are held at minimum power. Flow that was adequate for H100 can be marginal once memory power rises. Measure approach temperature at the cold plate before any power ramp. This catches restrictions that only show under the new heat profile.
How does NVLink connectivity differ between the two platforms?
It does not. Both generations route NVLink over the internal copper spine. Fiber and MPO trunks remain dedicated to the scale-out InfiniBand or Ethernet fabric. Confirm connectivity with nvidia-smi only after the copper links are seated. Never assume an MPO trunk carries NVLink traffic.
When should power capping be raised to the full TDP rating?
Only after confirming stable cold-plate approach temperatures at each incremental power step. Applying the full cap before flow verification is the most frequent cause of early throttling during commissioning. Log memory temperatures at every stage. Release the rack only when all sensors remain within OEM limits under sustained load.