Platforms_
B200 / HGX B200 Deployment Guide: Power, Thermal, and Rack Demands
Details the rack, power distribution, and liquid-cooling prerequisites that must be verified before an HGX B200 system leaves the integration floor, including separation of NVLink copper from scale-out fiber and the validation sequence used by field crews.
Key facts
- HGX B200 systems place all GPU-to-GPU NVLink traffic on the copper backplane inside the rack; MPO trunks carry only InfiniBand or Ethernet scale-out traffic between switches.
- Rack power feeds for these platforms must support the full OEM-specified draw through redundant A+B paths with monitored branch circuits sized to the PDU rating plate.
- Liquid-cooling loops require CDU-to-rack manifolds with quick-disconnect fittings matched to the HGX cold-plate layout and secondary containment per facility design.
- TIA-942-A and ASHRAE TC 9.9 define the environmental envelopes and redundancy tiers that govern rack placement and cooling validation before shipment.
- Factory-terminated MPO trunks are inspected with an MPO continuity tester and cleaned with the OEM one-click tool; field polishing of MPO ferrules is not performed.
- An OTDR or calibrated MPO tester is used to confirm insertion loss and polarity on every scale-out link before the rack ships.
- Leviathan Systems performs the final mechanical torque checks, leak tests, and power-on self-test sequence on the assembled rack prior to customer handover.
Rack structural and mechanical interfaces
HGX B200 trays mount on standard EIA-310 rails but require additional rear support brackets to carry the weight of the liquid-cooling cold plates and the copper NVLink spine. Crews verify rail depth and U-spacing against the OEM mechanical drawing before any tray is inserted.
The rack must also provide unobstructed airflow paths above and below the HGX chassis for the residual air-cooled components; blocking these paths forces the cold-plate loop to handle more heat than designed. Levelling feet are adjusted so the rack stands plumb within the tolerance required for manifold alignment.
Power distribution and branch-circuit sizing
Power enters the rack through two independent PDUs fed from separate UPS/PDU sources. Each PDU must be rated for the full continuous load of the HGX B200 configuration plus headroom for inrush at power-up; branch-circuit breakers are selected to trip only on genuine faults, not on the GPU power ramp.
Before the rack ships, crews measure voltage at the PDU input under simulated load and confirm that phase balance stays within the OEM tolerance. Any deviation triggers re-termination of the whip or adjustment of the upstream panel before the system is released.
Liquid-cooling manifold and CDU integration
The secondary cooling loop connects to the rack via quick-disconnect supply and return manifolds sized to the flow rate specified by the HGX cold-plate design. Manifolds are pressure-tested to the facility hydrostatic requirement and checked for proper flow direction before any GPU tray is installed.
A facility CDU must be online and circulating fluid at the target supply temperature before the rack is powered; crews record inlet and outlet temperatures at each quick-disconnect during the final commissioning run to confirm the loop meets the thermal budget.
Network fabric separation and cabling
All NVLink links remain on the copper backplane inside the rack; no fiber is used for GPU-to-GPU traffic. Scale-out connectivity uses MPO-terminated trunks that exit the rack to leaf or spine switches running InfiniBand or Ethernet.
Trunks are routed with minimum bend radius maintained and secured so they cannot be pinched when side panels are installed. Polarity is verified with a continuity tester before any transceiver is seated.
Pre-shipment validation sequence
The sequence begins with a full mechanical torque audit of all power and manifold connections, followed by a low-pressure helium leak test on the liquid-cooling loop. Once the loop holds, power is applied and each GPU tray is brought to idle while nvidia-smi reports NVLink status and temperatures.
Scale-out links are then validated with an OTDR or MPO tester; any port showing loss above the link budget is re-cleaned or replaced. Only after all checks pass is the rack released for shipment.
Common field failure modes and detection points
The most frequent issue is mismatched quick-disconnect gender or orientation on the cooling manifold, which prevents flow and triggers thermal shutdown within minutes of load. Crews catch this by performing a visual keying check and a brief flow test with the CDU before trays are powered.
Another recurring failure is phase imbalance on the A+B feeds that only appears under full GPU load; this is detected by monitoring PDU phase currents during the final burn-in rather than at idle. A third mode is MPO polarity reversal on scale-out trunks that passes continuity but fails link training; an end-to-end OTDR or traffic test before shipment eliminates the problem.
Standards referenced: TIA-942-A · ASHRAE TC 9.9 · EIA-310
Frequently asked_
Can NVLink traffic be carried on the MPO trunks that leave the rack?
No. NVLink between GPUs stays on the copper backplane inside the HGX B200 rack. MPO trunks are reserved exclusively for scale-out InfiniBand or Ethernet traffic between racks and switches. Attempting to use fiber for NVLink will fail link training and is outside the supported topology.
What flow and temperature must the CDU deliver before the rack ships?
The CDU must be circulating fluid at the supply temperature and flow rate listed in the HGX B200 thermal specification. Crews record inlet and outlet temperatures at each manifold quick-disconnect during the final run to confirm the loop meets the design point before the system leaves the floor.
How are power phases verified under load?
Voltage and current are measured at the PDU inputs while the GPUs are ramped to full power. Any phase deviation outside the OEM tolerance requires correction of the upstream termination before the rack is accepted for shipment.
Who performs the final leak and power-on checks?
Leviathan Systems integration crews complete the helium leak test, manifold flow verification, and power-on sequence with nvidia-smi validation. Only racks that pass every step receive the pre-shipment sign-off.
What test confirms MPO polarity before shipment?
A calibrated MPO continuity tester is used on every trunk. The test verifies both continuity and correct polarity mapping; any reversal is corrected before the rack ships so that link training succeeds on first power-up at the customer site.