Platforms_
H100 / HGX H100 Deployment Checklist: Power, Cooling, and Cabling Demands
This checklist specifies the exact physical-layer sequence for power distribution, cooling loop integration, copper NVLink backplane seating, and MPO-based scale-out cabling when deploying HGX H100 racks in either air-cooled or direct-liquid-cooled form factors.
Key facts
- HGX H100 trays draw power through dedicated 12 V and 48 V rails supplied by rack-level PSUs; each tray must be verified for proper sequencing before full population.
- Inside NVL72-class racks, all GPU-to-GPU NVLink traffic travels exclusively over the copper backplane or spine; fiber or MPO links carry only the scale-out InfiniBand or Ethernet fabric.
- MPO trunk cables used for scale-out networking are factory-terminated; field crews perform only routing, cleaning, inspection, and patching.
- Liquid-cooled HGX H100 manifolds require leak testing at the OEM-specified pressure before any power is applied to the GPUs.
- Air-cooled HGX H100 deployments must maintain front-to-rear airflow paths that satisfy the rack's total CFM requirement without recirculation.
- Structured cabling for GPU racks follows TIA-942 pathways with minimum bend radii dictated by the cable construction and the relevant MPO connector standard.
- Validation of every MPO link uses a calibrated continuity tester followed by end-face inspection with a fiberscope meeting IEC 61300-3-35 criteria.
Power rail verification and PDU sizing for HGX H100 racks
Confirm each rack PDU supplies the required 12 V and 48 V branches to the HGX backplane before trays are installed. Measure voltage at the tray connectors under no-load conditions and again after the first tray is seated to verify droop stays within the OEM tolerance. Sequence power-up so that the baseboard management controller boots and reports rail status before the GPUs receive full power.
Document the total rack draw against the upstream breaker and UPS capacity. HGX H100 systems concentrate high current on the 48 V rail, so crews must confirm each PDU phase is balanced before proceeding to cooling integration. Any imbalance requires redistribution of trays across phases rather than later corrective work.
Air versus liquid cooling manifold connections
For air-cooled HGX H100 racks, verify blanking panels and side baffles are in place so that all server fans pull air through the GPU heat sinks without bypass. Measure static pressure at the rack inlet and outlet to confirm the external CRAH or CRAHU units can sustain the required airflow volume once all trays are populated.
On liquid-cooled deployments, mate the quick-disconnect couplings to the rack manifold only after the entire loop has been flushed and pressure-tested. Position the supply and return lines so that they do not cross power or network cables, and install drip trays under every connection point. Once flow is established, record inlet temperature and differential pressure across each GPU cold plate before any electrical commissioning begins.
Copper NVLink backplane seating and inspection
Seat the HGX trays into the rack so that the NVLink connectors on each tray align with the copper spine or backplane. Visually confirm that every connector pair is fully mated and that no pins are bent or displaced before applying torque to the retention hardware. Because NVLink traffic never leaves the copper domain inside the rack, any seating fault will appear as a missing link in nvidia-smi rather than an optical power problem.
After all trays are installed, run the OEM link-training utility and compare the reported NVLink topology against the expected NVL72 or HGX mesh diagram. Re-seat only the trays that report degraded width or speed; random reseating of correctly reported links wastes time and risks connector damage.
MPO trunk routing and scale-out fabric patching
Route MPO trunks through the designated vertical managers on the rack sides, maintaining the manufacturer-specified minimum bend radius at every turn. Secure trunks with hook-and-loop straps only; never use zip ties that can crush the cable jacket. Label both ends of each trunk with the destination switch port and rack U-position before any patching occurs.
Clean every MPO end face with the OEM-approved cassette and inspect it with a fiberscope before insertion into the patch panel or transceiver. Record the inspection result for each fiber; any connector showing scratches, pits, or contamination must be re-cleaned or replaced before traffic is enabled.
Common field failure modes and detection points
The most frequent power-related failure is a single tray that draws excessive current on the 48 V rail because its onboard fuses were damaged during shipment; this fault is caught by measuring current on each branch during the initial low-power sequencing step. Liquid-loop failures usually trace to an incompletely seated quick disconnect that leaks under pressure; crews catch these by performing a hold test with a calibrated gauge at the OEM-specified pressure and duration before GPU power-up.
MPO-related outages most often result from end-face contamination introduced after inspection but before final mating; the only reliable prevention is to re-inspect immediately before insertion. NVLink backplane faults appear when a tray is not fully seated, producing a partial link width that only the link-training utility reveals. Documenting each of these checks in the commissioning checklist prevents the same fault from reaching production.
Final commissioning sequence and documentation
After power, cooling, and cabling are complete, execute the full rack bring-up script supplied by the GPU OEM. Confirm that every GPU reports correct PCIe link width, NVLink topology, and thermal sensor values before handing the rack to the network team. Capture baseline power, flow, and temperature readings at idle and at a controlled synthetic load.
Leviathan Systems crews archive the completed checklist, fiber inspection images, pressure-test logs, and nvidia-smi output in the project repository so that any later anomaly can be compared against the as-built state. This package becomes the reference for the next rack in the row and for any future maintenance activity.
Standards referenced: TIA-942 · IEC 61300-3-35 · ASHRAE TC 9.9 thermal guidelines
Frequently asked_
How do you confirm NVLink is operating before the scale-out fabric is patched?
Run the OEM link-training utility after all HGX trays are seated and before any MPO cables are connected to the switches. The utility reports link width and speed directly from the copper backplane; any missing links are corrected by reseating the affected tray. Only after NVLink topology matches the expected mesh is the rack released for scale-out patching.
What pressure is used for liquid-loop leak testing on HGX H100 manifolds?
Apply the pressure specified in the rack OEM manifold documentation and hold for the documented duration. Monitor with a calibrated gauge; any drop indicates a fitting or cold-plate leak that must be resolved before power is applied. Record the final stabilized pressure in the commissioning log.
When must MPO end faces be inspected relative to final mating?
Inspect immediately before insertion into the patch panel or transceiver. Earlier inspection does not protect against contamination introduced during routing or handling. Any connector failing the IEC 61300-3-35 criteria is cleaned or replaced on the spot.
Who performs the final NVLink topology validation on an NVL72 rack?
The deployment crew runs the validation utility after mechanical seating but before the rack is handed to the customer network team. Leviathan Systems includes the resulting topology diagram and nvidia-smi output in the as-built package so the operator can confirm the copper domain is intact.
What airflow verification is required for air-cooled HGX H100 racks?
Measure inlet and outlet static pressure with all trays installed and fans at full speed. Confirm that measured values meet the CRAH/CRAHU design point for the total rack CFM. Any shortfall requires adjustment of blanking panels or external ducting before load testing begins.