Liquid Cooling_
Rear-Door Heat Exchangers vs Direct Liquid Cooling: Choosing Your Path to High Density
Compares rear-door heat exchangers against direct-to-chip liquid cooling for GPU racks at H100 through GB300 NVL72 densities, with concrete decision criteria based on power, piping paths, and commissioning steps that Leviathan Systems crews apply in the field.
Key facts
- NVLink connections in NVL72 racks run exclusively over internal copper backplanes and never traverse fiber or MPO trunks.
- MPO trunk cables carry only the scale-out InfiniBand or Ethernet fabric between switches and racks.
- ASHRAE TC 9.9 thermal guidelines define liquid cooling classes based on facility water supply temperature and required component inlet temperatures.
- Rear-door heat exchangers reject heat to facility chilled water loops via a single CDU per row or rack group.
- Direct-to-chip loops require separate supply and return manifolds per rack plus leak detection at every quick-disconnect.
- Factory-terminated MPO connectors must be inspected with a calibrated microscope and cleaned before insertion; field polishing is not performed.
- TIA-942-B specifies minimum separation distances between power and liquid distribution pathways in data halls.
Rack power density thresholds for RDHx viability
Rear-door heat exchangers handle GPU racks when total IT load stays inside the heat rejection capacity of a single row-level CDU and the chilled-water delta-T supplied by the facility. Once rack power exceeds that limit, door coil surface area and through-coil airflow can no longer keep component inlet temperatures inside the target band without raising facility flow or accepting higher GPU inlet temperatures.
Crews measure actual rack power at commissioning and compare the reading to the OEM maximum rating for the selected RDHx model. When the measured load exceeds the door rating, the project moves to direct-to-chip planning instead of attempting to compensate with higher CDU setpoints.
Piping and manifold layout differences
RDHx routes facility water only to the rear door through one supply-return pair per door or per row. Liquid stays outside the rack envelope and never reaches cold plates inside the chassis.
Direct-to-chip adds two manifolds per rack plus quick-disconnects at each GPU tray. The added length and fittings raise both the number of potential leak points and the pressure drop that must be checked against the CDU pump curve before the system is accepted.
NVLink and scale-out network separation
GPU-to-GPU NVLink traffic remains on the copper spine inside the rack. MPO trunks serve only the InfiniBand or Ethernet scale-out fabric that links leaf switches to spines or to other racks. Liquid loops and MPO trunks are therefore routed and tested as separate domains.
Technicians complete and verify all copper NVLink backplane connections with nvidia-smi before any liquid lines are installed. MPO trunks for the scale-out network are dressed on the opposite side of the rack to satisfy the separation distances in TIA-942-B.
Deployment sequence and tool requirements
RDHx work starts with door alignment and chilled-water connection, followed by a hydrostatic test of the door coils. Direct-to-chip work adds cold-plate mounting, manifold leak checks at every quick-disconnect, and a full-system flush before the CDU is placed online.
Both paths require an OTDR for new MPO trunks and a calibrated MPO continuity tester for every patch cord. Factory-terminated trunks arrive pre-terminated; the OEM termination kit is used only for any LC or MPO field repairs.
Common failure modes and field detection
Micro-leaks at quick-disconnects that pass initial pressure tests but open under thermal cycling, and air pockets trapped at high points in direct-to-chip loops that starve individual GPUs of flow, are the most common issues. RDHx failures are usually limited to coil fouling or CDU pump cavitation when facility water quality changes.
Crews perform a multi-hour hold test at elevated pressure while monitoring every joint with moisture sensor strips, then run the CDU at design flow while logging differential pressure across each branch. Any branch that shows rising delta-P or falling flow is isolated before GPUs are powered.
Commissioning acceptance criteria
Final sign-off requires stable GPU inlet temperatures inside the OEM-specified range at full load, zero detectable leaks after the hold test, and verified flow balance across all branches. For RDHx racks an additional check confirms that rear-door airflow stays inside server fan curve limits.
When either path fails these checks, the crew returns to the manifold or door installation step rather than raising facility water setpoints to compensate.
Standards referenced: ASHRAE TC 9.9 liquid cooling classes · TIA-942-B data center infrastructure standard
Frequently asked_
At what rack density does Leviathan Systems typically recommend switching from RDHx to direct-to-chip?
The switch occurs when measured rack power exceeds the heat rejection rating of the selected rear-door coils at the available facility water temperature. Crews run a power audit first, then compare the result against the CDU and door specifications supplied by the rack OEM.
Do MPO trunks ever carry NVLink traffic in NVL72 racks?
No. NVLink remains on the internal copper backplane. MPO trunks are reserved for the InfiniBand or Ethernet scale-out fabric only.
What leak-test protocol is used before GPU power-on?
A multi-hour hydrostatic hold at elevated pressure with moisture sensor strips at every joint, followed by an operational flow test that logs differential pressure on each branch.
How are air pockets removed from direct-to-chip loops?
The loop is flushed at maximum CDU flow with vents open at the highest points until stable flow and pressure are observed. The vents are then closed and the system is rechecked under load.
Does RDHx require any changes to the internal rack cabling plan?
No. Because the liquid path stays at the rear door, copper NVLink and MPO scale-out cabling follow the same routing used in air-cooled racks.