Data Center Design_
White-Space Planning for Liquid-Cooled GPU Halls
Details the physical layout sequence for rows, CDUs, manifolds, and clearances in liquid-cooled GPU halls, including decision criteria and field verification steps used by crews performing rack assembly and commissioning.
Key facts
- GPU-to-GPU NVLink traffic stays on the copper backplane inside each rack; MPO trunks carry only the scale-out InfiniBand or Ethernet fabric between switches.
- CDU placement must maintain minimum service aisles on both sides per the OEM installation manual to allow heat-exchanger bundle removal without rack shutdown.
- Manifold stubs are terminated with quick-disconnect couplings rated for the loop pressure; drip pans are required under every connection point.
- Row orientation is set so that cold-aisle supply faces the facility chilled-water entry when possible, reducing hose lengths and pressure drop.
- TIA-942 and ASHRAE TC 9.9 both require documented clearance zones that remain unobstructed after final commissioning.
- Factory-terminated MPO trunks are used exclusively; field crews perform only routing, cleaning, inspection, and end-face testing with calibrated MPO testers.
Row Spacing and Rack Orientation
Row pitch starts from rack depth plus front and rear service aisles. Liquid-cooled racks add depth for rear-door heat exchangers or internal cold plates, so center-to-center distance is measured from the outermost manifold connection point rather than the frame.
Orientation places the facility chilled-water supply header on the cold-aisle side. This shortens supply and return hose runs, cuts the number of bends, and positions the return temperature sensor nearer the CDU inlet. Subsequent rows follow the same pattern only after confirming CDU piping reaches both sides without crossing egress paths.
CDU and Primary Manifold Placement
CDUs sit at the end of each row or in an adjacent mechanical bay. The unit rests on a housekeeping pad rated for flooded weight and requires clear access on three sides for filter changes and pump service. Secondary manifolds run above or below racks in dedicated trays or channels, with isolation valves every two racks so one rack can be drained without affecting the row.
Supply and return stubs drop to each rack position before racks arrive. Each stub ends with a keyed quick-disconnect and manual isolation valve oriented to prevent accidental opening. Pressure-test plugs remain on the stubs until final connection and loop re-balancing.
Service Clearances and Access Paths
Clearance on the CDU side holding the heat-exchanger bundle follows the OEM manual dimension and must stay free of trays, switches, and staging once live. Rack aisles accommodate the largest field-replaceable unit plus disconnected coolant hose length so a cold-plate assembly can exit without tilting the rack.
Power whips and fiber trunks route outside these zones. Cooling-loop priority prevails in any conflict because a blocked CDU path forces a row offline while cabling can be rerouted.
Scale-Out Network Integration
MPO trunks for the InfiniBand or Ethernet fabric install after liquid manifolds. Trunks use separate ladder rungs or trays so future hose work does not disturb optical links. Patch panels mount on the same row side as CDU connections so one technician can check both domains.
End-face inspection and continuity testing with a calibrated MPO tester occurs before any GPU rack receives power. NVLink stays inside the rack on copper, so only scale-out connections affect fabric health and are verified row by row before first rack startup.
Common Failure Modes in White-Space Layouts
The most common error places CDU service access against a wall or under a later-added tray. When the heat-exchanger bundle must be pulled, the row shuts down because the bundle cannot clear the obstruction. This is caught by walking the layout with the CDU OEM manual open and confirming the published bundle-removal dimension against every fixed object.
Return manifolds routed with insufficient slope or low points trap air and cause repeated CDU pump trips. Install manual or automatic air vents at every high point and verify flow arrows on isolation valves before first fill. Drip pans placed only under the CDU and not under every rack-level quick-disconnect allow a single leaking coupling to reach power whips below.
Pre-Commissioning Verification Sequence
After racks are set and hoses connected, Leviathan Systems crews run a low-pressure static test on each manifold segment before enabling the CDU. All quick-disconnects cycle three times and are inspected for elastomer damage. Flow starts at minimum pump speed while differential pressure is monitored across each rack; any rack above expected drop is isolated and checked for kinked hoses or closed valves.
Once stable flow is confirmed, MPO trunk validation begins. Cooling stability is proven before any scale-out link is brought up because the domains are independent. Final documentation includes marked-up drawings showing actual valve positions, vent locations, and measured aisle widths.
Standards referenced: TIA-942 · ASHRAE TC 9.9
Frequently asked_
How far apart should CDUs be placed when one unit serves two rows?
CDU spacing follows the maximum hose length allowed by the OEM pressure-drop calculation. One CDU is placed at the end of every two rows when row length keeps differential pressure inside the published operating band. Longer rows need an intermediate CDU or second manifold feed.
Do liquid manifolds share the same pathway as MPO trunks?
No. Manifolds and MPO trunks run on separate supports. MPO trunks are never installed below a liquid connection point because a drip during service would wet the fiber. Separate pathways also let one domain be modified without touching the other.
What clearance is required behind a rear-door heat-exchanger rack?
The rear aisle must fit the door swing plus the length of flexible hoses when the door opens for filter service. This dimension comes from the rack OEM drawing and is verified on site before the first rack is placed.
When is the static pressure test performed relative to rack installation?
The static test runs on the empty manifold before any GPU rack enters position. This catches leaks or misaligned couplings while the aisle is clear. After racks connect, only a low-pressure functional test repeats to confirm the new joints.
Who signs off on final white-space clearances?
The facilities engineer, the liquid-cooling OEM representative, and the deployment lead from Leviathan Systems walk the completed layout together. Each clearance zone is photographed with a tape measure visible and the marked-up drawing is added to the commissioning package.