Liquid Cooling_
Facility Water Loop Design for AI Halls: TCS to FWS Done Right
This article specifies the design sequence, parameters, and verification steps for facility water systems feeding CDUs in AI halls, including temperature and flow matching, water treatment, tie-in geometry, and field testing that prevents CDU trips during GPU rack commissioning.
Key facts
- FWS and TCS remain isolated by the CDU plate heat exchanger so that facility water never enters the rack cold plates or manifolds.
- CDU primary-side flow and secondary-side flow must be balanced to the rack heat load before the first GPU power-on sequence.
- Water quality sampling occurs at the CDU inlet and return headers using calibrated conductivity and particle counters.
- Tie-in points use isolation valves and drain ports sized to allow full loop drain-down without disturbing adjacent CDU circuits.
- Piping supports and expansion loops are required wherever FWS runs cross cable trays or under raised floors serving NVL72 racks.
- Pressure decay testing of each FWS branch occurs after flushing and before connection to any CDU.
- Leviathan Systems performs final FWS-to-CDU tie-in verification as part of the integrated systems test prior to liquid cooling commissioning.
Setting FWS Supply Temperature and Flow to Match CDU Performance
FWS supply temperature is selected from the CDU OEM data sheet so the secondary loop can maintain the required cold-plate inlet temperature at peak rack load while the facility chillers or dry coolers still reject the heat. Flow rate is calculated from rack thermal design power and the allowable temperature rise across the CDU heat exchanger; both values are taken directly from the CDU OEM curve rather than rack-level estimates.
Balancing starts at the main header and moves outward to each branch using calibrated balancing valves. Each CDU is then operated at design primary flow with the secondary loop isolated so measured pressure drop can be checked against the OEM curve. Deviations require rebalancing before the secondary loop is filled and brought online.
Water Chemistry and Filtration Requirements for the Facility Loop
FWS water must stay inside the CDU heat-exchanger material limits for conductivity, pH, and suspended solids to prevent scaling or corrosion on the plates. Side-stream filtration sized to a fraction of total system flow removes particulates that would otherwise lodge in the narrow plate channels. Biocide and corrosion inhibitor residuals are confirmed by grab samples at the CDU supply header before any CDU is enabled.
Make-up water enters only through a dedicated treatment skid. Conductivity and turbidity sensors on the return header feed the BMS so excursions trigger alarms before they reach operating CDUs. Sampling intervals follow the water-treatment vendor procedure and are recorded in the commissioning plan.
Locating and Configuring Tie-In Points for CDU Arrays
Tie-in points are placed on the FWS mains so that any single CDU can be isolated without shutting down an entire row. Each tie-in includes manual isolation valves, drain valves, and gauge ports on both sides. Branch piping is routed to let the CDU roll out on its casters without disturbing other services.
Flanges or grooved couplings are used at the final connection. Tie-in elevation maintains positive suction head at the CDU pump at minimum system fill level. All tie-ins receive circuit tags that match the as-built P&ID before water is introduced.
Piping Material, Support, and Routing in GPU Halls
Piping material is chosen from the water-treatment chemistry limits; copper is avoided when glycol or high-chloride water is present. Horizontal runs receive supports at intervals that prevent sagging under water weight plus insulation. Vertical risers include expansion loops sized for the expected seasonal temperature swing.
FWS piping maintains adequate clearance from high-voltage busways and from the copper NVLink spine at the rear of NVL72 racks. Fire-rated penetrations use fire-stop sleeves that preserve the rating while permitting pipe movement. All supports are rated for the filled-pipe condition before hydrostatic testing.
Common Failure Modes at the FWS-to-CDU Interface
Incomplete air removal after fill produces pump cavitation and CDU trips shortly after startup. Heat-exchanger fouling occurs when side-stream filtration is bypassed during flushing or when make-up water quality is not verified before use. Rapid valve closure creates pressure surges that deform gaskets at tie-in flanges and produce chronic leaks.
These failures are caught by a controlled fill with high-point vents open, followed by circulation while turbidity is monitored continuously. Each CDU is then exercised across its operating flow range while inlet and outlet pressures and temperatures are logged against the OEM envelope. Any reading outside limits halts commissioning until the cause is corrected and retested.
Verification Sequence Before GPU Rack Power-On
After tie-in, each FWS branch receives a hydrostatic test to the pressure and duration required by the applicable piping code. The system is flushed until return turbidity matches supply turbidity. CDU isolation valves are opened one row at a time while main-header pressure is watched to confirm flow stays inside the design band.
Final sign-off requires logged CDU controller data showing stable secondary-loop temperatures at design load, verified BMS alarms for conductivity and flow, and a completed leak inspection of every mechanical joint. Only after these items are documented does the rack receive its first liquid-cooled power-on sequence.
Standards referenced: ASHRAE TC 9.9 Liquid Cooling Guidelines · ASME B31.9 Building Services Piping
Frequently asked_
How is FWS flow balanced to multiple CDUs on the same header?
Balancing starts at the farthest CDU and works inward using calibrated balancing valves. Each CDU is set to its design flow with the secondary loop isolated so primary-side pressure drop can be verified against the OEM curve. The process repeats until header pressure remains stable when additional CDUs are brought online.
What water quality parameters must be checked at the CDU inlet?
Conductivity, pH, and turbidity are sampled at the CDU supply header before the unit is enabled. Side-stream filtration must be online and the treatment skid must be maintaining inhibitor and biocide residuals within vendor limits. Any parameter outside the CDU heat-exchanger specification halts commissioning until corrected.
Where should isolation valves be placed relative to the CDU?
Isolation valves sit on the FWS supply and return branches immediately upstream of the CDU connections, with drain ports between the valve and the CDU. This arrangement allows a single CDU to be drained and removed without affecting other units on the same header.
What test confirms the FWS loop is ready for CDU connection?
A hydrostatic pressure test followed by a controlled fill, air removal, and circulation with turbidity monitoring. After these steps, each CDU is run through its flow and temperature ramp while pressures and temperatures are logged against the OEM envelope.
Who typically performs the final FWS-to-CDU tie-in verification?
Leviathan Systems performs the final verification during integrated systems testing, confirming valve positions, pressure readings, and alarm functionality before any rack receives liquid cooling.