Liquid Cooling_
Direct-to-Chip vs Immersion Liquid Cooling for GPU Data Centers
An educational comparison of direct-to-chip and immersion liquid cooling for GPU infrastructure, covering deployment, maintenance and system-compatibility considerations. Liquid-cooling work is outside Leviathan's service scope.
Key facts
- Direct-to-chip cooling uses precision-machined cold plates attached to GPU hot spots, removing heat via a liquid loop; immersion submerges entire server trays in dielectric fluid.
- Contact arrangement and phase behavior are separate distinctions: both cold-plate and immersion systems can use single-phase or two-phase cooling. [LC2]
- NVIDIA's named DGX GB compute trays retain air-cooled networking and storage beside CPU/GPU cold plates. [N1]
- Temperature, flow, pressure drop, cleanliness and material compatibility are requirements of the selected cold-plate system; the cooling label establishes no universal junction temperature or performance ranking. [CP1]
- Immersion compatibility concerns the actual fluid formulation and every wetted component and material. A generic fluid category or dielectric property does not establish complete-system compatibility. [LC3]
- The supported serviceable assembly and required access determine the maintenance comparison. Neither quick-disconnects nor an immersion tank establish a GPU hot-swap capability, intervention time or mandatory tank-draining requirement.
- Compare energy and cost at matching equipment, workload and facility boundaries. Cooling architecture alone establishes no percentage saving, PUE, maintenance interval or total-cost winner.
Direct-to-Chip Cold Plates: Follow the Contact Heat Path
Direct-to-chip (DTC) cooling transfers component heat through a contacting cold plate into its coolant circuit. The cold plate is a heat exchanger, generally with metal fins and channels through which liquid absorbs heat. Its configuration-specific requirements include temperature, flow, pressure drop, material compatibility and cleanliness. The coolant travels within the cold plate; direct-to-chip does not mean direct liquid contact with silicon. [CP1]
In ASHRAE's liquid-to-liquid CDU example, a heat exchanger transfers heat from the technology cooling circuit to the facility-water circuit while keeping their fluids separate. The white paper also discusses non-CDU designs. [LC1]
The practical comparison starts with the supplied assembly. Which components have cold plates, which remain air cooled, and which supporting equipment is included? A factory-supplied cooling assembly and a proposal to modify an existing server are different offers. A drawing showing a cold plate establishes neither a supported field conversion nor permission to alter its attachment. The buyer needs the offered configuration's applicable requirements and service information; another product's pressure multiplier, attachment torque or thermal-interface prescription cannot complete that offer.
Immersion Cooling: Separate Contact Arrangement from Phase Behavior
Immersion places electronics wholly or partly in a dielectric fluid. Contact arrangement describes what the fluid surrounds; phase behavior describes how it transports heat. Single-phase coolant remains liquid as it warms and carries heat away. In a two-phase design, liquid boils while absorbing heat; vapor then condenses, rejecting heat before returning as liquid. The ASHRAE Episode 44 transcript describes both phase variants for cold plates and immersion. The contact label alone therefore cannot identify phase behavior. [LC2]
For a buyer, these are separate questions to answer for each offer. Identify the electronics actually immersed and the heat path from that fluid to the eventual heat-rejection system. Then identify whether phase change is part of the supplied design. A tank description alone says too little to rank the complete equipment's complexity or energy use.
Compatibility must follow the exact combination. OCP's November 2022 guidance expressly distinguishes generic fluid classifications from compatibility of a specific fluid and material. Removing fans, changing connectors or putting an otherwise air-cooled server into a bath cannot be justified by a fluid's dielectric label. Those changes would need applicable support for the resulting equipment configuration. [LC3]
Thermal Performance: Hot-Spot Management and Load Variability
Compare component temperature with the specified coolant inlet, operating conditions and heat load. The cold plate's configuration-specific requirements matter; matching inlet temperatures alone establishes no GPU-temperature or transient-response ranking. [CP1]
The equipment boundary matters just as much. NVIDIA's DGX GB Rack Scale Systems hardware guide, updated 17 September 2026, describes CPU and GPU cold plates in the compute trays while networking and storage components remain air cooled by fans. That is a named configuration's heat-path distinction, not a universal fraction of heat captured by liquid or a description of every liquid-cooled GPU system. [N1]
For either offer, compare the specified workload and operating conditions, the components included in the thermal data, remaining air-cooled loads, and the supporting heat-rejection equipment. A claim about processor cooling cannot silently cover separate switches, storage or power equipment. Ask whether auxiliary power is included in each energy figure before comparing them. Training and inference labels alone do not establish a transient-response or efficiency winner; the relevant evidence must describe the offered system under the workload being assessed.
Maintenance, Serviceability, and Failure Modes
Serviceability is a property of the supported assembly and its access arrangements. The replaceable item could be a component, board, tray or another defined assembly. A GPU appearing on a parts list does not establish that it can be replaced individually at the site, and a quick-disconnect does not establish that replacement can occur while powered or without interrupting other equipment.
An immersion proposal needs the same precision: which assembly is serviced, where the service operation occurs, which other equipment is affected and what access or handling provisions the supplier requires. Neither full-tank draining nor a particular lifting method follows from the word immersion. Without applicable service documentation, a minutes-versus-hours comparison would be invented.
Keep maintenance evidence tied to its boundary too. ASHRAE's 2019 white paper distinguishes manufacturer testing of supplied IT assemblies from testing of site-installed interconnecting piping. It provides no basis for imposing one annual pressure test on every installed system. [LC1] For the comparison, obtain the selected supplier's maintenance scope and support terms, including applicable consumables and fluid-care responsibilities, rather than assigning universal fluid replacement, dielectric-testing or filter-change intervals.
Integration with GPU Rack Infrastructure: NVLink and Networking
NVIDIA documents an NVLink passive copper cable cartridge backplane and cold-plate compute trays in the named DGX GB rack arrangement. That description does not qualify an immersion conversion. [N1]
A dielectric-constant cutoff cannot decide immersion eligibility for an NVL72-class system. The comparison needs support for the complete proposed electronic, mechanical, fluid and material combination. Public explanations of immersion do not establish such an alternative for a particular NVIDIA rack. If an offer proposes a conversion, its supported status remains a question for the actual OEM configuration owner.
The external network also needs an equipment-specific comparison. An InfiniBand or Ethernet label does not identify which cables, connectors or transceivers contact fluid, where they are accessible, or what the supplier supports. Ask the proposer to show those interfaces and the supported serviceable assembly. Do not infer that all immersion systems need sealed MPO bulkheads or that every transceiver replacement requires draining a tank.
Failure Evidence: Distinguish Compatibility, Integrity and Recovery
A fluid-property result, a material-compatibility assessment and a leak-integrity result answer different questions. OCP's Material Compatibility in Immersion Cooling, revision 1.0, calls itself guidance rather than a requirement and warns against treating generic fluid/material assessments as absolute compatibility. Its scope concerns wetted IT materials, including those in servers, storage, networking and power supplies. [LC3]
That matters when reviewing failures and recovery claims. Evidence about one fluid property cannot establish that a tank or cold-plate circuit is leak-free. Nor can the word dielectric prove that every affected material is compatible or that equipment exposed during a leak remains undamaged. A buyer should distinguish the supplier's compatibility basis, its containment and detection provisions, and its supported disposition of affected equipment. These are separate parts of an offer, not substitute names for one safety result.
Compare the maintenance and recovery commitments actually provided: the affected assembly, service access, applicable support coverage, replacement availability and responsibilities. Retain missing information as an open comparison point. Claims about the most frequent failure, a mandatory redundant-pump arrangement or a guaranteed recovery time need evidence for the relevant system; they cannot be supplied from the cooling category alone.
Decision Criteria: Choosing Between DTC and Immersion
Choose between complete supported offers. The useful comparison combines the contact heat path, phase behavior, actual equipment and fluid/material combination, remaining thermal interfaces, and supported serviceable unit. A favorable answer on one axis does not settle the others. Existing facility water, a GPU power rating or an attractive tank description cannot establish compatibility or site feasibility.
Consider two fictional offers, with properties stipulated solely for this example. Offer A supplies a cold-plate compute tray, retains a documented air-side load and identifies the complete tray as the supported replacement assembly. Offer B supplies an immersion enclosure, names its supported server/fluid combination and also identifies the complete server tray as the replacement assembly. B's attachment covers the immersed servers but leaves a separately supplied network switch outside that boundary. Neither offer includes measured energy or intervention-time results.
The comparison yields specific conclusions. A's liquid-cooled description cannot erase its remaining air interface. B's server compatibility attachment cannot cover the separate switch. Both offers describe tray replacement, so neither supports a claim of individual GPU hot-swap. The buyer can compare those supported boundaries and request the missing switch requirements, but cannot rank PUE, repair duration or total cost from these stipulated properties.
The linked Cooling Responsibility Guide explains the boundary with physical IT work. Leviathan can discuss separately agreed physical IT installation and cabling for customer-supplied equipment; it does not perform, coordinate or subcontract liquid-side work, cooling measurement or commissioning. The customer appoints the responsible OEM and cooling/facility specialists. Use Discuss a Project below to describe the equipment, permitted IT tasks and external readiness dependencies for a physical IT inquiry.
Standards referenced: LC1 — ASHRAE TC 9.9, Water-Cooled Servers: Common Designs, Components, and Processes, 2019; 10 October 2019 errata incorporated. Educational scope: separate facility/technology circuits and supplied-assembly/site-interconnect testing boundaries; no universal test requirement. https://www.ashrae.org/File%20Library/Technical%20Resources/Bookstore/WhitePaper_TC099-WaterCooledServers.pdf — checked 20 October 2026. · LC2 — ASHRAE Journal Podcast Episode 44, written transcript, Dustin Demetriou's explanation of contact arrangements and single-/two-phase variants. Publication date unverified; conceptual evidence, not product qualification. https://www.ashrae.org/news/ashraejournal/ashrae-journal-podcast-episode-44 — checked 20 October 2026. · LC3 — Open Compute Project, Material Compatibility in Immersion Cooling, revision 1.0, last updated 28 November 2022. Introduction, purpose and scope: informative guidance on specific fluids and wetted materials, not complete-system approval. https://www.opencompute.org/documents/material-compatibility-in-immersion-cooling-document-version-1-0-nov-28-2022-1-pdf — checked 20 October 2026. · CP1 — Dustin Demetriou and David Quirk, Liquid Cooling Cold Plates, ASHRAE Journal, September 2025, vol. 67, no. 9; public excerpt only. Heat-exchanger mechanism and configuration-specific requirements. https://www.ashrae.org/technical-resources/publications-library/ashrae-journal/featured-articles/september-2025-liquid-cooling-cold-plates — checked 20 October 2026. · N1 — NVIDIA, DGX GB Rack Scale Systems User Guide: Hardware; updated 17 September 2026. Named compute-tray cooling and rack arrangement only. https://docs.nvidia.com/dgx/dgxgb200-user-guide/hardware.html — checked 20 October 2026.
Frequently asked_
Can I use immersion cooling for NVL72 racks with copper NVLink backplanes?
No immersion conversion is established by the cited DGX GB documentation. Compatibility needs the complete proposed rack/fluid combination, not a dielectric-constant cutoff. This article identifies no supported immersion NVL72 offer. [N1]
What determines the maintenance schedule for direct-to-chip cooling?
Use the selected system's applicable service documentation and support responsibilities. ASHRAE's cited discussion distinguishes supplied-assembly testing from site-interconnection testing; it does not establish a universal annual pressure test, fluid-change interval or individual-GPU service procedure. [LC1]
Does a fluid-quality result establish leak integrity or equipment safety?
No. A result for a specified fluid property does not establish containment integrity, compatibility of every wetted material or the condition of affected electronics. Read the result against its stated method, scope and criteria, and retain the separate integrity and equipment-disposition questions for the responsible specialists.
Which cooling method is more energy-efficient for a 10 MW GPU cluster?
The stated 10 MW does not decide the comparison. First establish whether it describes IT input, a cooling duty or another boundary. Compare the actual equipment and workload, remaining air-cooled loads, auxiliary energy and facility heat rejection over the same operating period. Without applicable data, neither a percentage saving nor a PUE or total-cost ranking is supported.
What happens if a cold plate leaks during operation?
The outcome depends on the actual fluid, exposure, equipment and system provisions. A dielectric or deionized label cannot guarantee that the GPU remains undamaged. The applicable OEM and appointed specialists must determine the response and disposition of affected equipment. This comparison recommends no replacement fluid and supplies no isolation, seal-repair or return-to-service procedure.