Data Center Design_
MEP Coordination for AI Data Halls: Where Mechanical, Electrical, and IT Collide
Details the required sequence for electrical, mechanical, and structured-cabling trades when installing H100-to-GB300 NVL72 racks so that liquid cooling loops, power feeds, and scale-out MPO trunks are completed without rework.
Key facts
- GPU-to-GPU NVLink traffic runs exclusively over the copper backplane inside each NVL72 rack and never traverses MPO or fiber.
- MPO trunk cables for the InfiniBand or Ethernet scale-out fabric are factory-terminated and polished; field work is limited to routing, cleaning, inspection, and testing.
- Liquid cooling distribution units must be pressure-tested and flow-balanced before any rack is placed on the slab.
- A calibrated MPO continuity tester and OTDR are required for every trunk before patching into the spine switches.
- Rack power whips are landed and torqued only after the mechanical manifolds are aligned and anchored.
- TIA-942 and ANSI/TIA-568.3-D govern the separation and labeling of compute fabric cabling from power and cooling pathways.
- One of the largest hyperscalers in Texas requires the cabling contractor to complete all trunk routing and testing before the first GPU rack is rolled into the row.
Electrical rough-in before mechanical manifold placement
Power distribution units and busway drops must be set and aligned to the final rack footprints before cooling manifolds are anchored. This order prevents later removal of heavy copper or aluminum bus sections when a CDU or manifold requires adjustment. Crews verify PDU phase rotation and torque all terminations to the OEM value before any liquid lines cross the same aisle.
Once the whips are landed, each rack position receives a temporary protective cover over the connectors. The covers stay in place until the mechanical trade finishes welding and pressure-testing the cooling loops. Skipping this step has caused repeated contamination of power contacts during manifold fit-up.
Cooling loop commissioning ahead of rack delivery
Distribution manifolds and CDUs are pressure-tested, flushed, and flow-balanced while the slab is still clear of racks. Flow meters at each rack stub-out are recorded and compared against the design curve supplied by the cooling equipment OEM. Any deviation above the allowed tolerance requires rebalancing before the next trade is released to the row.
Only after the loops hold pressure for the specified duration and all air is purged do the racks enter the hall. This sequence eliminates the need to drain and refill loops after a rack is already bolted down and cabled.
Scale-out MPO trunk routing after mechanical and electrical rough-in
Factory-terminated MPO trunks for the InfiniBand or Ethernet fabric are routed in overhead or underfloor pathways only after both power and cooling rough-in are complete. This prevents fiber damage from welding spatter, manifold adjustments, or busway lifts. Trunks are dressed with the minimum bend radius respected and secured at intervals specified by the pathway manufacturer.
Each trunk is labeled at both ends with the destination rack and port range before it is pulled. The labeling matches the rack elevation drawings so that later patching crews do not cross-connect between different leaf switches.
Rack installation and internal NVLink verification
Racks are rolled in, leveled, and connected to the pre-aligned power whips and cooling stubs. Once power is applied, the internal copper NVLink backplane links are checked with the rack BMC before any external fiber is connected. This confirms that all GPU-to-GPU connectivity inside the rack is functional while the unit is still on temporary power.
Only after the internal NVLink check passes are the external scale-out cables patched. Reversing this order has led crews to chase phantom link issues that were actually internal backplane seating problems.
MPO inspection, cleaning, and test sequence
Every MPO connector is inspected with a fiberscope and cleaned with the OEM-approved cassette before insertion into the switch or NIC. A calibrated MPO continuity tester verifies pin-out and polarity on the spot. Any connector that fails is set aside and replaced from the slack loop rather than re-cleaned repeatedly.
An OTDR trace is then captured on each trunk from the end-of-row patch panel to the rack. The trace is archived with the project documentation so that future adds or changes can be compared against the baseline. This step catches macrobends or contaminated end-faces that would otherwise appear only after the fabric is under load.
Common field failure modes during MEP handoff
The most frequent failure is a cooling stub-out that lands offset from the rack quick-connect because the manifold was anchored before final PDU positioning. The rack must then be shimmed or the manifold cut and re-welded, both of which stop the row for days. The second most common issue is an MPO trunk pulled before the overhead ladder rack is fully loaded with power whips, resulting in crushed fiber when the heavier cables are added later.
A third failure occurs when liquid loops are not fully vented before racks are powered; air pockets cause flow faults that are misdiagnosed as pump or CDU problems. Crews now require a signed flow-balance report and a completed OTDR archive before any rack is released for GPU bring-up. These checks have eliminated the majority of schedule slips on recent NVL72 deployments.
Standards referenced: TIA-942 · ANSI/TIA-568.3-D
Frequently asked_
When should the MPO trunks be pulled relative to rack delivery?
Trunks are routed only after power whips and cooling manifolds are anchored and tested. This order prevents damage from later mechanical adjustments or busway lifts. The trunks remain coiled on the slack loop until the racks are in place and the internal NVLink check has passed.
Who performs the final MPO end-face inspection on an NVL72 row?
The structured cabling crew performs the inspection with a fiberscope and MPO continuity tester before any patching into the leaf switches. Results are logged against each trunk serial number. The GPU deployment team then verifies link status only after the cabling contractor signs off.
What document must be complete before racks enter the row?
A signed pressure-test report for every cooling loop segment plus recorded flow values at each rack stub-out. Without these, the rack delivery is held. The same gate applies to the OTDR archive for all scale-out trunks serving that row.
How are NVLink and scale-out fabric issues isolated during commissioning?
Internal NVLink status is checked first through the rack BMC while the unit is on temporary power. Only after those links report healthy are the external MPO cables patched and tested. This sequence prevents misdiagnosis of backplane seating problems as fabric link faults.
Does Leviathan Systems handle both the copper NVLink verification and the MPO work?
Leviathan Systems performs the full sequence: rack placement, power and cooling connections, internal NVLink checks, and MPO trunk testing. The same crew owns the handoff documentation so that any later issue can be traced to the correct installation step.