Buyer's Guide_
Coordinating a Multi-Site GPU Rollout: Standardize Once, Deploy Everywhere
This article details how to create reusable rack templates, cabling sequences, and acceptance checklists that let crews execute identical GPU deployments at every site without re-engineering each location.
Key facts
- NVLink domains inside NVL72-class racks use copper backplanes; MPO trunks carry only the scale-out InfiniBand or Ethernet fabric between racks.
- Factory-terminated MPO trunks require cleaning and inspection before every mating; field polishing of MPO ferrules is not performed on these deployments.
- TIA-942 provides the tiered availability framework that drives redundant power and cooling paths in multi-site GPU halls.
- A calibrated MPO continuity tester verifies pin mapping and polarity on every trunk before it is dressed into the rack.
- An OTDR trace is required on every fiber link longer than the short-reach copper NVLink domain to confirm loss and event locations.
- Leviathan Systems maintains a single master bill of materials and torque sequence that is version-controlled and issued to every crew before mobilization.
- Rack-level liquid cooling manifolds are pressure-tested to the OEM hydrostatic specification before GPUs are installed.
Create a single master rack template before any site mobilization
Lock the rack elevation, PDU placement, manifold routing, and switch-to-GPU patch schedule into one controlled drawing set. Every subsequent site receives the identical drawing package with only the site-specific row and aisle identifiers changed. This eliminates ad-hoc decisions that otherwise appear when crews interpret incomplete prints on day one.
The template also fixes the exact sequence of copper NVLink spine installation relative to fiber trunk dressing. Because the two fabrics occupy separate physical domains, the template enforces that backplane copper work finishes and is verified before any MPO trunk enters the rack. Crews then follow the same order at every location, reducing the chance that a fiber trunk is routed through a space reserved for the NVLink spine.
Standardize the fiber trunk handling and testing workflow
All MPO trunks arrive factory-terminated. On receipt, crews perform a visual inspection with a fiberscope followed by a continuity test on a calibrated MPO tester to confirm polarity and pin mapping. Only after both checks pass is the trunk labeled and coiled for later installation.
During installation the trunk is routed along the designated path with minimum bend radius maintained and no tension on the connectors. Once both ends are mated, an OTDR trace is captured and compared against the baseline stored in the master template. Any event exceeding the allowable loss budget triggers immediate cleaning or replacement before the next rack is started.
Train and certify crews against the same checklist at every site
Leviathan Systems runs a two-day certification course that covers the master template, torque values for manifold connections, and the exact order of GPU insertion after cooling loops are proven. Only technicians who have completed the course and passed the practical assessment are assigned to multi-site rollouts.
The same checklist travels with each crew. It lists every step from rack alignment to final nvidia-smi verification of the scale-out fabric. Because the checklist is identical, site leads can audit any rack and immediately see whether the prior crew followed the sequence or skipped a step.
Define acceptance criteria that travel with the template
Acceptance requires three independent sign-offs: mechanical, optical, and compute. Mechanical covers manifold pressure test results and PDU torque marks. Optical covers MPO inspection photos, continuity reports, and OTDR traces. Compute covers GPU enumeration, NVLink status within the rack, and successful ping across the scale-out fabric.
All three data sets are uploaded to the shared repository before the crew leaves the row. The next site begins only after the prior site’s package is reviewed and closed. This gate prevents a defect discovered at site three from propagating to sites four through ten.
Common field failure modes and how the template catches them early
The most frequent failure is an MPO connector left dirty after initial inspection, producing intermittent errors once traffic starts. The template requires a second inspection immediately before mating and again after the OTDR trace; any connector that fails is replaced before the rack is powered.
A second common issue is reversed polarity on a trunk that was re-labeled on site. The master template fixes polarity at the drawing level and the continuity tester is used on every trunk before it leaves the staging area. Crews that skip the test discover the error only during fabric bring-up, after the rack is fully populated. Enforcing the test at receipt prevents that rework.
A third failure is coolant manifold leaks that appear only after GPUs are installed and the loop is pressurized. The template places the hydrostatic test immediately after manifold assembly and before any GPU is racked. Leaks are therefore found while the rack is still empty and easy to repair.
Sequence logistics so each site receives identical material kits
Kits are packed at the central warehouse according to the master bill of materials and shipped with the same part numbers and quantities. Any substitution requires written approval against the current template revision. This prevents crews from improvising with locally sourced parts that later fail acceptance.
Delivery windows are staggered so that the fiber trunk kit arrives after the racks are set but before the cooling loop is filled. The sequence is written into the project schedule and reviewed at the daily stand-up. When a kit is delayed, the crew shifts to the next rack that already has its fiber on site rather than starting partial work that will require later rework.
Standards referenced: TIA-942 · TIA-568.3-D · IEC 61754-7
Frequently asked_
How many MPO trunks are tested per rack before the scale-out fabric is considered ready?
Every MPO trunk that connects a rack to the leaf or spine switch is tested. The master template lists the exact count for the chosen topology. Crews record the OTDR trace and continuity report for each trunk and upload them before moving to the next rack.
What happens if a site discovers a template error after the first two racks are built?
Work stops on the third rack. The site lead submits a change request with photos and measurements. The template owner updates the drawing set, re-issues the affected pages, and notifies all other sites. Only after the revised package is acknowledged does work resume.
Do crews perform any field termination of MPO connectors?
No. All MPO trunks are factory-terminated and polished. Field work is limited to inspection, cleaning, routing, and testing. Any connector that fails inspection is replaced with a new factory-terminated assembly.
How does Leviathan Systems keep torque values consistent across crews?
The master template includes the OEM torque specification for every fastener on the manifold and rack hardware. Each crew carries calibrated torque wrenches and marks each fastener after it is torqued. The acceptance checklist requires a second technician to verify the marks before the rack is released.
When is the copper NVLink spine verified relative to the fiber fabric?
The spine is installed and checked first. Only after nvidia-smi reports all intra-rack NVLink links up does the crew begin dressing MPO trunks for the scale-out network. This order prevents fiber work from interfering with the backplane installation.