Networking_
NVIDIA Spectrum-X vs Quantum InfiniBand: The Cabling and Optics View
This article compares Spectrum-X Ethernet and Quantum InfiniBand fabrics strictly through the cabling, optics, and topology tasks performed by field crews during NVL72-class rack deployments, including switch port mapping, MPO trunk routing, and link validation steps.
Key facts
- NVLink GPU-to-GPU traffic inside NVL72 racks stays on the internal copper spine regardless of whether the scale-out fabric is InfiniBand or Ethernet.
- MPO trunk cables for scale-out networks are factory-terminated; field crews perform only patching, routing, cleaning, inspection, and testing.
- Spectrum-X deployments use Ethernet switches with QSFP-DD or OSFP ports carrying RoCE traffic over single-mode or multimode fiber.
- Quantum InfiniBand fabrics rely on HDR or NDR switches using QSFP ports with the same MPO-terminated trunks for inter-rack connectivity.
- Polarity must be verified on every MPO link because a single reversed pair produces a non-functional fabric port.
- An OTDR or calibrated MPO continuity tester is required before any high-speed link is brought into the fabric.
- Bend-radius violations on trunk cables are the most frequent cause of intermittent errors once the fabric is under load.
Scale-out topology mapping before cable pulls
Spectrum-X Ethernet fabrics connect leaf and spine switches in a Clos arrangement sized to the number of racks, with each leaf switch uplink count determined by the desired oversubscription ratio. Quantum InfiniBand uses a similar fat-tree layout but enforces stricter credit-based flow control that makes any single link loss immediately visible to the subnet manager.
Crews therefore lay out trunk counts and lengths from the same rack elevation drawings for either fabric, yet they must confirm the exact port-to-port mapping supplied by the network architect because Ethernet and InfiniBand port numbering conventions differ on the same switch hardware. The copper NVLink domain inside each rack remains untouched; only the fiber leaving the rack for the scale-out network changes with the chosen fabric.
Switch port hardware and breakout decisions
Both fabrics terminate on switches that accept QSFP or OSFP modules, but Spectrum-X Ethernet switches often arrive with ports pre-configured for 400 GbE or 800 GbE operation while Quantum switches default to 400 Gb/s or 800 Gb/s InfiniBand. Field teams verify the installed transceiver type against the bill of materials before any trunk is patched because an Ethernet optic will not train on an InfiniBand port and vice versa.
When higher radix is required, crews install MPO breakout cassettes at the switch faceplate rather than at the rack top, reducing the number of individual jumpers that must be managed during later moves. This placement also keeps the higher-loss cassette connections inside the controlled switch environment rather than in the cable tray.
Optic selection and reach constraints
Single-mode DR4 or FR4 modules are the default for runs between rows or across halls in either fabric because the power budget supports the distances typical in AI halls. Multimode SR4 modules appear only for intra-row leaf-to-spine links where the shorter reach keeps cost and power lower.
The decision between the two optics is made from the same measured tray lengths and loss budgets; crews do not default to single-mode simply because the fabric is InfiniBand or Ethernet. Once the optic type is fixed, the same cleaning and inspection sequence applies to both fabrics because contaminated end-faces produce uncorrectable bit errors at 400 Gb/s and above.
MPO trunk routing and polarity management
Factory-terminated MPO trunks are pulled from the designated fiber zone to each rack in a single continuous run, maintaining the manufacturer-specified minimum bend radius throughout the tray system. Polarity is set at the patch-panel end using Type-B or Type-C MPO adapters according to the fabric design; reversing any pair during this step produces a crossed link that will not train.
Leviathan Systems crews label both ends of every trunk with rack, panel, and port identifiers before the pull begins so that later verification does not require tracing through dense bundles. The same trunks and panels serve either fabric; only the switch-side optic and the subnet or VLAN configuration change.
Link validation sequence
After patching, every MPO connection is cleaned with the OEM-approved tool and inspected with a fiber microscope before any power is applied to the switches. A calibrated MPO continuity tester then confirms correct polarity and continuity on all twelve or sixteen fibers; only after this check is an OTDR trace performed on longer trunks to quantify loss and locate any macro-bends introduced during installation.
Once optical parameters pass, the switches are powered and the fabric manager or Ethernet controller is allowed to bring links up one leaf at a time. This staged approach isolates any mis-patched trunk before the entire fabric attempts training and prevents the subnet manager from marking large sections of the network as degraded.
Common failure modes observed in the field
The most frequent cause of post-commissioning link flaps is residual contamination on MPO end-faces after repeated insertions during testing; even a single particle larger than the core produces enough loss to exceed the receiver budget at 400 Gb/s. Crews therefore re-inspect any port that shows marginal power readings rather than simply re-seating the connector.
A second recurring issue is polarity reversal on one or more trunks when multiple vendors supply patch panels with different keying conventions; the error is invisible during visual inspection yet prevents the fabric from forming. Documenting the exact adapter type on the as-built drawings and verifying with a polarity tester before the fabric is declared ready eliminates this class of fault. Bend-radius violations created when trunks are cinched too tightly in vertical managers produce intermittent errors only after the hall reaches operating temperature; measuring the actual radius at each support point during the initial pull prevents later rework.
Standards referenced: TIA-568.3-D · IEC 61754-7 · InfiniBand Architecture Specification Volume 2 · IEEE 802.3cu · IEEE 802.3ck
Frequently asked_
Can the same MPO trunk be used for both Spectrum-X and Quantum InfiniBand?
Yes. The trunks themselves are identical; only the optics inserted at each end and the switch software configuration differ. Polarity and loss requirements remain the same for both fabrics at 400 Gb/s and above.
What test equipment must be on site before any trunk is accepted?
A microscope for end-face inspection, a calibrated MPO continuity tester for polarity, and an OTDR for loss and event location on runs longer than 30 m. No link is considered ready until all three checks pass.
How does NVLink inside the rack interact with the choice of scale-out fabric?
It does not. NVLink traffic stays on the copper backplane inside each NVL72 rack; the fiber leaving the rack carries only the scale-out network whether that network runs InfiniBand or Ethernet.
What is the first step when a new trunk fails to train?
Re-inspect and re-clean both end-faces. Contamination is the dominant cause of training failures; only after optical integrity is confirmed should crews examine polarity or switch port configuration.
Who typically performs the final fabric bring-up after cabling is complete?
Leviathan Systems field teams that installed the trunks perform the optical validation and staged link-up sequence, then hand the verified fabric to the network operations team for workload integration.