LEVIATHAN SYSTEMS
← Back to Glossary

What Is InfiniBand?_

InfiniBand is a high-bandwidth, low-latency network fabric used for GPU-to-GPU communication across multiple racks in AI training clusters. InfiniBand NDR (400 Gb/s) and XDR (800 Gb/s) are the current generations. InfiniBand uses specialized cables (copper DAC for short runs, fiber AOC/transceivers for longer distances) and requires dedicated switch infrastructure.

Technical Details

InfiniBand provides RDMA (Remote Direct Memory Access) capability, which is essential for distributed AI training. RDMA allows GPUs to read and write directly to each other's memory across the network without CPU involvement, dramatically reducing latency. InfiniBand networks typically use a fat-tree topology with leaf and spine switches to provide full bisection bandwidth. NDR InfiniBand operates at 400 Gb/s per port, while the newer XDR generation reaches 800 Gb/s. Cable selection depends on distance: passive DAC up to 3m, active DAC up to 5m, and AOC or fiber with transceivers for longer runs. Cable quality and termination precision directly impact network performance.

How Leviathan Systems Works with InfiniBand

Leviathan Systems installs and tests InfiniBand network infrastructure for GPU clusters, including cable routing, switch placement, and performance validation with insertion loss and return loss testing.

Related service

Network Testing & Commissioning →

Related Terms

RDMAFat-Tree TopologyDAC CableAOC Cable

Appears In

White-Space Planning for Liquid-Cooled GPU HallsWarm-Water Cooling for GPU Clusters: Why Hotter Water WinsTransceiver Breakout & Splitter Cabling for GPU FabricsNVIDIA Spectrum-X vs Quantum InfiniBand: The Cabling and Optics ViewGetting Ready for Rubin / VR200: Deployment-Readiness Planning NowRigging & Lift Plans for Heavy GPU Racks: Moving Them Without IncidentRFP & SOW Checklist for a GPU Build: Scope It So Nothing Falls ThroughRear-Door Heat Exchangers vs Direct Liquid Cooling: Choosing Your Path to High DensityRaised Floor vs Slab for High-Density GPU HallsScaling from Pilot to Production GPU Cluster: What Breaks and What to PlanOptical Transceiver Handling & Cleanliness on the FloorNCCL All-Reduce Validation as a Cluster Acceptance GateCoordinating a Multi-Site GPU Rollout: Standardize Once, Deploy EverywhereMEP Coordination for AI Data Halls: Where Mechanical, Electrical, and IT CollideBack-End vs Front-End Network Build-Out for GPU ClustersInfiniBand NDR/XDR vs RoCE: What Changes for the Cable PlantValidating an InfiniBand Fabric with ibdiagnet: Errors, Width, and RoutingHow to Choose a GPU Deployment Partner: A Buyer's ChecklistPre-Terminated Trunks vs Field Termination for GPU FabricH100 / HGX H100 Deployment Checklist: Power, Cooling, and Cabling DemandsThe GPU Deployment Site Survey: What to Measure Before You CommitGPU Data Center Deployment in Phoenix, Arizona: Hiring the Build CrewGPU Data Center Deployment in Ohio: Staffing the Physical BuildGPU Data Center Deployment in Northern Virginia: Who Does It and How to HireGPU Data Center Deployment in Georgia: Who Builds It and How to HireGH200 Grace Hopper Deployment Guide: Cabling and Cooling a Superchip NodeFloor Loading for NVL72 Racks: Will Your Slab Hold ~1.4 Tonnes?De-Racking & Decommissioning a GPU Cluster for MigrationDAC vs AOC vs AEC: Choosing GPU Rack Interconnect CablesContainment & Pathway Build-Out for a GPU RowCondensation & Dew-Point Control in Liquid-Cooled GPU HallsCommissioning Levels L1–L5 for GPU Data Centers, ExplainedB300 Deployment Guide: The Physical-Layer Step Up from B200B200 / HGX B200 Deployment Guide: Power, Thermal, and Rack DemandsThe As-Built & Handoff Package Every GPU Deployment Should DeliverAir-Cooled vs Liquid-Cooled GPU Platforms: Picking the Variant for Your SiteWriting an Acceptance Test Plan (ATP) for a GPU Cluster400G vs 800G vs 1.6T Optics: Selecting Transceivers for AI FabricSingle-Mode vs Multimode for AI Fabric: The Cable-Plant DecisionOM4 vs OM5 vs OS2: Choosing Fiber for AI Cluster ReachesCable Pathways & Containment for GPU Rooms: Overhead Tray, Ladder, Fiber RunnerPatch-Panel & Cassette Design for GPU Halls: Breakout, Density, ServiceabilityNVIDIA B200 vs GB200: HGX vs Rack-Scale, and What Changes to DeployMPO Trunk Planning & Fiber-Count Math for GPU HallsAMD MI300X vs NVIDIA H100: Infrastructure & Deployment DifferencesNVIDIA GB300 NVL72 Explained: Specs, Power, and What It Takes to DeployDell vs Supermicro vs HPE GPU Servers: A Deployment ComparisonData Center Migration for AI Infrastructure: A Practical Field GuideNVIDIA H100 vs H200 vs B200: What Changes for DeploymentGoogle TPU vs NVIDIA GPU: What It Means for Your AI Infrastructure BuildDirect-to-Chip vs Immersion Liquid Cooling for GPU Data CentersData Center Structured Cabling Standards: TIA-942, TIA-606-C, BICSIData Center Rack-and-Stack Services for GPU Builds: What's IncludedHow to Choose a Data Center Liquid Cooling CompanyWho Deploys GB200 / GB300 NVL72 Infrastructure?How Long Does GPU Cluster Deployment Take?GPU Data Center Deployment in Texas: Who Does It and How to HireGPU Commissioning & Acceptance: What to Demand Before You Sign OffSite Readiness Before the GPUs Arrive: Power, Cooling, Floor, PathwaysRail-Optimized vs Fat-Tree: The Field Wiring Plan, Port by PortIn-House vs. Outsourced GPU Deployment: How to DecideGPU Rack Assembly: What Drives the CostSpectrum-X vs InfiniBand: What's Different for the Cable PlantHGX vs DGX: What's Different When You Deploy ThemPre-Power Inspection: The Walkdown Before Energizing a GPU HallGB300 NVL72 Deployment: Power, Cooling, and the Cable PlantGB200 vs GB300 NVL72: What Changes for DeploymentNCCL Bandwidth Validation: Proving a GPU Fabric Before ProductionGPU Rack Receiving, Staging & Lift Plan: Moving ~1,360 kg Racks Without DamageAir-to-Liquid Cooling Retrofit: The Install SideThermal Burn-In for GPU Clusters: Duration, Watch Items, Pass/FailStructured Cabling QA/QC for GPU Racks: Bend Radius, Slack, Torque, DressingOTDR & Insertion/Return-Loss Testing for GPU Cluster FiberMPO Polarity (Method A/B/C) for GPU Fabric — and the #1 Cause of Dead LinksCable Labeling & As-Built Documentation for 100k-GPU Builds (TIA-606-C)