What Is RoCE?_
RoCE (RDMA over Converged Ethernet) carries RDMA traffic over an Ethernet network, giving GPU clusters the direct, CPU-bypassing memory access of RDMA without an InfiniBand fabric. RoCEv2 is routable across IP networks and is the basis for Ethernet-based AI back-end fabrics, including NVIDIA Spectrum-X. It requires a carefully tuned, lossless or near-lossless Ethernet fabric to perform well.
Technical Details
RoCEv2 encapsulates RDMA in UDP/IP, making it routable across standard Ethernet networks while preserving the sub-microsecond, zero-copy semantics of RDMA. Because RDMA is sensitive to packet loss, RoCE fabrics rely on congestion- and flow-control mechanisms — Priority Flow Control (PFC) and Explicit Congestion Notification (ECN) — to approximate lossless behavior, and increasingly on adaptive routing and telemetry-based congestion control to keep tail latency low at scale. Physical-layer quality matters as much as on InfiniBand: dirty connectors or marginal links cause bit errors and retransmissions that erase the latency advantage. RoCE competes with InfiniBand for AI back-end networks; the trade-off is Ethernet's broad ecosystem versus InfiniBand's mature, purpose-built congestion handling.
How Leviathan Systems Works with RoCE
Leviathan Systems installs and certifies the physical fabric underneath RoCE deployments, ensuring clean, low-loss links so that lossless-Ethernet tuning is not undermined by cable-plant errors.
Related service
Network Testing & Commissioning →