RoCE Congestion Signals: A Priority Pause Is Not an ECN Notification

Read the mode in its release context

A proposal using the phrase RoCE congestion control needs to distinguish link flow control from endpoint congestion feedback. Naming one mechanism does not establish that the others are present or that their observations are interchangeable.

NVIDIA's Cumulus Linux 5.4 RoCE page provides a concrete qualification. Its mode comparison enables ECN in both lossy and lossless modes, while enabling PFC in the lossless mode and leaving it disabled in the lossy mode. The broad claim that every RoCE deployment universally requires PFC would therefore misrepresent this named example.

That versioned comparison explains why the proposal's actual mode matters. It does not choose a mode for another switch, software release or workload.

References: [N1]

Follow a pause on the adjacent link

Use a fictional data direction from sender A through switches S1 and S2 to receiver B. The names are teaching aids, not a proposed topology or an operating trace.

Suppose S2 sends a PFC pause toward its adjacent peer S1 for a selected priority. The requested transmission pause acts on that priority from S1 toward S2. NVIDIA's Cumulus Linux 5.4 QoS explanation distinguishes this behavior from pausing all traffic on a link: a peer receiving a PFC pause stops transmission for the affected priority while other priorities can continue.

The selected object is a priority on that link. Naming A's application flow does not make the pause exclusive to that flow; other flows sharing the affected priority can also be held. IEEE's 802.1Qbb overview likewise describes flow control per traffic class on full-duplex links.

The S2-to-S1 pause is not forwarded through the network as an instruction addressed to A. Any additional pause on another adjacent link would be a separate link-level event. A chain of effects must not be confused with one pause frame traveling end to end.

References: [N3][I1]

Follow the marked packet and the receiver's CNP

Separately, consider an ECN-capable data packet from A that S2 marks for congestion and forwards toward B. RFC 3168 distinguishes the Congestion Experienced indication in the IP header from dropping the packet. The indication travels with the data toward the receiver; the mark is not itself a new return packet.

In the RoCEv2 behavior described by the Cumulus Linux 5.4 page, the receiver generates a congestion notification directed to the packet's source. Here, B can generate a CNP toward A. B is the receiver of the data and the sender of this feedback packet.

NVIDIA's DOCA 3.3.0 archive explains the Data Center Quantized Congestion Notification (DCQCN) roles: the switch marks passing traffic, the receiving NIC supplies a CNP, and the sending NIC uses feedback to adjust transmission rate. Its page was updated August 27, 2026. This names the particular control loop being explained rather than assuming every RoCE sender uses the same algorithm.

S2's pause and B's CNP are separate control packets with different recipients. The pause acts on the adjacent priority link; the CNP supplies feedback toward A. Their placement in this explanation asserts no fixed order between pause and marking events.

References: [R1][N1][N2]

Ask a different question of each count

Consider three supplied observations: a priority-pause count, an ECN-marked-packet count and a receiver CNP count. Before interpreting them, retain the device, interface, direction, event definition and observation interval. A heading saying congestion events removes distinctions the reader needs.

A priority-pause field defined as a packet count asks how many pause packets were counted for the selected priority and direction. It does not say how many application packets were held or how long transmission stopped. As a concrete counter example, the Linux kernel's mlx5 Ethernet documentation defines received per-priority pause packets separately from pause duration and pause transitions. Those definitions concern the named driver documentation, not every switch counter.

The ECN-marked-packet count asks how many packets the stated marking observation counted. A report must distinguish a switch's marking activity from an endpoint's observation of marked arrivals. Neither is a count of CNPs. The packet's congestion indication and the receiver's subsequent feedback are different events at different observation points.

For the third observation, specify that receiver CNP count means CNPs generated or sent by B, the data receiver. A field counting CNPs received at a NIC would answer another question. B's generation or transmission count alone does not establish receipt at A or A's resulting transmission rate.

References: [K1][R1][N2]

Keep the three observations separate

Adding the three values would mix link-control frames, marked data packets and feedback packets. Even if each field has a packet unit, the objects and scopes differ. Their sum is not a count of independent congestion episodes or affected application flows.

Nor should a reviewer demand that the marking and notification totals match merely because the mechanism connects them. The supplied labels have not established matching traffic populations, observation points or counting rules. The causal illustration supplies no universal conversion between those totals.

A reported increase also needs a defined comparison interval and continuous counting history. A cumulative total is not automatically the number of events during the current workload. Missing or unsupported observations must remain distinct from measured zeros. These are conditions for interpreting an existing report, not instructions to collect or reset counters.

A priority-pause count does not prove losslessness. An ECN-marked-packet count does not prove a failed cable. A receiver CNP count does not prove a healthy network or an accepted workload. The reviewer needs evidence for the particular conclusion being requested, beyond the presence or absence of these signals.

Return the unresolved question to the right owner

When a proposal treats PFC pauses and CNPs as the same signal, request an explanation of the actual mechanism and the meaning of the supplied observations. If the purchase requires demonstrated workload behavior, the appointed technical owner must define and assess that evidence. A list of congestion counters does not supply the result.

The physical connection-package guide remains the place to compare actual switches, interfaces and channel components. The FEC evidence guide addresses corrections, packet-error observations and passive-fiber reports. This article adds the distinct question of where congestion signals travel and what each one observes.

Network design, configuration and validation remain with their appointed technical owners. Leviathan's structured cabling service covers agreed physical installation, routing, dressing, labeling, checks and connection records; the congestion discussion does not add network services to that scope.

For a physical cabling inquiry, share the site, approved endpoint and assembly schedule, routes, work window and required labels or handoff records. The linked structured cabling service and project inquiry provide the next step for discussing those deliverables.