HTTP/2 Streams Do Not Count GPU Batches or Server Workers

Identify the connection and the peer advertising the limit

A proposal saying six concurrent streams leaves a buyer's first question unanswered: six at which connection, advertised by which endpoint? Start with the two endpoints and the observation time before relating that number to the proposed inference system.

RFC 9113, published in June 2022, permits multiple streams within one HTTP/2 connection. A server's SETTINGS_MAX_CONCURRENT_STREAMS constrains client-initiated streams; a client's setting constrains server-initiated streams. The receiver must respect that limit. Open and either half-closed state count; reserved states do not. A half-closed stream has ended message sending in only one direction. Stream identifiers and cumulative response totals are not a current-state tally.

Keep this transport description separate from the proposal's application claim. A number labeled concurrency may instead refer to requests awaiting replies, work assigned to replicas or a benchmark's outstanding population. Ask the evidence owner to define the counted object. Reusing the same word does not supply a mapping between those populations.

References: [R1]

HS-234: four open plus two half-closed accounts for six

Original fictional case HS-234 contains one HTTP/2 hop between a client and a server. Stipulate that the server has advertised a limit of six client-initiated concurrent streams, that the client has applied it, and that the value remains unchanged. At the client's snapshot, four relevant streams are open and two are half-closed.

The count is 4 + 2 = 6. Counting only the four open streams would omit two members of the applicable population. Six is a stipulated value for this example, not a standard default or recommended setting.

A seventh logical request also exists, but the supplied snapshot does not show it on a newly opened seventh stream. Its local disposition is unspecified. There is no supplied evidence of a server rejection, a server queue position or a GPU wait for that request. A known transport constraint cannot fill in an unobserved application event.

The example establishes current occupancy, without reporting any actual test. It supplies no elapsed handling times, completed-request rate or identity of executing work. Even a correct six-stream count leaves those purchase questions open.

References: [R1]

Four model-batch records require their own request mapping

Separately stipulate that HS-234's model batch contains four input records. The batch membership is known only as a record count; no mapping to the transport snapshot is supplied. Four records therefore establish neither four connections nor which of the six streams correspond to executing work.

The missing information is substantive. In one possible additional account, a single application request supplies all four records. In another, four application requests each supply one record and the server combines them. These are alternative fictional mappings, not events established for HS-234. Neither can be selected from the numbers six and four.

A real implementation can assemble batches after requests arrive. NVIDIA's unversioned Triton Batchers guide, checked on October 20, 2026, describes dynamic batching combining inference requests for stateless models and distributing the resulting batches to configured model instances. That named mechanism illustrates why request receipt and batch formation need separate evidence; it supplies no HTTP/2 mapping for this example.

For the actual offer, ask how application request identities correspond to input records, batch membership and model-instance handling at the relevant time. Any claim about GPU execution needs its own observation. Subtracting four records from six streams to announce two requests waiting for a GPU mixes populations whose relationship has not been established.

References: [N1]

A terminating gateway leaves another connection to explain

Suppose the server endpoint in the transport account is a gateway that terminates the client's HTTP/2 connection and forwards requests toward a serving backend. RFC 9110's June 2022 intermediary model describes separate connections along a request path and permits a gateway to use another protocol toward its servers.

That changes what the buyer must ask next. Identify the gateway-to-backend protocol, connections and request mapping independently. The observed client-facing stream population supplies no downstream connection count, queue depth or worker assignment. A diagram joining the client, gateway and model server with one line would conceal that missing relationship.

This case concerns an HTTP participant terminating a hop. Establish that role before applying the example to a device merely carrying the traffic. Gateway behavior and backend scheduling belong to the actual proposed implementation.

References: [H1]

Byte-flow control measures another allowance

RFC 9113 flow control is receiver-directed credit in octets for DATA frames, at stream and connection level, within one hop. An octet is eight bits. This byte allowance is separate from the simultaneous-stream limit.

A proposal might display both an active-stream count and an available byte window. The latter answers how much flow-controlled data the sender is permitted to transmit under that window, not how many application requests the model server has admitted. Application records need their actual encoding and handling relationship before bytes can be related to useful work.

Likewise, an observed pause in data transfer leaves its effect on model execution unresolved unless the report connects those events. A transport observation alone cannot locate waiting work inside the application or identify a GPU bottleneck.

References: [R1]

Decide which claim the reported cap actually supports

A useful review of HS-234 can state its conclusion precisely: the supplied snapshot accounts for the stipulated transport allowance, while application admission and execution remain unaccounted for. The extra logical request has an unknown disposition, and the four-record batch has no supplied relationship to those stream identities.

For a real serving proposal, request the same connection identity, advertising endpoint, applicable setting and stream-state observation alongside the application's request identities and handling evidence. Ask what worker means in that implementation and which requests each observed worker is handling. Treat server waiting work, replica occupancy and GPU execution as distinct observations. They describe different stages even when the same request passes through them.

If the commercial claim is a completed useful-request rate, require a workload definition, observation interval and outcomes that support that rate. Increasing an allowed population supplies no processing-rate result: the proposal still needs evidence of completed work under the intended response conditions. A larger reported cap alone gives no basis to prefer one offer.

The related request-rate article addresses how demand was supplied. The deadline article addresses caller outcomes versus continuing server work. Use them for those decisions after resolving what the HTTP/2 figure actually counts. This article interprets proposal evidence; operating changes belong with the appointed application and platform owners.

Connect a separate equipment need to physical deployment

If the selected plan also includes customer equipment that needs installation, Leviathan's Complete Physical Infrastructure Deployment service can discuss agreed receiving, rack integration, onsite installation, IT cabling and physical handoff records. Use the related service and project inquiry with a nonconfidential equipment, site, connection and handoff outline.

The inquiry applies to that separate physical project. HTTP gateway delivery, serving-software configuration, load testing and tuning remain with separately appointed providers.

References: [L1]