CUDA Timing: Host Submission and GPU Events Measure Different Intervals

Read the host timer's finish event

A runtime offer may label a value CUDA time without saying where its clock stops. Ending at the host call's return answers how long the calling thread took to reach that point. It leaves useful-output completion unresolved when the submitted work continues asynchronously.

NVIDIA's CUDA C++ Best Practices Guide 13.4, section 9.1.1, explains that asynchronous work can continue after control returns to the CPU. A CPU timer can also span completed GPU work when its stopping condition establishes that completion. The clock's location alone does not define the interval.

Have the supplier describe the start and finish as observable events. Before enqueue to return, request arrival to output availability, and the execution of two device markers are different pairs. The term host timing should therefore be accompanied by its endpoints, rather than treated as a synonym for submission overhead.

References: [N1]

Establish that the recorded markers actually completed

The CUDA Programming Guide, whose index is dated September 10, 2026, describes events as markers of progress in a stream. A timing event receives its device timestamp when execution reaches that position. The host request to record it occurs at a different point in the process. The guide's timing example establishes completion before interpreting the elapsed interval.

The CUDA Runtime API 13.4 Event Management reference supplies the corresponding readiness qualification: its elapsed-time calculation returns a not-ready status if either recorded event remains incomplete. Merely issuing the two recording requests does not establish a usable completed interval.

For supplied evidence, ask which start and stop recordings the value describes and how their completion was established. A pending marker cannot be treated as though it has already observed its intended finish. A completed stop marker still needs the right placement: completion of the selected region leaves any required later work outside that region.

References: [N2][N3]

Interpret the elapsed stream interval within its actual scope

An event pair selects a region of execution. Identify the operations and dependencies between its markers before calling the resulting value kernel time. A region containing several kernels or transfers answers a different question from a region surrounding one operation.

The Runtime API warns that other stream operations can affect elapsed event timing in non-NULL streams, including when both events use the same stream handle. A reported interval is therefore not automatically the exclusive active duration of one kernel, or the sum of all work performed across the GPU.

Work before the start marker or after the stop marker also needs its own account. If the caller requires a returned host result, an event pair ending before that return path leaves the caller's finish unobserved. If completion depends on additional streams, the supplied evidence must connect their required work to the claimed endpoint.

The Best Practices Guide describes event timing on the GPU clock. A CPU duration and an event duration do not provide a shared absolute timeline. Their values alone cannot locate the markers relative to host return or establish how much their intervals overlap.

References: [N1][N3]

CT-231: three reports for one identified operation

Fictional CT-231 identifies one operation with fixed inputs and a required completed output. Offer A supplies three stipulated reports. Every value is invented; the example supplies durations and endpoint descriptions, without a correlated execution trace.

The host report is 0.3 ms from the beginning of enqueue work to return to the calling thread. The device report is 8 ms between two completed timing markers around a selected stream region. The caller report is 12 ms from a declared request start to the required output becoming available to that caller.

These reports can all describe their stated intervals without disagreeing. The 0.3 ms report answers the submission-return question. The 8 ms report answers the selected device-region question. The 12 ms report answers the declared caller-completion question. None can silently replace another on an offer comparison.

Adding 0.3 ms and 8 ms does not establish a total: the example has not supplied the relative placement, overlap or full coverage needed for that calculation. Subtracting 8 ms from 12 ms does not turn the difference into measured overhead. Even where a narrower region sits within a caller interval, a residual difference does not identify the activities responsible for it.

Likewise, 8 / 0.3 is not a speedup. It divides durations with different endpoints rather than comparing two implementations reaching the same finish. The example identifies no alternative implementation that completes the operation in 0.3 ms.

Offer B then quotes only 0.2 ms from enqueue start to host return. Its smaller host value supplies no evidence that useful output becomes available sooner. B has left both the selected device-region duration and the complete caller interval unstated.

Compare offers at the finish the buyer needs

If the decision concerns caller response, ask B for the same declared request start and required output-availability endpoint as A's 12 ms report. Keep the operation, input population and output acceptance criterion equivalent. If one offer stops when a result remains on the device and the other includes making it available to the caller, resolve that difference before ranking the values.

If the decision instead concerns the selected device region, compare equivalent included work, marker placement and dependency coverage, with completion established for each reported interval. A shorter region produced by leaving out a required transfer answers a narrower question. Retain the GPU and system configuration, CUDA and application versions, workload state and competing load so the reader can see which offered arrangements were observed.

For a host-submission decision, the 0.3 ms and 0.2 ms values become relevant once the suppliers establish comparable enqueue boundaries and operating conditions. Their comparison concerns the elapsed host interval. Choosing B for earlier useful output still requires B's missing caller-completion evidence.

This separates a useful partial finding from an unsupported purchase conclusion. A can be assessed against a caller requirement using its stipulated 12 ms report. B's smaller host-return number leaves that same requirement unanswered. Neither a larger device number nor a smaller host number is inherently the better runtime.

Keep preparation and the complete workload visible

A completed event interval can answer a narrow question well while omitting costs that matter to the buyer's job. Identify whether preparation, warmup, input handling and result processing fall before, inside or after the supplied measurement. When the requirement begins before those stages, a later starting marker leaves part of that requirement unmeasured.

A repeated-operation result also needs the workload population it represents. A selected prepared iteration does not establish the elapsed time for an entire short job that includes preparation and several operations. Conversely, a complete short-job duration cannot identify one operation's device interval without corresponding observations.

The related model-warmup article covers readiness and first-response boundaries. The CUDA graph article explains recurring host launch work and captured-workload coverage; the pinned-memory article covers transfer staging and destination completion. Use those owners for their mechanisms. Here the decision is whether each supplied interval reaches the finish being compared.

Ask how observation changed the execution being described

The Best Practices Guide notes that CPU-to-GPU synchronization points can stall the GPU processing pipeline. Establishing an isolated timing can therefore change how execution proceeds. A result collected with additional waiting may describe a different overlap pattern from the offered application's ordinary operation.

Have the software supplier explain whether the observation introduced waiting or isolation and which operating condition the result represents. Evidence for the constrained run can still be useful, provided its conclusion stays attached to that run. It does not automatically establish response time under the buyer's normal concurrent workload.

References: [N1]

Carry the selected system into a separate physical project

Once the appointed platform and application owners have selected the system, Leviathan's Complete Physical Infrastructure Deployment service is relevant if the equipment also needs receiving, rack integration, onsite installation, IT cabling or physical handoff records. Use the service and project-inquiry links below with a nonconfidential equipment summary, site, target window and receiving team.

This article provides software-measurement education. Leviathan's physical deployment scope does not imply a profiling, tuning, benchmarking or software-commissioning service, and the timing discussion supplies no field-work approval. Keep the software performance evidence with the owners responsible for that result while defining the separate physical installation package.

References: [L1]