Who Can Observe a CUDA Atomic Update—and What Else Does It Order?

Identify the object, the participants and the ordering

Suppose a runtime proposal says a shared atomic counter makes results available to the GPU and CPU. The buyer needs to know what results available means. Does the evidence establish an indivisible counter update, or does it establish that the intended consumer can safely use the associated data?

Start with the object: one identified scalar integer counter, updated through a supported read-modify-write operation. Its indivisibility applies to that operation on that object. A separate payload remains a separate object. Next identify the participating threads and their scopes. Finally identify the ordering relationship claimed between the payload accesses and the counter accesses.

These questions prevent an atomic label from silently expanding into a complete communication guarantee.

References: [N2]

Read scope as a relationship between participating threads

Block scope relates threads in the same CUDA thread block. Device scope relates GPU threads on the same CUDA device and within the same memory synchronization domain. System scope describes the wider relationship that can include CPU and GPU threads, subject to the applicable atomicity conditions.

Scope does not report which readers actually observed an update or when they did so. Nor does access to a common address establish the required relationship between concurrent operations.

NVIDIA documents memory synchronization domains from CUDA 12.0 and compute capability 9.0, the Hopper architecture. Domain selection does not change which memory a kernel may legally access. Consequently, same GPU is incomplete evidence for device-scope synchronization when work occupies different domains. Ask the platform owner to identify the domain relationship represented by the claim.

References: [N4][N3]

Two 32-thread blocks and one CPU thread define three different populations

Consider an abstract population containing block A with 32 GPU threads, block B with 32 GPU threads, and one CPU thread. Stipulate that both blocks belong to one CUDA device and the same memory synchronization domain. This is a hypothetical population, not a reported execution.

For an operation issued in block A, the block-scope population contains its 32 threads, including the issuer. Block B's 32 threads and the CPU are outside that relation. Under the stipulated device and domain, the device-scope population contains 32 + 32 = 64 GPU threads. The CPU remains outside it. The system-scope population contains 64 + 1 = 65 candidate threads, with actual atomicity still dependent on support and the participating operations.

The arithmetic counts candidate participants. It counts neither atomic instructions nor memory transactions, completed updates, successful synchronizations or readers that actually observed a value.

For a buyer requiring a handoff from A to B and then to the CPU, a demonstration confined to A leaves both later relationships unestablished. A demonstration covering both GPU blocks still leaves the CPU relationship to be explained. Expanding the population changes the claim that needs evidence.

References: [N4]

System scope still needs the right memory and platform support

For the scalar read-modify-write counter considered here, NVIDIA's current guide identifies distinct system-scope atomicity routes: system-allocated memory with pageableMemoryAccess equal to 1; managed memory with concurrentManagedAccess equal to 1; or mapped memory with hostNativeAtomicSupported equal to 1. Its system-allocation route also has a host-page-table qualification for memory-mapped-file and hugetlbfs allocations.

Separate mapped-memory provisions for naturally aligned loads and stores do not establish support for a counter increment. The guide's GPU-memory alternatives require GPU-only access, so they cannot establish a CPU/GPU counter claim.

Have the platform owner connect the actual operation, object type and alignment, allocation mechanism, CPU/GPU arrangement and CUDA environment to the applicable documented conditions. A shared allocation label or system-scoped name cannot substitute for that connection.

References: [N1]

A scope mismatch can invalidate the program

NVIDIA's message-passing example changes a producer's atomic flag operation to block scope while its consumer remains in another block with device scope. The producer's scope excludes the consumer; the documented concurrent conflicting accesses create a data race and undefined behavior.

The wider scope on one side does not repair the other side's exclusion. This is not a valid program with a promised slower or predictably stale result. A proposal describing such a race as an acceptable latency tradeoff has left correctness unresolved.

References: [N4]

Legacy relaxed atomics are only one part of the current API

The current CUDA Programming Guide distinguishes legacy functions such as atomicAdd from extended CUDA C++ atomics, standard-style atomics and compiler built-ins. Legacy functions support relaxed memory ordering and do not introduce synchronization fences. Their naming distinguishes device scope without a suffix, block scope with the block suffix, and conditional system scope with the system suffix.

That restriction does not apply to every CUDA atomic. The guide describes cuda::atomic and cuda::atomic_ref with C++ atomic semantics and explicit thread scope. Compiler built-in atomic functions, available since CUDA 12.8, also expose memory-order and scope choices. Stronger ordering can therefore be expressed by current forms.

Request the actual family, operation, scope and memory order behind the supplier's claim. Neither atomic nor system-scoped supplies all four answers. A broader scope alone does not turn relaxed ordering into payload publication.

References: [N2][C1]

An observed counter update does not automatically publish the payload

In a separate hypothetical case, one GPU producer has a payload object P and a counter C initially equal to zero. The producer writes a new payload and then increments C in its own program sequence. A consumer in the other GPU block atomically reads C and obtains one. Stipulate supported memory, scopes covering both participants, and relaxed ordering for the counter operations. No other synchronization relationship is supplied.

The supplied counter observation concerns C. It does not establish that the consumer can safely read the new P. The C++ ordering rules distinguish relaxed atomic indivisibility from synchronization of other memory actions. Merely placing the payload write earlier in the producer's sequence does not supply the missing cross-thread ordering.

If P has ordinary non-atomic accesses and the producer's write and consumer's read are potentially concurrent conflicting actions without a happens-before relationship, there is a data race. The example therefore does not promise an old payload as a defined fallback. It establishes the absence of a handoff guarantee, without reporting a measured wrong output.

The buyer's requirement should identify which payload generation must be usable after which consumer observation. Evidence limited to C reaching one cannot answer that requirement. The responsible software owner must explain the complete relationship.

References: [C1][C2]

Match the benchmark to the communication being bought

A fictional offer A reports rapid counter updates among threads within one block. Offer B explains its supported memory arrangement, participating GPU and CPU threads, scopes and the ordering required for its payload handoff, with validation tied to that arrangement. Neither offer is an actual benchmark reported here.

A can be relevant to the cost of its stated atomic-update operation. It leaves the broader handoff requirement unanswered. B addresses more of that requirement, but its evidence still needs to represent the buyer's application and selected system. A favorable update rate cannot resolve a missing correctness relationship, and a correct handoff alone supplies no throughput forecast.

Keep the adjacent decisions separate. The related Unified Memory guide addresses logical allocation and physical residence. The NVMe atomic-write guide addresses bounded storage updates, persistence and application transactions. Here the decision concerns runtime participants and memory ordering. None of those neighboring meanings of shared or atomic can settle it.

Bring a separate equipment requirement to the physical project inquiry

If the selected system also needs receiving, preparation or rack assembly, use Leviathan's Rack Assembly & Equipment Preparation service and the project inquiry linked below.

Share a nonconfidential equipment summary, approved configuration, supplied delivery state, proposed location and requested physical tasks. Identify the team receiving the prepared or assembled package and the physical records it needs. That brief supports a discussion of the equipment work while the appointed platform owner retains the synchronization implementation and software acceptance.

References: [L1]