CUDA Local Memory Is Private to a Thread, Not Necessarily Close to It

Identify what local is local to

A proposal can use local in two different relationships. GPU-local HBM describes memory associated with a GPU rather than host or remote backing. CUDA local memory describes storage private to an individual thread. The first relationship does not identify the second address space.

NVIDIA's PTX ISA 9.4 reference distinguishes per-thread registers from addressable per-thread local storage. Registers are a limited, fast resource; local storage is typically standard memory with caching. Sharing scope and storage characteristics are separate properties.

Consequently, private workspace is an incomplete description of location. It identifies whose values are being retained, but leaves open whether they occupy registers, local space or some other explicitly identified representation. The useful question is which resource the actual implementation uses for that workspace.

References: [CM1][PT1]

A source variable does not reserve a particular storage resource

The archived CUDA 13.1.0 Programming Guide says an ordinary automatic device variable generally resides in a register, while allowing compiler placement in local memory. The accompanying Best Practices Guide identifies large structures or arrays and arrays with dynamic indexing as likely local-memory candidates.

Register pressure supplies another route: values that cannot remain in the available registers can spill. Local storage therefore does not imply that every variable using it first had a physical register and was then displaced. An addressable private array and a register spill can lead to local accesses for different reasons.

Compilation for the target matters, including later compilation stages. Counting declarations or assuming every automatic array spills cannot establish a kernel's allocation. The distinction is between the values expressed by the program and the storage selected for the resulting implementation.

References: [CM1][CM2]

Device-backed local memory can still be served through caches

The legacy Programming Guide places local space in device memory and states that local accesses are cached in L2 on compute capability 5.x and newer. It also describes local-memory organization that permits coalesced access when warp threads use the same relative address in their respective private objects.

Those threads still access different private objects. Grouping their accesses for service does not merge their values into one shared variable. Likewise, a cached value remains associated with local space; a cache hit does not turn the variable into a register.

This explains why neither universally uncached nor register-fast is a sound interpretation of local. A storage label leaves the actual cache behavior, traffic and elapsed time unresolved. Count bytes at the reported boundary before treating them as HBM transfers.

References: [CM2][PT1]

Block-shared workspace has a different ownership contract

Ordinary block-shared memory is on chip. A shared object is instantiated for a block, is available to its threads and has that block's lifetime. The CUDA guide separately documents supported cluster access to other blocks' shared memory; that facility does not make thread-local space shared.

PTX also describes function-scoped local storage on the stack when using its application binary interface. The word local therefore supplies no promise that a function's temporary object persists beyond its lifetime or into another kernel invocation.

For workspace accounting, distinguish one object used by a block from separate private objects used by its threads. Merely replacing one label with the other leaves the number and meaning of the retained values unsettled.

References: [CM1][CM2][PT1]

CM-305: private value payload is not a compiler allocation

Stipulate one logical block with 64 simultaneously active logical threads. Each retains 16 distinct private four-byte values at the instant considered. These are invented logical conditions, without a compiled kernel or a named GPU's resource allocation.

Each thread's stipulated payload is 16 x 4 = 64 B. Across the 64 threads it is 64 x 16 x 4 = 4,096 B. This calculation counts 1,024 private value slots. It establishes neither 4,096 B of local-memory allocation nor 1,024 allocated hardware registers.

Now separately stipulate a single block-wide array containing 16 four-byte values. That array's value payload is 16 x 4 = 64 B. It contains 16 slots shared by the block, rather than 16 independent slots for each thread. Dividing 4,096 B by 64 B gives a ratio of 64 between the stipulated payloads. It supplies no speed ratio and does not establish two equivalent programs.

If each thread must preserve its own different intermediate results, one common set of 16 slots has not represented all 1,024 independent slots. Conversely, if all threads need one common set, assuming 64 private copies describes a different logical requirement. The sharing contract must be established before comparing the byte counts.

CM-305 leaves compiler elimination, placement, padding and object lifetimes unresolved for any real implementation. Its simultaneous logical values also do not determine how much backing a runtime reserves. Use the arithmetic to expose an accounting assumption, then obtain implementation-specific resource evidence.

CUDA 13.0 introduced a specific exception for register spills

NVIDIA's August 27, 2025 article documents a CUDA Toolkit 13.0 feature that can use shared memory as register-spill backing. In that release it is opt-in and disabled by default. When enabled, the compiler prioritizes shared memory for spills and falls back to local memory for remaining spills when shared space is insufficient.

The feature consumes shared memory and has constraints. The article specifies whole-program compilation and excludes kernels using dynamically allocated shared memory or dynamic register reallocation across warps. Its existence is not evidence that a particular kernel uses it.

This compiler-managed backing for private values is different from CM-305's separately stipulated common application array. The exception is why all spills always go to off-chip local storage is too broad. Identify the toolkit, applicable compilation arrangement and actual backing in the evidence supplied for the kernel.

References: [CM3]

More HBM does not identify the kernel's on-chip resources

Consider a fictional supplier response: the proposed GPU has more HBM, so the kernel's private workspace will fit on chip. The response names a device-memory capacity without identifying the claimed register or shared-memory requirement. The inference remains unsupported even if the HBM specification is correct.

Registers and shared memory are finite resources used by work resident on a streaming multiprocessor, or SM. Their availability and allocation constraints are distinct from total device-memory capacity. A larger HBM figure alone cannot establish the kernel's register placement or how many blocks can reside together.

Ask the platform owner to tie the claim to the identified GPU, kernel build, CUDA version and block configuration. Retain each quantity's definition and units: logical value payload, compiled storage requirement and traffic accumulated during execution answer different questions. A local-memory field alone cannot establish a bottleneck or its contribution to application time.

The related occupancy guide explains resident-work limits; the Unified Memory guide addresses GPU-local page residence. Use the workload benchmark guide when the purchase decision depends on completed output or elapsed time. A workspace explanation helps interpret that evidence but does not supply a performance result.

References: [CM1]

Carry a separate installation need into the physical project brief

If the selected system also needs physical installation, Leviathan's rack-and-stack service covers agreed equipment mounting, IT cabling, labeling and installation records. Use the related service and project inquiry with a nonconfidential equipment list, site, released rack and cable plans, target window and required physical handoff.

That brief gives the installation discussion concrete inputs. Kernel implementation and application-performance acceptance remain with the appointed platform and application owners.

References: [L1]