A Unified Memory Allocation Does Not Put Every Byte in GPU-Local Memory

Follow the logical allocation to its backing pages

When an offer says a workload can use 96 GiB through Unified Memory, first establish what the number describes. Is it the logical allocation size, the portion resident on a particular GPU, or a capacity claim about the complete machine? Attach the accessing processor and the relevant application stage to the answer. Without those boundaries, the same number can appear to answer several different questions.

Virtual addresses are mapped to physical backing through pages. NVIDIA's current guide separates these virtual and physical entities. Its full-support allocator table lists cudaMallocManaged with first-touch/hint placement and CPU/GPU accessibility; other allocator rows have different placement and migration semantics.

The buyer's useful distinction is between the application's logical range and the resources backing that range at a stated instant. An allocation report supplies the former. To establish local residence, request evidence identifying how much of that allocation is backed by memory local to the specified GPU. Keep that evidence tied to the same allocation and time.

References: [N1]

Distinguish page migration from access to remote backing

Migration moves backing pages between memory resources. Remote access lets a processor reach backing held elsewhere through a supported path. Both can make data usable, but only the first necessarily changes the pages' residence.

NVIDIA's October 5, 2021 oversubscription article illustrates on-demand migration: a GPU access faults on host-resident data, prompting migration across the CPU-GPU interconnect. Under memory pressure, its illustrated sequence also evicts GPU-resident pages to make room. The current programming guide separately demonstrates GPU access to CPU-resident data without first making that backing GPU-local.

Therefore, a statement that the GPU accessed the allocation leaves a placement question open. Ask whether the supplied evidence concerns local accesses, page movement, supported remote accesses or a mixture during the relevant stage. The path must belong to the proposed environment; the existence of one documented example does not establish that every Unified Memory system behaves that way.

References: [N1][N3]

Identify the supported Unified Memory arrangement

NVIDIA's current Unified and System Memory chapter, checked October 20, 2026, distinguishes full support for explicit managed allocations from full support that also covers system allocations. The latter can use software or hardware coherence. Full support for managed allocations alone does not mean every ordinary host allocation is GPU-accessible.

The chapter identifies Windows, WSL and some Tegra systems with limited support. Its limited-support description excludes GPU-memory oversubscription and constrains concurrent CPU/GPU access. Do not carry the full-support capacity example below into that paradigm.

For an actual comparison, obtain the exact GPU and complete system, operating system, relevant Linux kernel where applicable, driver and CUDA versions, CPU-GPU interconnect, and allocation mechanism. Ask the platform owner which documented paradigm applies and what supports that conclusion. A product generation or a Unified Memory label alone leaves the arrangement unresolved.

This is a memory-system comparison. Allocation implementation, software configuration and validation belong to the appointed platform and application owners; the article supplies no operating procedure or hardware approval.

References: [N2]

UM-232: 96 GiB logical, 64 GiB local and 32 GiB on the host

Fictional example UM-232 stipulates a supported full-Unified-Memory environment, sufficient host backing, one GPU with 80 GiB of total local memory, and one 96 GiB logical allocation. At one stipulated instant, every byte of that allocation has physical backing, with no duplicate page copies: 64 GiB resides GPU-locally and 32 GiB resides on the host. These are invented conditions, not a named product's usable capacity or a recommended allocation.

The allocation reconciles as 64 GiB + 32 GiB = 96 GiB. Its observed GPU-local portion is 64 GiB. Calling the entire 96 GiB GPU-local would assign the host-resident portion to the wrong resource. Adding the 96 GiB allocation to the GPU's 80 GiB total to claim 176 GiB local capacity would combine a logical allocation with a physical capacity that already contains its local portion.

Now give the buyer an explicit requirement: all 96 GiB of that allocation must reside locally on this one GPU at the same time. The stipulated 80 GiB gross physical capacity fails that screen before accounting for any other allocations. The successful 96 GiB allocation does not remove the shortfall.

An implementation that uses host backing and processes data out of core is a different proposal. It needs its own supported access arrangement and workload evidence. The example neither predicts an allocation failure under those revised conditions nor demonstrates an acceptable runtime.

An unchanged logical size leaves later residence unobserved

At a later stage in UM-232, suppose the application accesses another subset of the same 96 GiB allocation. The logical size remains unchanged. No later page-residence observation or transfer record is supplied.

Those facts do not establish how much remains GPU-local, which pages move or how long the stage takes. In particular, the earlier 32 GiB host-resident portion is not a measured count of bytes that must now cross the interconnect. The new accesses might involve only part of that portion, and their supported handling has not been observed.

A residence snapshot describes a state. A movement record describes activity over an interval. Even if a later snapshot showed the same local and host totals, those equal totals would not establish that the same pages stayed in each place or that no transfers occurred. Conversely, the unchanged logical allocation supplies no evidence that migration occurred.

Keep the allocation identity, the residence observation and the access interval together when reviewing a report. This prevents a capacity total from being presented as a transfer measurement or a timing result.

Compare usability evidence with evidence for the actual workload

Consider two fictional proposals based on UM-232. Proposal A establishes that the named supported arrangement can allocate and correctly access the logical range. It supplies no representative timing result. Proposal B also supplies results for the buyer's intended application stage, data size and access pattern, with the hardware, software versions and concurrent load identified.

A can address a supported-usability requirement. It leaves a completion-time or response-time requirement unanswered. B provides more relevant evidence for that second decision, but only if its conditions represent the proposed use and its result meets the buyer's stated target. Neither proposal can satisfy the original requirement for 96 GiB simultaneously local on the stipulated 80 GiB GPU.

NVIDIA's 2021 study compares sequential and random access patterns on specified V100 and A100 systems. Its useful lesson here is that access pattern, residency and platform affect oversubscription performance. Its historical microbenchmark results supply no speed forecast for UM-232 or a current offer.

Ask the application owner to identify the stages that must meet the target and explain the access behavior represented by the evidence. Separate allocation success, correct access, observed residence and the measured application outcome in the comparison. Larger addressability and fewer manually managed copies do not by themselves establish a faster job.

References: [N3]

Keep host capacity, CPU locality and allocator accounting in context

Use the related CXL guide when the offer adds host-memory capacity. That changes the host-side resource being offered; it does not turn all managed backing into GPU-local memory. Use the NUMA guide when a CPU-side stage depends on host pages being local to particular CPU resources. CPU locality and GPU-local residence are different relationships.

The PyTorch allocator guide addresses another reporting boundary: tensor occupancy can sit inside an allocator-managed pool. It helps reconcile overlapping totals, but those totals do not replace the resident-page evidence required for this managed allocation. Keep the workload requirement explicit as these accounts are brought together.

For UM-232, the resulting decision is precise. The all-local requirement is excluded by gross capacity. A supported out-of-core implementation remains a separate candidate whose access behavior and performance need evidence. The shared pointer and successful allocation do not choose between those requirements.

Carry the selected equipment into a physical project brief

Once the responsible owners have selected a supported complete system, any receiving, preparation or rack-assembly requirement can be scoped around that equipment. Leviathan's Rack Assembly & Preparation service and its project inquiry are linked below.

Share a nonconfidential outline of the exact supplied configuration, delivery state, proposed location, permitted physical tasks and required handoff records. Identify what is already factory-integrated and which team receives the prepared or assembled package. That gives the physical work a defined input and output while the appointed software owners retain the memory implementation and workload acceptance decisions.

References: [L1]