A GPU Server’s Total RAM Does Not Describe NUMA-Local Memory

State which CPU resources need which host memory

A quote reading 512 GiB host RAM answers an aggregate capacity question. If the buyer needs a host-resident allocation close to particular CPU resources, the quote must also describe the relationship between execution and memory. The GPU count does not supply that missing relationship.

Write the requirement around the actual application stage. For example, a buyer might specify that one CPU-side preprocessing stage needs 350 GiB of distinct resident host pages wholly local to the CPU resources assigned to that stage. Identify those resources and what the buyer means by local. Here, an execution domain means that stipulated group of CPU resources; it is not an assumption that every process, socket or operating-system node forms the same boundary.

Ask the application owner whether wholly local residence is a firm requirement or a proposed way to achieve a timing target. Those lead to different comparisons. A firm locality requirement needs evidence that the layout and placement satisfy it. A timing target needs application evidence; local-capacity arithmetic alone cannot select the faster offer.

Separate execution location, page residence and addressability

NUMA means non-uniform memory access. CPU cores can share an address space while memory access differs between local and remote locations. Intel’s historical architecture explanation establishes that distinction; its older-system experiment does not establish performance for either offer here.

Track three things: where the CPU work executes, where its host pages reside, and which paths connect them. Reaching another domain’s memory can involve a system interconnect instead of the CPU group’s local memory path. The same address can refer to memory that is local to one CPU group and remote from another. Addressability therefore does not answer the locality question.

The Linux kernel’s NUMA primer distinguishes physical resources from the operating system’s node abstractions. It also explains that moving a task can leave it remote from its previously allocated memory. Placement evidence must therefore cover the relevant execution period, rather than treating an allocation-time description as a permanent relationship.

References: [NM2][NM3]

Read the actual domain map instead of counting sockets

Do not divide installed RAM by socket count and call the result local capacity. Linux 6.18’s resource-control documentation describes Sub-NUMA Cluster nodes sharing an L3 cache. The kernel’s NUMA primer also describes nodes without memory and emulated nodes. These are reasons to obtain the offered layout and its physical interpretation rather than assume one NUMA node per socket or identical paths within every reported node.

For each alternative, request the exact server and processor configuration, the supported memory arrangement, and the CPU-to-memory relationships exposed in the proposed firmware and operating-system environment. Have the platform owner identify which reported resources correspond to the buyer’s execution domain and which capacity is local under the agreed definition.

Keep installed capacity per domain separate from capacity actually available to the application there. A layout drawing establishes an arrangement. It does not report how much usable memory remains under the intended concurrent load.

References: [NM3][NM4]

Work the 350 GiB requirement through two 512 GiB layouts

Consider two invented host-memory layouts. Stipulate that both complete systems and their described arrangements are supported. Layout A has two separate memory domains, each with 256 GiB local to its own CPU resources. Layout B has one domain with 512 GiB local to the CPU resources needed by the stage. GiB is used consistently throughout this example. These are comparison assumptions, not specifications for a named server.

The buyer requires 350 GiB of distinct resident host pages wholly local to one execution domain. In A, the stage executes within one of the two domains. Its largest possible local capacity under the assumptions is 256 GiB. Adding the other domain gives 256 GiB + 256 GiB = 512 GiB across the server, but leaves that local ceiling unchanged. A fails the stated wholly local capacity screen whichever of its two domains is chosen.

Suppose all 350 GiB is allocated and resident across A’s assumed domains, with 256 GiB available locally to the executing CPU group. At most 256 GiB can be local, so at least 350 GiB − 256 GiB = 94 GiB must be nonlocal to that group. If less local capacity is available, the nonlocal portion must be larger. This is a placement bound for the stipulated allocation, not an out-of-memory prediction, a count of memory traffic or a latency estimate.

B’s 512 GiB domain exceeds the 350 GiB requirement by 162 GiB on gross capacity arithmetic. That difference is not a measured operating margin. Other allocations, usable capacity and actual placement still need confirmation. B clears this initial screen; it has not demonstrated application fit or performance.

If A’s supplier proposes dividing the stage between both CPU groups and placing each group’s data beside it, assess that as a revised application arrangement. It may change the requirement and access relationships. It does not show that the original 350 GiB allocation was wholly local to one domain.

Check placement evidence beyond a local preference

Linux 6.18’s NUMA memory-policy documentation describes allocation policy in terms of nodes. Its preferred policy can fall back to other nodes, and local allocation is a preference starting from the allocating CPU’s node. Administrative cpuset restrictions take priority over memory policy. A local-preference statement therefore cannot certify the buyer’s wholly local allocation.

Request evidence of the relevant CPU execution locations, allowed memory resources and resident-page distribution for the defined stage. Ask the platform and application owners to explain how the proposed placement meets the stated requirement over that stage’s execution. A whole-process memory total omits the distribution; a CPU assignment alone omits the pages’ residence.

For the fictional offers, A’s supplier must resolve the local-capacity shortfall or propose an explicit requirement change. B’s supplier must still establish usable local capacity and the intended placement. Apply the same evidence request to both, with the exact configuration, software versions, allocation size and concurrent load identified. If the purchase also has a response-time or completion target, assess that result separately.

References: [NM1]

Use the right evidence for the neighboring hardware questions

The related DDR5 population article compares channels, DIMMs per channel and operating rates. Those fields help describe the memory population but do not establish that the required allocation is local to the executing CPU group. The CXL article examines added host capacity and memory presentation; the GPUDirect RDMA article examines GPU-to-NIC attachment. Keep those decisions alongside this one rather than using any of them as a substitute for the host-locality answer.

For each offer, record the configuration, execution domain, usable local capacity, required resident allocation and placement evidence. Then state whether the supplied layout excludes the requirement, leaves it unresolved or has supporting evidence. For A, the fictional arithmetic excludes the original requirement. For B, the arithmetic leaves a candidate to investigate. Equal whole-server RAM is not a reason to label the offers equivalent.

Carry the selected system into a defined physical project

Once the responsible technical owners have selected a supported complete system, use its configuration identity in the physical project brief. Leviathan’s Rack Assembly & Preparation service and project inquiry are linked below for receiving, preparation or agreed rack assembly around that selected equipment.

Share a nonconfidential outline of the supplied system, approved configuration, delivery state, proposed location, permitted physical tasks and required handoff records. Identify what is factory-complete and what remains to be done. NUMA placement, firmware or operating-system configuration, and application acceptance remain with the appointed technical owners; this article explains an offer comparison and provides no configuration procedure or work approval.

References: [L1]