CXL Memory Regions: Follow the Address Map Behind the Capacity

Start with the memory the host can actually address

A CXL expansion offer can list two devices, their combined installed capacity and a larger platform address window. Those figures can all be correct while describing different resources. The buyer needs to identify the capacity actually mapped for its intended use before comparing the price of usable host memory.

Linux 6.18's CXL theory documentation distinguishes potential root-decoder ranges from regions representing active mapped capacity across the topology. A device inventory therefore cannot replace the active-region description. Nor is a large potential window an additional memory device whose capacity can be added to the installed total.

The existing CXL expansion guide explains host memory versus GPU-local HBM and supported memory presentation. This article follows the next question: once a host-side resource is selected, how does its address map associate the region with the devices behind it?

The discussion uses named Linux 6.18 documentation and one fictional mapping. It does not establish support for a specific server, change a decoder or prescribe a memory-configuration procedure.

References: [CM1]

Follow routing and device-local translation separately

Linux 6.18's CXL Driver Operation describes routing decoders and endpoint decoders. In its supported model, root and switch decoders route host physical addresses; endpoint decoders translate them into device physical addressing. A device-local offset and a host physical address can consequently identify related positions without being the same number.

A region can involve multiple endpoints. Interleave ways identifies the participating count, while granularity and target ordering help determine which target receives an address. Settings at an intermediate decoder concern its immediate downstream targets; they should not be read as an inventory of every endpoint in the whole region.

For an offered system, the platform owner must establish the complete supported map. The terms two devices and two-way interleave leave other information unresolved. A topology drawing can show possible paths without establishing the active range or the mapping selected for it.

The example below collapses the routing hierarchy into an explicitly stipulated two-target mapping boundary. That simplification lets us follow addresses without pretending to describe every platform's decoder chain.

References: [CM2]

CM-01: 512 GiB installed,256 GiB actively mapped

CM-01 is an original fictional offer with two devices, A and B. Each contains 256 GiB of relevant volatile CXL memory. Their installed total is 512 GiB. Stipulate a valid, supported platform and one active 256 GiB region to which A and B each contribute 128 GiB. All capacities use GiB consistently.

A separate 512 GiB potential decoder window is also reported. It describes potential address space in this case, not another 512 GiB of media and not proof that the whole installed capacity is active. There is no additional active CXL region in the case.

The active-region account is 128+128=256 GiB. The other 128 GiB on each device remains outside that region. The offer has not established that this remaining capacity is available through some other application interface. Conversely, being outside this one region does not prove it can never be used.

Do not add installed 512, potential 512 and active 256 to claim 1280 GiB. Those figures are not independent stores. Processor-attached DRAM and GPU HBM are outside this account; the 256 GiB result is not the server's total free memory or a GPU-memory upgrade.

The remaining sections examine a tiny slice of the stipulated valid region. They do not propose a 2 KiB physical region, device population or supported allocation size.

Follow a 2 KiB slice through the two targets

Let H0 identify an appropriately aligned host physical base within the region. For the first illustrative map, the ordered targets are A then B and the granularity is 256 bytes. The selected 2048-byte slice has offsets 0 through 2047 relative to H0.

Offsets 0–255 select A;256–511 select B;512–767 select A;768–1023 select B. The pattern repeats over the next 1024 bytes. Each target backs four 256-byte portions, totaling 1024 bytes of the slice.

For this simplified modulo-interleave map, divide the offset by 256, discard the remainder and take the result modulo 2. Result 0 selects A and result 1 selects B. This is the explicit case rule, not a universal formula for arbitrary CXL interleave arithmetic or deeper decoder hierarchies.

In the example, A's contributed device resource begins at its own local offset 0. The second A-selected host stripe, at H0+512 through H0+767, corresponds to its next local 256-byte portion, offsets 256–511. B separately has local offsets 0–255 behind H0+256 through H0+511. Local offset 0 on A and local offset 0 on B therefore name different backing.

The example illustrates association and translation. It supplies no actual physical addresses, device contents or observed traffic. H0 is not an application virtual pointer, and an application allocation has not been shown contiguous across this host physical slice.

The same capacity can carry a different interleave map

Now consider a separate hypothetical supported map with the same two ordered targets and the same region size, but 512-byte granularity. Do not interpret this as a live change to the first map.

Offsets 0–511 select A;512–1023 select B;1024–1535 select A;1536–2047 select B. Each target still backs 1024 bytes of the selected slice. Device count, total capacity, active capacity and equal contributions remain unchanged. The address-to-target relationships have changed.

For example, H0+256 selects B in the 256-byte map and A in the 512-byte map. H0+512 selects A in the first and B in the second. An offer naming only two-way interleave cannot distinguish these results.

The target order matters too. Reversing A and B in the first map reverses which device backs every selected stripe while retaining the same two participants and granularity. A list of device identities considered as an unordered set loses that mapping information.

Linux's documentation describes interleave parameters and target relationships at region and decoder boundaries. These fictional alternatives use those concepts; they do not show that a particular OEM exposes both choices or that a memory region can be remapped while retaining live contents.

References: [CM1][CM2]

Equal region contributions do not establish balanced access

Suppose a purchaser's comparison describes four selected byte addresses:H0,H0+512,H0+1024 and H0+1536. Consider only which target backs each address. This is a query of the stipulated maps, not a claim about four CPU transactions, cache misses or device requests.

Under the 256-byte map, all four offsets select A. The region still draws half its backing from B, but this particular selected-address set does not touch it. Under the 512-byte map, the selections are A,B,A,B.

The contrast explains why two-way interleave alone cannot establish equal workload traffic or double bandwidth. Workload address patterns, the physical backing of its pages, caching, transactions, links and device service behavior all matter to an actual performance result. This case establishes only the map selections.

A sequential range covering the whole 2048-byte slice has equal backing from A and B under both maps. Even that coverage statement supplies no access timing, request count or measured throughput. Byte membership and observed traffic remain different quantities.

For an AI host buying additional dataset or preprocessing capacity, compare the active usable resource first. If the purchase also depends on bandwidth or completion time, request evidence for that particular workload and complete supported configuration. Neither the installed total nor an interleave label supplies it.

Interleaving distributes backing; it does not create a second copy

CM-01 counts distinct capacity: 128 GiB from A plus 128 GiB from B. The illustrated stripes distribute different addresses between the targets. They do not show A and B holding duplicate copies of every byte.

A buyer seeking a protected copy needs the separately supported protection mechanism and its actual coverage. It cannot infer mirroring from two devices in a region. The linked host-memory mirroring guide explains capacity consumed by duplicate copies and designated-page placement.

Likewise, this address map cannot certify behavior after a device or path failure, safe device removal, recovery time or persistence. The case assumes volatile memory and supplies no fault-handling evidence. The complete region's dependence on its contributing resources is a reason to resolve those requirements with the platform owner, not an instruction to test failure or remove a device.

NUMA locality is another question. Knowing A backs an address does not identify which CPU resources are local to it or where a particular application page resides. Use the related NUMA guide for that execution-to-memory relationship.

Compare the offered resource and the result you need

For CM-01, a return showing both installed devices has established an inventory. A return identifying the active 256 GiB region and its participants explains the mapped capacity. A complete applicable interleave map answers the selected-address questions. Workload evidence must then answer any requested usable-capacity, access or timing result. These returns build on one another without being interchangeable.

Ask the supplier and platform owner to identify the offered server, firmware, operating-system version, selected region, contribution from each resource and supported memory presentation. Have them explain the relevant map and result at the boundary the application uses. Root-window capability alone leaves the active-resource question open.

These questions do not require the buyer to select decoder values or treat a public example as a system configuration. They define what the supplier's description must make understandable before the purchase is compared on capacity, price or performance.

If the selected complete system also needs receiving, preparation or agreed rack assembly, Leviathan's Rack Assembly & Preparation service is linked below. Bring the supplier-approved equipment identity, delivery state, permitted physical tasks and installation records. CXL configuration, application integration and qualification remain with the appointed platform and application providers.