A Model File Can Be Mapped Before All Its Tensor Pages Are Resident

A file mapping can give tensor data an address before every page is resident

Tensor storage holds the underlying data used by a tensor. A file-backed mapping connects that storage to a region of the file through virtual addresses. Mapping the region and having every corresponding byte present in physical RAM are different conditions.

PyTorch 2.14's torch.load reference describes memory mapping as replacing the initial disk-to-CPU storage copy with lazy access to tensor data. Linux's current memory-management concepts explain demand paging: virtual memory allows only needed information to occupy physical memory. The Linux explanation here concerns systems with a memory management unit.

The resulting question is precise: at the reported loading milestone, which data has merely become addressable and which is already resident? Establishing the mapping does not shrink the artifact, reduce its parameter count or establish the pages needed by the next stage.

References: [ML1][ML3]

Module construction and parameter population can bring the deferred access forward

A loaded state dictionary and a module ready to execute are separate objects in the loading account. PyTorch's checkpoint-loading tutorial distinguishes reading the saved state, constructing the module and populating its parameters. Ordinary module construction can allocate and initialize parameter storage; loading the state commonly copies saved values into that storage. Those source reads can require mapped data before any inference request arrives.

The tutorial also describes tensors carrying metadata without data and a different parameter-assignment mechanism. These are distinct construction and population choices, rather than consequences of file mapping alone. No particular combination is prescribed here.

For a supplied result, ask whether its stopping event is the returned checkpoint object, the completed module or a later application milestone. A report ending before parameter copying has left a different amount of work outstanding from one ending after it. Additional parameter storage also makes mapped-payload residency an incomplete account of host memory.

References: [ML2]

The CPU-storage stage does not cancel the device destination

PyTorch 2.14 documents CPU deserialization of storage followed by movement to the saved device or an explicitly remapped destination. The CPU destination makes that second stage unnecessary; a GPU destination can require it during the loading call. Mapping therefore does not imply that every saved-device path leaves transfer until inference.

For a proposal that retains CPU-backed storage first and moves tensors to a GPU later, identify that later stage explicitly. PyTorch 2.14's Tensor.to reference describes a changed device as producing a copy. Destination allocation and data movement remain work to account for, alongside completion evidence before dependent GPU work.

A host mapping supplies no GPU-memory fit result. If the proposal changes numerical representation during placement, its destination payload needs its own stated size. Do not silently carry a host-file byte total into that different representation.

References: [ML1][PT2]

ML-216: 4 GiB mapped, 1 GiB resident, then 4 GiB resident

Fictional example ML-216 stipulates a mapped tensor-payload region of exactly 4 GiB. At the first observation, 1 GiB of that payload's pages is resident in host physical memory and 3 GiB is not resident. These are invented states, not observations from a loader. Here, resident counts payload physically present in host RAM, including file-backed page-cache data. It is distinct from process RSS and from whether a process page-table entry is already present. The payload account excludes file headers, object metadata, page tables and other allocations.

Use binary units: 1 GiB is 1,073,741,824 bytes. The mapped payload is 4,294,967,296 bytes; its initially resident portion is 1,073,741,824 bytes; its nonresident portion is 3,221,225,472 bytes. The first state reconciles as 1 GiB + 3 GiB = 4 GiB.

Now stipulate that the example's later work uses the entire payload, makes all 4 GiB resident and incurs no eviction between these observations. The increase in raw resident payload is 4 GiB - 1 GiB = 3 GiB, or 3,221,225,472 bytes. The mapped extent and stored parameters remain unchanged.

This is a comparison of resident states. It gives no measured disk-read count, fault count or startup duration, and no total process or host-memory figure. Full eventual use is an explicit assumption of ML-216. A different application may never access some artifact tensors; the existence of a mapping alone cannot decide its eventual working set.

Resident growth cannot substitute for an I/O record

Linux's page cache can retain file data for reuse. An access fault can therefore be resolved using data already in host memory, while readahead can bring in data beyond the immediate request. Reclaim can later remove cached pages. A fault satisfied from existing page-cache data need not increase host-resident payload. These mechanisms prevent a one-fault, one-disk-read interpretation.

ML-216 deliberately fixes two resident states and excludes eviction between them. It does not establish the storage history that produced those states. Calling its 3 GiB increase a measured SSD read would add an observation the example never supplied. In an actual evaluation, identify the storage boundary and the interval used for any read-traffic claim.

Likewise, a real mapping's resident size can change as the application proceeds and other work competes for RAM. Repeated loading can reuse cached data without demonstrating an equally fast uncached start. The related page-cache guide addresses those storage-accounting boundaries in detail.

References: [ML3][ML4]

Compare offers at the same required readiness stage

Consider fictional reports A and B for the same identified model revision, hardware and numerical representation. A stops its loading clock when a CPU-backed checkpoint mapping is available. B stops after the required module population and GPU placement have completed. No durations or test results are stipulated. Even if A reports a shorter interval, those finish events cannot establish which offer reaches the buyer's required state sooner.

If the requirement is an acceptable first application response, continue both accounts from the same agreed start through that response. Include any deferred page access, construction, population and placement needed on the actual path. These stages may overlap; separately named stages should not be added as though they were necessarily serial.

Then identify the representative work that must remain acceptable after that first response. One input may leave other required paths unexercised. Conversely, representative coverage does not require inventing a rule that every artifact tensor must be touched. Define the relevant inputs and model behavior with the application owner.

Keep cold and reused conditions explicit: prior artifact access, relevant page-cache state, retained application state and competing host work. Ask for evidence spanning the required interval, rather than treating the word loaded as a common finish line. The related warmup article covers readiness clocks and first-request versus steady-state results; this article locates the file-backed CPU work those clocks can omit.

A low early resident count leaves peak capacity unanswered

A result showing ML-216's initial 1 GiB resident payload would describe that observation only. It would not establish that the complete proposed application can operate within 1 GiB of host RAM. In the stipulated full-use comparison, this payload alone later reaches 4 GiB.

For an actual capacity claim, require coverage through the stages that matter to the application. Identify whether the memory evidence includes mapped file-backed pages, separate module parameters, temporary copies and other host allocations, alongside the separate device account. State what each counter includes so overlapping views of the same backing are not added twice.

A point-in-time resident observation and a peak over a stated interval answer different questions. A peak ending at loader return can omit a later population or placement peak. A steady-state device-memory figure can omit a loading peak. Neither a mapped extent nor a low initial resident count repairs that missing coverage.

The useful decision is whether the supplied startup and memory evidence spans the readiness and workload the buyer actually needs. Mapping can be part of an efficient loading arrangement, but its benefit must be assessed at that complete boundary.

References: [ML2]

Keep artifact assessment and physical equipment work with their owners

Memory mapping changes data access; it provides no safety boundary or permission to deserialize an artifact. The related checkpoint-deserialization guide explains that assessment. This article explains a loading claim and supplies no loading or configuration procedure.

If the separately selected hardware needs receiving, preparation or rack assembly, Leviathan's Rack Assembly & Preparation service and project inquiry below address that physical package. Share a nonconfidential equipment summary, approved rack arrangement, proposed location and required handoff. The appointed platform and application owners retain software implementation and workload acceptance.