An AI Dataset Version Can Be Retained While Its Key Looks Deleted

Separate version identity from current selection

For an AI dataset stored in Amazon S3, a familiar object name can conceal an important distinction. AWS's versioning documentation describes multiple versions sharing one key, with different version IDs. An ordinary read that specifies no version ID selects the current version. A version-specific reference identifies a different relationship: particular stored contents, whether current or historical.

Those references answer different buyer questions. A dataset consumer may need whatever is currently published under a name. A recovery proposal may need the specific contents used for an earlier training run. Recording only the key leaves that historical identity unresolved.

For a dataset spread across several objects, identifying one object's version also leaves the complete dataset release to be established. The object's version identity and the application's grouping of objects are separate subjects.

References: [A4][A5]

State retention at the version boundary

AWS documents S3 Object Lock as a feature of versioning-enabled buckets. Its retention information belongs to individual object versions. Protecting one version does not prevent a new version with the same key or a delete marker from being added.

The mode matters to the protection claim. During an unexpired compliance retention period, users cannot delete or overwrite the protected version or shorten its retention period. Governance mode has different semantics: users with the required authority can bypass retention under the applicable conditions. A proposal that says only locked omits a material qualification.

This is an explanation of the named S3 behavior. Storage administration, permission decisions and validation belong to the appointed providers.

References: [A1]

Follow one invented object through three states

Consider one invented dataset object in a versioning-enabled S3 bucket with Object Lock enabled. V1 and V2 are symbolic labels, not actual S3 version IDs. V1 has an unexpired compliance retention period throughout the example. The later write and simple deletion are stipulated to be permitted by the applicable permissions and controls. No real data or successful recovery is represented.

Initially, V1 is current. The retained contents and the contents selected by an ordinary read of the key therefore refer to the same version.

A later permitted write under that same key creates V2 and makes it current. V1 remains stored as a noncurrent version; its protected contents have not been modified. An ordinary key-only consumer now selects V2. The example leaves V2's own retention settings unspecified; V1's protection does not establish them.

A subsequent permitted simple deletion, which specifies no version ID, makes a delete marker current. The marker has no dataset payload. V1 and V2 remain underlying versions, and V1 remains within its compliance retention period. An ordinary read without a version ID now returns 404 Not Found because the current version is the marker.

Across the three states, the protected historical version stays V1. The current selection changes from V1 to V2 to the delete marker. That changing relationship explains how the key can appear deleted while V1's retained bytes remain.

References: [A1][A2][A4]

Interpret not found without inventing an outcome

In this example, the not-found result describes the ordinary read's encounter with a current delete marker. AWS explicitly distinguishes that marker from deletion of the underlying object versions. The result therefore cannot establish erasure of V1.

The reverse inference is also too strong. Knowing that a protected version remains does not establish that a particular operator can retrieve it or that an application can use it. Nor does an unexplained 404 from some other environment identify a delete marker as its cause. The example's known state supplies the explanation; the status code alone does not.

When comparing proposals, replace the phrase the key is locked with a description of the protected version and the intended consumer behavior. Does the consumer follow the current key, or does the proposed return identify a historical version? Who establishes that the intended recipient can use that reference? These questions expose the missing relationship without asking the buyer to change storage settings.

References: [A2][A5]

Connect retained contents to the dataset return

Retention also leaves encryption and application state unresolved. AWS's Object Lock considerations explicitly say that Object Lock does not protect against losing encryption-key access or deletion of encryption keys. An encrypted version can remain retained while its contents become unreadable. The lock does not preserve a lost key.

A useful proposal therefore identifies the intended retained contents, the recipient's authorized access and the dependencies needed to interpret those contents. For a dataset made of several objects, the application team must explain which versions belong to the same intended release. A collection of individually retained versions does not, by itself, identify that coherent set.

The related Training Checkpoint or Storage Snapshot guide addresses the next recovery question: what application state was captured, and how does it relate to the copy's timing? This article adds the earlier identity question: which stored object version does the reference select? Copy time, version identity and captured application state describe different relationships.

Ask the appointed application and storage providers to connect those relationships to the claimed result and its supporting evidence. Object Lock alone does not establish a complete application backup.

References: [A3]

Scope physical storage connections separately

If the wider AI infrastructure project also needs storage-facing physical cabling or connection records, the related GPU Storage Network Handoff guide helps define that interface. Leviathan's Structured Cabling & Fiber Installation service and its project inquiry cover a relevant physical work discussion.

Share a nonconfidential equipment summary, approved endpoints, requested cabling work and required handoff records. That physical package concerns installed connections; object-version administration and recovery validation remain with the appointed storage and application providers.