---
title: "Equal Dataset Bytes, Different S3 Request Costs"
description: "Leviathan Systems — Technical content Published 2026-10-20; updated 2026-10-20."
canonical: "https://www.leviathansystems.co/articles/ai-object-storage-request-count-dataset-size"
last-updated: "2026-10-20"
---

# Equal Dataset Bytes, Different S3 Request Costs

Leviathan Systems — Technical content
Published 2026-10-20; updated 2026-10-20.

Equal retained dataset bytes do not establish equal S3 request costs. The request population depends on how many objects the consumer accesses, which operations it makes and how often it repeats discovery or retrieval. In a stipulated full-object pass through the same 100 GiB payload, 100 objects require 100 GETs while 102,400 smaller objects require 102,400 GETs. That 1,024-fold operation-count difference describes neither the total bill nor performance. Compare named operations and their applicable price categories over the required access interval.

## Hold the payload constant and identify the objects

Consider two fictional access proposals for the same AI dataset. Layout A contains 100 objects, each with 1 GiB of payload. Layout B contains 102,400 objects, each with 1 MiB. Since 1 GiB equals 1,024 MiB, both contain exactly 100 GiB of the same stipulated payload. Only the object boundaries differ.

GiB and MiB are binary payload units here. This equality is not a conversion into a provider's billing GB, a claim about physical storage consumption or a complete storage-price comparison.

The named provider scope is ordinary objects in Amazon S3 general purpose buckets using S3 Standard, under AWS documentation checked October 20, 2026. The objects are already complete, the population stays unchanged during the comparison, and the consumer's required read and list permissions are stipulated. The example grants no access.

Start with those populations rather than a catalogue of charges. A retained-byte total identifies how much payload the proposal holds. It leaves open how many separately addressed objects the consumer will retrieve.

References: [OR2](https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectsV2.html), [OR3](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html)

## Count one complete pass as a named access behavior

For the first comparison, stipulate one successful full-object GetObject request for every object, exactly once. There are no range requests, retries, HEAD requests, cache reuse, extra versions, writes or other retrieval operations. Discovery is counted separately in the next section.

Layout A therefore requires 100 GETs. Layout B requires 102,400 GETs. Dividing 102,400 by 100 gives 1,024: the smaller-object layout produces 1,024 times as many GET operations for this specified pass, despite the equal payload.

AWS defines GetObject as retrieving an object. The one-call-per-object rule is this example's workload assumption; it is not a universal description of an S3 client. The API also supports retrieving a byte range, so a real consumer's object count alone cannot establish its GET count.

The stipulated pass returns the complete payload in either layout. Nothing here specifies elapsed time, concurrency, network overhead, backing-device I/O or throughput. The operation ratio cannot be reused as a performance ratio or a ratio of total charges.

References: [OR3](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html)

## Discovery adds LIST operations without reading every payload

Now add a separate discovery task: obtain a complete flat listing of exactly the selected dataset's keys. No delimiter or CommonPrefixes grouping is used. For this fictional comparison only, stipulate pages containing 1,000 keys except for the final page, with no repeated or additional listing calls.

Layout A fits its 100 keys in one response, so discovery uses 1 LIST request. Layout B uses 102 full pages containing 102,000 keys and a final page containing 400 keys. Its discovery count is therefore 103 LIST requests.

ListObjectsV2 permits at most 1,000 keys in a response and can return fewer. Its continuation mechanism supports further pages. The 103-call result follows from the additional full-page premise; the documented ceiling alone does not guarantee that exact count for a real listing.

A listing supplies keys and associated information within its response scope. It does not retrieve every listed object's payload. Listing and full-object retrieval consequently remain separate populations even when both are described informally as dataset access. ListObjectsV2 uses an HTTP GET method, but that does not make its LIST operation a GetObject payload read.

References: [OR2](https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectsV2.html)

## Keep each request type beside its count and price basis

AWS's pricing page bases request charges on operation type and quantity. It states that LIST uses the S3 Standard PUT, COPY and POST rate. Preserve GET and LIST separately when reading a quote; an undifferentiated total called read requests can hide the applicable category.

Use a deliberately linear toy model to translate the example's counts. Let G mean accounting units per 1,000 GET requests, and L mean accounting units per 1,000 LIST requests. Both proposals use the same G and L. Assume no free allowances, minimums, rounding or credits. These are unevaluated symbols, not current AWS prices.

Layout A's one-pass request component is (100 / 1,000)G + (1 / 1,000)L = 0.1G + 0.001L. Layout B's component is (102,400 / 1,000)G + (103 / 1,000)L = 102.4G + 0.103L.

The GET coefficient differs by a factor of 1,024; the LIST coefficient differs by a factor of 103. Combining the two does not justify applying either factor to the combined component. Their respective accounting values still matter.

Storage, data retrieval, data transfer and every other charge are outside this request component. The expressions are not AWS invoice forecasts. A quote for another class, an included request allowance, failed calls, special endpoints or a bundled service needs its own applicable terms before any monetary comparison.

References: [OR1](https://aws.amazon.com/s3/pricing/)

## Specify when discovery and retrieval repeat

The object layout is only one input. A consumer that already has an accurate, usable inventory of the required keys can address them without a fresh listing. That changes discovery's premise; it does not remove the GETs when the consumer still retrieves every object.

Extend the fictional case to four complete passes through the unchanged dataset. Stipulate that the first listing's keys remain available to the consumer and are reused without additional remote calls. Keep the one-successful-full-object-GET rule for every pass, with no payload cache reuse.

Across those four passes, layout A now contributes 400 GETs and 1 LIST; layout B contributes 409,600 GETs and 103 LISTs. Their toy components become 0.4G + 0.001L and 409.6G + 0.103L. Discovery occurred once, while retrieval occurred four times.

If discovery were instead repeated before every pass under the same full-page premise, the LIST totals would be 4 and 412. Multiplying the entire first-pass expression by four would describe that different behavior, rather than the retained-inventory case.

Other consumers can make repeated or range GETs, perform metadata lookups, or reuse cached payload on later passes. Each can change which calls reach S3. These are possible differences to establish in the actual access plan, not measured optimizations or a recommendation to rewrite the dataset. The number of training epochs or model requests alone does not establish that call population.

References: [OR3](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html)

## Use the count to explain the offer, then assess workload fit

The worked comparison supports a narrow, useful conclusion: equal retained payload can require very different numbers of named storage operations. Once the operation population and applicable terms are known, the request component becomes explainable. A single GB-month line cannot provide that explanation.

A lower request count still leaves the layout's latency, parallel access, update behavior and usefulness for the intended training workload unresolved. Those properties require evidence for the actual consumer and dataset. Combining records into fewer objects could also change what must be fetched for a selected subset; the full-pass premise supplies no answer for that different workload.

Keep the required contents fixed when comparing access offers. A key inventory does not by itself establish that its objects form the intended dataset generation. Likewise, a small Parquet result does not identify storage calls, and bytes returned to an application do not identify their physical source. Release coherence, scan elimination and cache measurement remain their own evidence questions.

Object completion and minimum storage residence concern different decisions from this access count. An existing upload-part count cannot replace the completed-object population, and a residence charge cannot reveal how often objects were read. No event-notification count or fee is inferred from the GET population.

## Ask for an operation population with the storage proposal

Take the immediate question to the customer's appointed storage and account teams. Ask them to identify the exact dataset release and access scope, the completed-object population and sizes, and the consumer behavior assumed for the required interval. The answer should distinguish a complete pass from a selected subset and initial access from later reuse.

Request separate quantities for the named retrieval, discovery and any additional operations. Establish whether key discovery repeats, whether payload is reused, and which ranges, retries or metadata calls the estimate includes. Keep unknown behavior visible rather than substituting the retained-byte total for it. These are questions about a proposed service, not instructions to run calls or change a bucket.

Then match each population to the quoted product, storage class, location, request category, included allowances and charging terms. Request a clear separation between the request component and the rest of the offer. A supplier can explain the arithmetic without claiming a completed workload test or guaranteeing the final bill.

The useful returned answer can reproduce why the base example has 100 GETs plus 1 LIST, or 102,400 GETs plus 103 LISTs, and explain every departure from those premises. Evaluate the delivered access behavior and separately priced components together before choosing an offer. The related cloud-versus-colocation ownership guide helps assign the broader operating responsibilities.

References: [OR1](https://aws.amazon.com/s3/pricing/)

## Scope any selected equipment's physical connections separately

If the wider program also includes a selected equipment installation that needs physical compute-to-storage connections, Leviathan's Structured Cabling & Fiber Installation service and the related project inquiry provide the appropriate route to discuss that work.

Share a nonconfidential equipment and site summary, approved endpoints and routes, appointed material-supply responsibilities and required connection records to discuss physical scope and availability. This conditional inquiry concerns agreed physical cabling and handoff. Cloud request accounting, object-store administration and workload evaluation remain with the appointed storage, account and application teams.

References: [L1](https://www.leviathansystems.co/services/structured-cabling)

## Primary sources

- [OR1 — AWS: Amazon S3 pricing - requests and data retrievals](https://aws.amazon.com/s3/pricing/). Checked 2026-10-20. Request type and quantity determine the applicable request component; LIST is priced with S3 Standard PUT, COPY and POST. Supports category separation, without adopting current monetary rates or an account-specific invoice.
- [OR2 — AWS S3 API: ListObjectsV2](https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectsV2.html). Checked 2026-10-20. Defines the up-to-1,000-key response, possible shorter pages, continuation and key/metadata listing. Exact page counts in the example require its stated flat-listing and full-page assumptions.
- [OR3 — AWS S3 API: GetObject](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html). Checked 2026-10-20. Defines named-object retrieval and byte-range access. One successful full-object call per object is an explicit illustrative premise, rather than a universal client behavior.
- [L1 — Leviathan Systems: Structured Cabling & Fiber Installation](https://www.leviathansystems.co/services/structured-cabling). Checked 2026-10-20. Agreed physical routes, endpoints, installation, labels and connection records support a separate equipment-cabling inquiry. The service scope does not establish cloud billing or object-store administration.

## Related Leviathan resources

- [GPU cloud or colocation: assign the operating responsibilities](/articles/gpu-cloud-vs-colocation-ownership)
- [Structured Cabling & Fiber Installation](/services/structured-cabling)
- [Project inquiry for a separate physical storage-network installation](/contact?service=structured-cabling&from=%2Farticles%2Fai-object-storage-request-count-dataset-size)
- [Equal Image Arrays Can Occupy Different DICOM Patient Coordinates](/articles/healthcare-ai-dicom-pixel-index-patient-geometry)
- [RDMA Message Reception Needs More Than Spare Buffer Bytes](/articles/rdma-send-posted-receive-resource-availability)

## Does selected equipment need physical storage-network connections?

Share the equipment, site, approved endpoints and routes, material-supply responsibilities and required records to discuss physical cabling scope and availability.

[Discuss the scope](/contact?service=structured-cabling&from=/articles/ai-object-storage-request-count-dataset-size)