Provisioned Inference or Token Billing: Separate Rate Capacity from Consumed Work

Identify the model-service unit

Read the noun after the quantity. AWS's Bedrock documentation associates a Model Unit, or MU, with a specified model and throughput level. Its API reference separately identifies the model and the number of units allocated. A proposal for two MUs therefore needs the model identity and the applicable unit definition before the quantity becomes useful.

That identity does not tell you how many GPUs or servers the provider uses. Nor does one MU for one model establish equivalent application capacity for another. Record the exact model/version, inference route and Region alongside the offered quantity. Treat a changed model as a reason to recheck the capacity and metering basis.

This guide uses AWS primary documentation checked October 20, 2026. It addresses the model-service unit that a buyer must interpret after distinguishing cloud availability from a pricing commitment. The existing GPU Capacity Reservation or Savings Commitment guide covers the separate EC2 decision.

References: [S2]

Check input and output separately

Bedrock's Provisioned Throughput guide defines an MU using the input tokens it can process across requests in one minute and the output tokens it can generate across requests in one minute. Preserve those two dimensions. Adding them into a single allowance can hide the direction that limits an offer.

Consider an unnamed fictional offer with stipulated limits of 6,000 input tokens/minute and 2,000 output tokens/minute. The requirement in the relevant minute is 7,000 input tokens plus 500 output tokens. Its combined count is 7,500, below the arithmetic sum of the limits, 8,000. Yet it exceeds the input limit by 1,000 tokens. The unused output allowance supplies no evidence that the input requirement fits.

These invented counts are not a real Bedrock MU definition. Input and output dimensions cannot be exchanged, or unused allowances banked for another interval, unless the applicable offer explicitly establishes that behavior. The example compares stated limits only; it supplies no arrival-pattern test or latency guarantee.

References: [S1]

Keep traffic shape beside the rate

An hourly token total loses the distribution within that hour. The same total can describe evenly spaced work or a concentrated burst. Even a minute-level allowance leaves a response question open: how do simultaneous requests behave, and which responses finish within the application's deadline?

Give the capacity reviewer the distribution of prompt lengths and requested response lengths, the expected simultaneous demand and the periods when demand concentrates. Preserve which observations are measured and which are forecasts. An average input/output ratio can conceal the busiest input-heavy period or the period dominated by long responses.

State the response requirement in application terms: acceptable initial wait, completion deadline and the proportion of requests that must meet them. An interactive assistant and a queue with a later batch deadline can have different acceptable outcomes at the same token volume. Comparing counts with rate limits is an initial fit check; evidence for the actual traffic and response requirement must resolve service performance.

Separate purchased hours from consumed tokens

AWS documents hourly billing for purchased Provisioned Throughput. Its pricing page also shows token-priced offerings with distinct input and output meters, alongside hourly Model Unit pricing headings. Those are different billing bases, and neither is a complete workload comparison by itself.

For a provisioned offer, identify the purchased unit quantity, hourly rate and entire billable interval. Distinguish the application's intended running hours from the hours the offer actually charges. Idle time inside a paid allocation still matters to the comparison. Retain the applicable commitment and billing conditions instead of assuming that charges stop when the application becomes quiet.

For a token-priced offer, calculate input quantity times its input rate, plus output quantity times its output rate, using consistent units. Record any other applicable meters and charges separately. A single blended token price is usable only for the mix from which it was derived; changing the output share can change that average even when the quoted rates remain unchanged.

References: [S2][S3]

Work two fictional bills without assuming capacity fit

The following offers use invented cost units, not currency or current provider prices. They are separate from the earlier rate example: neither offer has a capacity definition here. Offer P costs 10 units/hour for an eight-hour purchased window, giving 10 x 8 = 80 units. That window is a fictional term, not an AWS commitment option.

Offer U costs 20 units per million input tokens and 80 units per million output tokens. For a stipulated quiet workload during the comparison window, 1 million input tokens cost 20 units and 0.25 million output tokens cost 20 units. U's charge is 40 units; P remains 80 under its stipulated terms.

For a second workload of 2 million input tokens and 1 million output tokens, U costs (2 x 20) + (1 x 80) = 120 units. P remains 80. The first workload's output charge equals its input charge despite the smaller output count; in the second, output accounts for 80 of U's 120 units.

The arithmetic changes which listed charge is lower. It does not establish that P or U can serve either traffic profile. A different output mix changes U's charge, while paid idle hours affect the comparison with P. Resolve capacity and response fit separately before treating either calculation as a reason to buy.

Confirm the eligible inference route

Check whether the selected model and configuration support the routes being compared. A billing comparison between two labels is unusable if one route is unavailable for the required model or Region.

In particular, do not apply a blanket rule that every customized Bedrock model requires Provisioned Throughput. The current custom-model inference overview describes both Provisioned Throughput and custom-model on-demand deployment. The dedicated on-demand page specifies supported base models, Regions and prerequisites. That establishes an on-demand route for eligible cases, rather than universal eligibility for all custom models.

Ask for the actual model/version and the applicable dated eligibility reference. Keep any proposed substitution visible to the workload owner: changing the model to obtain another billing route also reopens the quality and performance comparison.

References: [S4][S5]

Distinguish forecast usage from useful output

A token forecast estimates metered work. It does not show how many application tasks finish correctly and on time. Keep the billing record and the accepted-task record separate, then relate them over the same observation period.

Define useful output before comparing evidence. For a document task, that might be a completed response meeting an agreed quality check within its deadline. Retain input/output counts, the traffic pattern, model settings and the acceptance result together. Identify retries, failed requests and responses rejected for quality or lateness, then reconcile their treatment with the actual charging rules. Do not assume that every generated token contributes to an accepted task.

Where comparable measurements exist, relate the applicable charge to the same accepted outcome and response requirement. Where only forecast volumes exist, label the result a billing scenario. The existing Which GPU Is Fastest? guide explains the broader billed-cost-versus-completed-work comparison; its benchmark evidence does not become a measurement of the proposed model service.

Request a comparison that answers both questions

Ask the provider for the model-specific capacity definition and billing basis against one shared workload description. The capacity answer should address input and output limits, their counting interval, concurrent demand and the evidence relevant to the required response times. The billing answer should identify units, rates, paid duration, commitment exposure and any separately applicable charges. Leave an unanswered field open.

This produces two decisions: whether the service can satisfy the stated work, and how its charge changes across credible usage scenarios. A lower forecast bill cannot close a missing performance answer. Equally, a service that meets the response requirement still needs a comparison using the full paid interval and relevant output mix.

If the separate infrastructure decision involves installing physical GPU equipment, Leviathan's Complete Physical Deployment service covers an agreed equipment receiving, rack integration, onsite installation and cabling package. Use its project inquiry with the equipment, site and intended physical handoff. Model-service purchasing and application acceptance remain with the selected provider and the customer's responsible teams.

References: [L1]