An 8-Bit Tensor Still Needs Its Scale, Zero Point and Clipping Rules

Start with the code and the value it represents

A uint8 tensor stores unsigned integer codes from 0 to 255. Reading a code as though it were already the intended model value can change the meaning of the supplied tensor. For example, code 128 can represent zero under the explicit scale and zero point used below. The code is 128; the reconstructed value is 0. Both statements can be correct.

An affine quantization maps numerical inputs onto a grid of representable values. The scale determines the grid spacing, and the zero point locates zero within the code range. In the named ONNX operators, these parameters are inputs to the operation. They are not inferred from an eight-bit filename or a general statement that a model is quantized.

This explanation uses the historical main-domain QuantizeLinear and DequantizeLinear definitions with operator version 21. It isolates per-tensor uint8 quantization with scalar parameters. That scope keeps the example precise while leaving other integer and floating-point formats to their own contracts.

References: [Q1][Q2]

Follow quantization and reconstruction in their stated order

For this uint8 case, QuantizeLinear divides the input by its scale, rounds that quotient to the nearest integer with ties to even, adds the zero point, and limits the result to the range 0–255. Saturation means keeping a value at the nearest endpoint when the result would otherwise exceed that range.

DequantizeLinear takes a code, subtracts its zero point and multiplies by its scale. It produces a full-precision output tensor with the same shape. That is reconstruction under the supplied mapping, rather than a record of the original value before quantization.

The scalar parameters make this per-tensor quantization: the same mapping applies to each element. ONNX also defines other granularities, so an actual offer must identify the parameters applying to the element being interpreted. The scalar example cannot stand in for an unspecified per-axis or blocked arrangement.

References: [Q1][Q2]

QA-75: six inputs through one mapping

Fictional example QA-75 stipulates a positive scale of 0.25 and a uint8 zero point of 128. Its six input values, in order, are −40, −1, 0, 0.3, 0.375 and 40. The calculations illustrate the mathematical mapping; they are not a backend execution, a measured model or a recommended calibration.

For −1, division by 0.25 gives −4. This is already an integer; adding 128 gives code 124. Reconstruction subtracts 128 and multiplies by 0.25, returning −1. Zero follows the same path through code 128 and reconstructs exactly as zero.

For 0.3, the scaled input is 1.2. Rounding gives 1, so the code is 129 and reconstruction gives 0.25. The input 0.375 exposes the tie rule: its scaled value is 1.5, exactly halfway between 1 and 2. The nearest-even result is 2, giving code 130 and reconstructed value 0.5. Rounding is therefore not simply dropping the fractional part.

The extremes reach saturation. For −40, the scaled input is −160; after adding 128 the intermediate integer is −32, which is limited to code 0. For 40, the corresponding integer is 288, which is limited to 255. Reconstruction then gives −32 and 31.75 respectively.

References: [Q1][Q2]

Identify what the wider output cannot recover

Under QA-75, the 256 available reconstructed levels run from (0 − 128) × 0.25 = −32 to (255 − 128) × 0.25 = 31.75. Neighboring codes are separated by 0.25. This describes the reconstruction grid for the stipulated parameters.

Now consider a seventh possible input, 32, without adding it to the original six-element tensor. Dividing 32 by 0.25 and adding 128 produces 256, which saturates to 255. Thus both 32 and 40 produce the same endpoint code and reconstruct as 31.75. A recipient given only code 255 and these parameters cannot determine which of those inputs was originally supplied.

The same loss of distinction appears inside the range through rounding. QA-75 reconstructs 0.3 as 0.25, an error of −0.05, and 0.375 as 0.5, an error of +0.125. The clipped 40 reconstructs as 31.75, an error of −8.25. The half-step rounding distance does not bound that clipped error.

A wider output type can hold the reconstructed 31.75 with greater representational capacity. It has received no extra evidence distinguishing 32 from 40. Increasing output width therefore cannot invert this many-to-one mapping. That conclusion follows from the supplied values; it does not predict whether either error matters to a particular model.

References: [Q1][Q2]

Keep identical bytes paired with their actual parameters

Suppose a second fictional offer supplies the same code 128 and the same scale 0.25, but uses zero point 0. Its reconstructed value is (128 − 0) × 0.25 = 32. QA-75 instead gives (128 − 128) × 0.25 = 0. The code width, code value and scale match, yet the numerical meaning differs.

This does not by itself establish that either offer is defective. It establishes that they are different representations. To compare them against one requested tensor, the buyer needs the intended values and the actual parameters used by the consuming graph. A label attached to a source artifact is insufficient if a different mapping reaches that consumer.

The difference also explains why parameter granularity matters. A parameter belonging to another axis, block or tensor is not interchangeable merely because its numeric type matches. Establish the relevant mapping for the supplied object before attributing a result difference to the hardware or model. No parameter change or software configuration is prescribed here.

References: [Q1][Q2]

Compare the representation, then the outcome being bought

Imagine two offers for the same intended inference input. Offer A states only that its tensor is eight-bit. Offer B identifies the ONNX operator version, uint8 code type, scalar scale and zero point, and shows how the requested values map and reconstruct. B makes the representation assessable. Its explanation still has to match the actual delivered graph and input.

If the requested comparison requires the original −40 and 40 to survive exactly, QA-75 does not satisfy that requirement: its endpoint reconstructions differ. If the requirement instead permits approximation, the appointed model team must establish whether the resulting output meets the task criteria. The arithmetic alone supplies no such quality result. A correct interpretation is necessary for that evaluation, but it is not its outcome.

Keep resource claims with their own evidence too. The related weight-memory guide accounts for model representation and runtime populations; the mixed-precision guide separates operand, accumulator and retained-array roles. Neither a short file nor a wider accumulator changes what these particular codes represent. The ONNX and TensorRT guide addresses the separate question of which model or compiled artifact a runtime can use.

Connect a separate equipment installation requirement

Where the wider inference project also requires physical equipment installation, use Leviathan’s Rack-and-Stack Installation service and the linked inquiry to discuss the selected enterprise or GPU equipment, approved rack positions, mounting instructions, site conditions and required physical handoff. Share a nonconfidential outline so the scope, requirements and availability can be discussed.

Representation selection and model-quality evaluation remain with the appointed software and model owners. The equipment inquiry concerns the separate physical installation; it does not supply quantization, runtime configuration or model-validation services.

References: [L1]