An INT8 Matrix Operation Still Needs Its Overflow Contract

Identify the operation behind the INT8 claim

NVIDIA's archived CUDA 13.0.0 manual, PTX ISA 9.0, documents integer mma as D = A*B+C. Its m8n8k16 form permits signed eight-bit A and B, with signed 32-bit C and D, and requires sm_75 or higher. A is 8 by 16, B is 16 by 8, and C and D are 8 by 8. With the .satfinite qualifier, an overflowing accumulated value is limited to the signed 32-bit endpoints; without it, the value wraps. Assume valid matrix layout and the required warp participation throughout this explanation. This named instruction contract does not identify the kernel selected by cuBLAS or a framework.

References: [P1]

Count the products and the value already being carried

Each output entry combines one row of A with one column of B, then includes its corresponding C entry. There are 16 products per output in this shape, not 16 products shared among all 64 output entries. Even very small operands can therefore reach a range limit when C is already near an endpoint. The relevant question is how much room remains in that entry of C before the next contribution arrives. Reducing the stored operand width does not reduce a previously carried value. The related ONNX quantization article explains the separate mapping from input values to integer codes; the calculation here starts with already specified integer operands.

References: [P1]

I32-16: sixteen small products cross the upper endpoint

Fictional example I32-16 starts with every entry of C equal to 2,147,483,632, a valid signed 32-bit value. Every entry of A and B is 1, a valid signed eight-bit value. Each output therefore receives 16 products of 1 x 1, adding 16. The unrestricted mathematical result is 2,147,483,632 + 16 = 2,147,483,648. That is one above the signed 32-bit maximum, so it cannot be represented in D as that exact integer. Saturation returns 2,147,483,647; wrapping returns -2,147,483,648. For the latter calculation, subtracting 2^32, or 4,294,967,296, gives the signed wrapped value. All 64 entries behave identically in this constructed case. These are deductions from fictional matrices and the documented rule, not measured GPU or model results.

References: [P1]

Carry each actual output into the next matrix step

Now stipulate a second operation of the same shape, using its preceding actual D as its new C. Set every entry of the second A to -1 and every entry of B to 1, contributing -16 per output. The unrestricted two-step reference is 2,147,483,632 + 16 - 16 = 2,147,483,632; its first intermediate was never a representable signed 32-bit C. The saturated path instead carries 2,147,483,647, then subtracts 16 to finish at 2,147,483,631. Saturation discarded one at the first step, and the later subtraction does not recover it. The wrapping path carries -2,147,483,648. Subtracting 16 crosses the lower endpoint; adding 4,294,967,296 gives 2,147,483,632. It matches this final reference despite disagreeing with the first exact intermediate. The final outputs are within range in both paths, yet they differ by one.

References: [P1]

Establish the range before relying on cancellation

For the first step's positive contribution of 16, the largest starting C that avoids upper overflow is 2,147,483,647 - 16 = 2,147,483,631. I32-16 starts one above that threshold. With C instead zero, the same all-one matrices produce 16 per output, with no overflow under either rule. Thus the example establishes a missing condition, not an inherent failure of INT8. For a longer carried computation, establish the bounds of every relevant intermediate, including negative excursions, rather than checking only the final sum. Here all products within each step have the same sign, keeping the example independent of cancellation inside one matrix operation. Starting values, contribution ranges and carry boundaries belong in the numerical comparison.

Compare the requested result at its actual observation points

Consider two fictional offers for this exact two-step calculation. Offer A identifies INT8 inputs, signed 32-bit accumulation and saturation, and calls that sufficient for exactness. Offer B specifies the archived operation, applicable target, both matrices, starting C, carry between steps, wrapping rule and final-only observation requirement. If the requested final integer is 2,147,483,632, B's stated arithmetic matches it; A's returns 2,147,483,631. This is a mathematical comparison of the descriptions, and a real offer still needs evidence that the delivered computation follows its description. If the requirement includes the exact first intermediate, neither described 32-bit path satisfies it; preserving that integer requires a representation capable of holding it. Wrapping is therefore not a general preference: the required operation may instead be a saturating sequence or arithmetic modulo 2^32. Agree which contract is intended. Model task quality remains a separate evaluation with the appointed software and model owners. The related FP8 guide helps distinguish operand, accumulator and retained-output roles.

Connect a separate physical deployment requirement

When the selected equipment also needs receiving, rack integration, onsite installation or IT cabling, use Leviathan's Complete Physical Infrastructure Deployment service and the related project inquiry to discuss those physical stages. Share a nonconfidential equipment list, quantities, destination, rack layout, included cable endpoints and target window so scope, requirements and availability can be confirmed. Numerical implementation and model acceptance remain with their appointed owners; the physical deployment inquiry does not offer kernel development, tuning or software validation.

References: [L1]