Preserve Decimal Amounts When a Parquet Schema Changes

Read precision and scale before comparing storage width

For an enterprise AI buyer receiving an amount column, a revised schema needs to preserve the agreed values at the intended consumer. A statement that every integer was retained leaves their interpretation unresolved.

The DECIMAL definition in Apache Parquet format 2.10.0 gives value = unscaled integer × 10^(-scale). Here, DECIMAL(p,s) denotes precision p and scale s. Precision caps the decimal digits in that integer; scale locates the decimal point. Precision must be positive, and scale must be nonnegative and no greater than precision. INT32 permits declared precision from 1 through 9.

Physical width, declared capacity and numerical meaning therefore need separate comparisons. A larger physical container alone changes neither the multiplier nor a precision limit left unchanged. The following case isolates those differences using the named historical format definition; it predicts no automatic conversion by an unspecified exporter or reader.

References: [PQD]

Offer A: the original three amounts total 20.15

DEC-P is an original fictional case with three identified rows, R1, R2 and R3. All use USD and one stipulated reporting scope; none is missing. The buyer requires each amount to survive exactly. No real dataset, export or reader execution is represented.

Offer A supplies INT32 storage annotated DECIMAL(4,2), with unscaled integers 1250, 990 and -225 in row order. Dividing each by 100 gives USD 12.50, USD 9.90 and USD -2.25. All three integers fit the four-digit declaration.

Their unscaled sum is 1250 + 990 - 225 = 2015. At scale 2, that represents 20.15, agreeing with 12.50 + 9.90 - 2.25. This is exact explanatory arithmetic, without assigning a result type to a software aggregation.

Offer A provides a consistent reference for the requested amounts. Any proposed change must be compared with these row identities and values, rather than with the appearance of a revised type name.

References: [PQD]

Offer B: unchanged integers at scale 3 change the amounts

Offer B retains INT32 and the integers 1250, 990 and -225, but declares DECIMAL(4,3). Under its stated scale, the amounts are now 1.250, 0.990 and -0.225. Their total is 2.015.

The stored integers survived; each interpreted amount became one tenth of the requested amount. Adding a third fractional place to the declaration did not append a harmless display zero. It changed the divisor from 100 to 1000.

Each of B's three integers still fits precision 4. Its problem in this case is a changed meaning, even though the supplied values fit that declaration. Merely widening its physical container would leave 1.250, 0.990 and -0.225 unchanged under the same scale.

B could describe a different set of intended amounts. It cannot substantiate DEC-P's requirement to preserve A's amounts simply by demonstrating identical integer payloads.

References: [PQD]

Offer C: rescale the integers and supply five-digit capacity

To preserve DEC-P's amounts at scale 3, Offer C supplies 12500, 9900 and -2250. Dividing by 1000 gives 12.500, 9.900 and -2.250. These are numerically the original 12.50, 9.90 and -2.25.

The larger unscaled integers require another capacity check. R1's 12500 has five decimal digits, so DECIMAL(4,3) is insufficient. INT32 can physically hold 12500; that fact does not make it fit a four-digit declaration. DECIMAL(5,3) supplies enough precision for all three stated rows while remaining within the named format's INT32 precision allowance.

Their rescaled sum is 12500 + 9900 - 2250 = 20150. At scale 3 it represents 20.150, numerically equal to 20.15. The extra trailing zero records the chosen scale; it supplies no new evidence of measurement accuracy.

Compare two distinct failures before choosing C. B retained the integers but changed the amounts. Rescaling those integers correctly while retaining precision 4 would preserve the intended numerical interpretation but exceed the declaration for R1. C resolves both issues for this case.

References: [PQD]

Check the required value range as well as the examples

At precision 4 and scale 2, the largest positive amount is 99.99. At precision 4 and scale 3 it is 9.999. Raising the scale without raising precision has reduced the available whole-number capacity. Precision 5 and scale 3 allows 99.999, which contains the original positive range as well as DEC-P's rows. The corresponding negative endpoints have the same magnitudes.

This explains why the buyer should request both the required scale and the required amount range. Three fitting samples establish those samples, not a complete future input population. A target declaration must accommodate the agreed range, including negative values where required.

A precision-only change answers a different question. Moving A from DECIMAL(4,2) to DECIMAL(5,2), while retaining its integers, leaves all three amounts unchanged and extends the declaration's capacity. It does not introduce a third fractional place.

For a fixed amount, raising scale by one requires multiplying its unscaled integer by ten. Reducing scale requires an integer result after the corresponding division if the amount is to remain exact. In DEC-P, -2.25 cannot survive exactly at scale 1: its required unscaled value would be -22.5. If a proposed value does not fit the selected scale or precision, the actual exporter must identify and substantiate its rounding or overflow policy. No default rejection, truncation, rounding or saturation behavior is assumed here.

References: [PQD]

Treat Arrow decimal128 as its own declared type

Apache Arrow 20.0.0's decimal128 reference describes fixed-point values encoded as scaled integers in 128 bits, with precision from 1 through 38. Its example decimal128(7,3) can represent 1234.567 but not 12345.67 or 123.4567. The common width has not removed the type's declared magnitude and fractional-place constraints.

An offer of a 128-bit decimal is consequently incomplete without its precision and scale. It still needs to represent the agreed values at the actual delivery boundary.

Arrow also permits negative scales. The Parquet 2.10.0 DECIMAL definition used here does not. An Arrow type's validity alone therefore cannot establish that an unchanged declaration is valid in this Parquet format. A proposed export needs its actual Parquet schema and the behavior of the named exporter and consumer; this API reference establishes no conversion outcome.

References: [AR20][PQD]

Keep the amount's business meaning and project scope explicit

DEC-P supplies USD and its reporting scope as separate premises. A DECIMAL annotation does not identify a currency, measured concept or reporting period, and preserving its represented amount does not establish accounting correctness. The related XBRL guide explains how concept, entity, period and unit affect a financial comparison. XBRL 2.1's decimals attribute expresses reported accuracy in decimal places; it does not provide this Parquet scaling rule.

For the separate question of how much data a query scans, use the related Parquet projection and filtering guide. Correct decimal interpretation supplies no byte count or elapsed-time result.

If the wider project independently requires installation of customer-selected enterprise or GPU equipment, Leviathan's Rack-and-Stack Installation service and the linked project inquiry provide a route to discuss equipment, approved rack positions, mounting instructions, site conditions and the requested physical handoff. Share a nonconfidential outline to discuss scope and availability. Decimal conversion, data engineering and financial interpretation remain with the appointed data and software providers.

References: [XB21][L1]

Choose from the delivered schema and the amounts received

Bring the comparison back to the same requested rows. A declares DECIMAL(4,2), supplies 1250, 990 and -225, and represents 12.50, 9.90 and -2.25. B declares DECIMAL(4,3) with those same integers and instead represents 1.250, 0.990 and -0.225. C declares DECIMAL(5,3), supplies 12500, 9900 and -2250, and represents 12.500, 9.900 and -2.250.

A and C satisfy the numerical representation requirement for this fictional case. If the buyer additionally requires scale 3, C supplies that declaration and enough precision; A remains the scale-2 reference. B fails the amount-preservation requirement. A claim of correctly rescaled integers under DECIMAL(4,3) also leaves R1 outside the declared capacity.

For a real offer, request the before-and-after schema, the physical representation, corresponding identified values and their exact intended amounts. Tie that comparison to the actual delivered file or column and the named exporter and consuming component, including versions and any intervening conversion. The provider still needs evidence that the values reaching that consumer follow the proposed interpretation. A schema proposal or a display of 20.15 alone does not establish that path.

Compare individual rows before relying on a total: two row errors can offset each other. Keep the amount range and any non-exact conversion policy alongside the returned comparison. The useful result is an explained match between requested amounts, delivered declaration and consumer-received values, with any unresolved difference visible.

References: [PQD]