An Avro Reader Default Is Not a Value the Source Measured

Keep the writer schema and the selected reader schema together

This explanation uses Apache Avro 1.12.0. The writer schema describes how the supplied data was written. The reader schema describes what the receiving application expects. Resolving their differences can preserve useful older data while returning additional fields or supported type promotions.

Binary record data does not contain its own field names or type descriptions. Its fields are encoded in the writer's declaration order. Supplying a differently ordered reader schema therefore requires resolution using the writer schema; treating the old payload as though it had been written in the new order would lose that interpretation.

Avro object container files carry the writer schema with their records. That does not make every isolated binary datum self-describing. The Java 1.12.0 guide's deserialization explanation expressly distinguishes the writer schema needed to interpret the written fields from the reader schema used for expected fields and added-field defaults.

References: [AV1][AV2]

AV-39: two versions of the same named record

W1 and R1 are labels for separate schema versions. Both have type record, name SensorReading and namespace example.av39, giving the same full record name, example.av39.SensorReading. They use no aliases or logical types.

Writer W1 declares exactly two fields, in this order: sensor of type string, then reading of type int. Neither field has a default. There is no quality field.

Reader R1 declares exactly three fields, in this order: quality of type string with the JSON string default "unknown"; reading of type long; then sensor of type string. The reading and sensor fields have no defaults. These declarations specify the conceptual schemas without prescribing a programming-language API.

Stipulate three valid W1 records. Each has sensor equal to "S7". Their reading values, in input order, are 24, 25 and 26. No physical unit or quality observation is stipulated. AV-39 is an original synthetic example, not a captured file or an executed software result.

References: [AV1][AV2]

Follow the three readings into the reader's fields

Resolution matches sensor to sensor and reading to reading by name. R1's first declared field, quality, does not take W1's first field value, "S7". A change in reader field order does not change the identities of the written fields.

For reading, writer int can promote to reader long. The values remain 24, 25 and 26. For quality, W1 supplies no corresponding field, so R1 supplies its default for each record.

The three logical returned records therefore have sensor "S7" throughout; reading values 24, 25 and 26 represented as long; and quality "unknown", "unknown" and "unknown". All three quality values come from R1. The result contains three readings and three default placeholders, adding zero observed quality measurements to the input.

These are named field results, not a promised low-level delivery sequence. Java 1.12.0's ResolvingDecoder documents writer-order delivery of the fields read from the writer and exposes field-order information to its caller. Reader field identity and decoder delivery order are separate concerns.

References: [AV1][AV3][AV4][AV5]

The same added field without a default changes the result

Define R2 as exactly R1 except that quality has no default. W1 still contains no quality field. There is now neither a written quality value nor a reader default to supply it, so resolution signals an error.

The successful reading promotion does not make the complete record resolvable. R2 cannot obtain three complete records under these rules by silently omitting quality, inventing an empty string or borrowing R1's default. The selected reader schema matters, including its actual default declarations.

The release-1.12.0 Java resolver shows the same distinction: its record-resolution logic obtains defaults for reader fields absent from the writer and produces a missing-required-field error when such a field lacks a default.

References: [AV1][AV5]

An encoded null follows its selected union branch

Now define writer W2 with the same record type, name and namespace. Its fields, in declaration order, are sensor:string, reading:int and quality with union type ["null", "string"]. None has a default. Stipulate a valid W2 datum whose quality value selects the null branch.

R1 still requests quality:string with default "unknown". W2 has a field named quality, so the absent-field rule does not apply. The writer union must resolve using the branch selected for this datum. Here that branch is null, which does not match R1's string type. Resolution signals an error; it does not return "unknown" for this quality value.

The presence of a string branch elsewhere in W2's union does not allow a reader to substitute that branch for the selected null. Conversely, a W2 datum selecting the string branch with value "approved" can resolve that field to R1's string as "approved". Its reader default is again unused. This branch-specific success does not establish that R1 can read every W2 datum.

A reader that accepts null can preserve an explicitly encoded null. Allowing null is different from allowing a field to disappear during encoding. Avro's record-field defaults do not authorize omitted writer fields or automatic replacement of malformed values.

References: [AV1][AV6]

A populated quality field can still contain no observed label

Suppose a consumer counts every populated quality field as a completed quality annotation. AV-39 would appear to contain three annotations, even though all three values were supplied by R1. The application can legitimately treat "unknown" as its chosen representation for unavailable quality, but it cannot count those placeholders as source observations.

The string alone may not reveal its origin. Stipulate a different W2 datum that explicitly writes the string "unknown" in its quality field. R1 returns the same visible string as it supplied for an older W1 record. In one case the string was encoded; in the other it came from reader resolution. Equal returned values do not establish equal provenance.

Where that distinction affects the consumer's decision, the supplied writer-schema identity and the selected reader schema need to remain traceable to the affected records. Ask the software provider how its returned representation preserves the distinction needed by the application. Do not assume every Avro reader attaches a provenance flag to each returned field.

Defaults can make evolution honest and useful: older records remain usable with an explicit unavailable state. Promoting 24 from int to long likewise changes its type without supplying a physical unit, checking a sensor or creating a trustworthy AI feature. Those meanings require their own source and application rules.

References: [AV1][AV3]

Compare the software offer at the consumer's actual result

An offer showing only three R1-shaped records establishes less than an offer that connects those records to W1 and explains how each field was obtained. For AV-39, the useful explanation identifies both complete schemas, the selected reader implementation and relevant version, and the consumer-facing result: three promoted readings and three quality defaults.

A comparison should also account for the changed cases. W1 with R2 lacks a required default and fails. W2 with selected quality null cannot resolve that field to R1's string. W2 with a selected quality string uses the written string. An offer that reports all of these as successful default substitution is describing different behavior from the specified resolution.

Direction matters too. If R1 were instead used as a writer schema, its reading field would be long. A W1 reader expects int, and long-to-int is not an allowed promotion even when a particular number happens to be 24. The W1-to-R1 result therefore establishes neither bidirectional compatibility nor compatibility across every historical schema. Registry policy and an operational migration plan are separate decisions.

If the consumer merely needs an explicit unavailable quality state for older records, the default can meet that requirement. If it needs observed quality labels, neither adding the field nor returning three populated strings supplies them. Have the appointed data-platform or application provider identify the additional source evidence the requirement needs.

References: [AV1][AV3]

Scope a separate equipment connection requirement

If the wider platform project also needs physical connections between installed compute and storage equipment, use Leviathan's Structured Cabling & Fiber Installation service and the related project inquiry to discuss that work. Share a nonconfidential equipment and site summary, approved endpoints and routes, material-supply responsibilities and required connection records.

That service concerns the agreed physical cabling and handoff. Avro integration, schema management and application value handling remain with the appointed software providers.

References: [L1]