---
title: "An Arrow List Can Retain Child Values Behind a Null Parent"
description: "Leviathan Systems — Technical content Published 2026-10-21; updated 2026-10-21."
canonical: "https://www.leviathansystems.co/articles/arrow-list-validity-offsets-visible-child-values"
last-updated: "2026-10-21"
---

# An Arrow List Can Retain Child Values Behind a Null Parent

Leviathan Systems — Technical content
Published 2026-10-21; updated 2026-10-21.

An Arrow List parent can be null even when its offsets enclose stored child values. Those values do not become elements of a logical list at that parent. Reconstruct each parent from its validity and offset range, then interpret the child slots within valid lists. In the example below, five stored integers represent [[10,20], null, []]: three parent records, two valid lists and two logical child elements.

## Parent validity decides whether the segment supplies a list

Apache Arrow's Columnar Format 1.5, documented with Arrow 21.0.0, represents a variable-size List using a parent validity bitmap, offsets and a child array. A set validity bit means a non-null parent. For a valid parent at position i, child elements run from offsets[i] up to, but excluding, offsets[i+1]. These offsets count child positions, rather than bytes.

A List array of length n has n+1 offsets in this layout. Offsets remain nondecreasing across null parents too. The format permits a null parent's segment to contain arbitrary content: its span does not establish a logical list length or recovered observations.

This article uses an unsliced List<Int32> with 32-bit offsets. LargeList has 64-bit offsets. ListView has a separate sizes buffer and different ordering rules; the adjacent-offset reasoning here concerns List. The schema identifies which representation the consumer receives.

References: [AL1](https://arrow.apache.org/docs/21.0/format/Columnar.html#list-layout), [AL3](https://raw.githubusercontent.com/apache/arrow/apache-arrow-21.0.0/format/Schema.fbs)

## AL-71: five stored integers produce two logical elements

AL-71 is an original stipulated layout for three fictional records, A, B and C. The parent List<Int32> has length 3, validity bits [1,0,1] in record order and null count 1. Its offsets are [0,2,5,5]. The child Int32 array contains [10,20,30,40,50], has length 5 and contains no nulls. The view begins at parent offset zero. This is a reasoning example, not a captured dataset or executed conversion.

With Arrow's least-significant-bit numbering, the three parent validity bits occupy a byte whose value is 0x05 when the unused bits are zero. A is valid and spans child positions 0 and 1, so its list is [10,20]. B is null. Its offsets enclose positions 2, 3 and 4, but the stored 30, 40 and 50 supply no logical elements through B. C is valid and spans [5,5), so its list is empty.

The reconstructed sequence is therefore [[10,20], null, []]. All three records remain present. A supplies two child elements, B supplies no logical list, and C supplies a valid list containing zero elements. The five-position child array and the three-position parent array are different populations.

References: [AL1](https://arrow.apache.org/docs/21.0/format/Columnar.html#list-layout)

## Choose the population before reporting a count or average

AL-71 contains three parent records, two valid lists, one null list and one valid empty list. Across the valid lists there are two child slots, both non-null in this example. Calling the child array's five integers five observed features would import three entries that the parent representation masks.

The valid lists have lengths 2 and 0. Their mean length is (2+0)/2 = 1 element per valid list. Restricting the question to nonempty valid lists gives one list of length 2 and a mean of 2. Both calculations have explicit populations. Dividing the raw child count 5 by the two valid parents gives 2.5, which describes neither requested mean.

B's physical span of three positions supplies no missing logical length to add to either calculation. A request for a mean over all three records would need a separate application rule for the null parent. The layout alone cannot assign B an empty list, three observations or a replacement value.

## A null list, an empty list and [null] preserve different information

B and C contribute no child elements to the logical element population in AL-71, but their parent states differ. B supplies a null list; C supplies a valid empty list. A return containing only [10,20] loses the information needed to distinguish those two records. The same element sequence could accompany a different ordering of null and empty parents.

Now stipulate a separate variant in which A's second child slot is null. Keep the parent validity and offsets unchanged. The logical values become [[10,null], null, []]. A remains a valid list of length 2, while only one of its two child slots supplies a non-null integer. The parent null count remains 1. Child validity adds a separate distinction inside a valid parent.

For AI input review, these states answer different questions: whether a record supplies a list, how many slots that list contains, and which slots supply numbers. None establishes why a source value is unavailable or how a model should treat it. In particular, a valid empty list does not, by itself, prove that no real-world events occurred.

References: [AL1](https://arrow.apache.org/docs/21.0/format/Columnar.html#list-layout)

## Underlying child access can expose more than the logical elements

PyArrow 21.0.0 documents that ListArray.values exposes underlying child values, including segments backing null list elements. Its documentation distinguishes that access from logical unnesting and also notes that underlying values ignore a parent array's slicing offset.

Applied conceptually to AL-71, a physical child view can expose all five integers without establishing that all five belong in the requested input population. A child array with no nulls still leaves B's parent mask unresolved. The recipient needs the association between each parent, its segment and its validity.

A report described only as exported values therefore needs a more precise meaning. Does it describe backing storage, logical list elements, or an application transformation? A flat numeric sequence can be useful, but it cannot preserve the three parent states and their membership on its own.

References: [AL2](https://arrow.apache.org/docs/21.0/python/generated/pyarrow.ListArray.html#pyarrow.ListArray.values)

## Twenty stored child bytes do not mean twenty bytes of logical numbers

Each Int32 slot occupies 4 bytes. AL-71's five child integers account for 5 × 4 = 20 bytes of child value storage before padding. The two logical non-null elements account for 2 × 4 = 8 bytes within that storage. The other 12 bytes lie in B's masked segment; they remain stored bytes in this example.

The four 32-bit offsets add 4 × 4 = 16 bytes. Three parent validity bits require at least one packed byte. Stipulate that the all-valid child omits its optional validity bitmap. The named, unrounded buffer content then totals 20 + 16 + 1 = 37 bytes.

That 37-byte figure excludes alignment, padding, metadata, allocator overhead and serialization framing. It is neither an actual allocation measurement nor a transfer-size estimate. The 8-byte logical-number subtotal excludes the offsets and validity needed to preserve this representation's parent information.

References: [AL1](https://arrow.apache.org/docs/21.0/format/Columnar.html#list-layout)

## Different child storage can preserve the same three values

Consider a second stipulated representation with the same parent type, length, validity and null count. Its offsets are [0,2,2,2], and its all-valid child array contains only [10,20]. A still supplies [10,20], B is still null, and C is still empty. Both representations therefore supply the same ordered parent values.

The second layout has 8 bytes of child values, 16 bytes of offsets and the same one-byte minimum parent bitmap: 25 named, unrounded bytes under the previous assumptions. The 12-byte difference comes from the segment hidden by B in AL-71. It demonstrates how backing storage can differ while logical values stay equal; it predicts no implementation's memory savings or runtime.

By comparison, returning [[10,20], [30,40,50], []] would make B valid and introduce a list that AL-71 did not supply. Returning [[10,20], [], []] would replace B's null state with an empty list. These are changes in parent values, even if a downstream numeric summary happens to hide the difference.

References: [AL1](https://arrow.apache.org/docs/21.0/format/Columnar.html#list-layout)

## Require the nested input meaning that the consumer actually needs

For a request to preserve AL-71's logical input, the useful return identifies A as [10,20], B as null and C as []. A provider can preserve that meaning with either illustrated backing layout. A requirement to preserve the original physical representation would be a separate, stricter request.

Before comparing offers, have the appointed data-platform or application provider explain parent membership and order, list boundaries, parent validity and child validity at the consumer boundary. If the requested output intentionally removes parent structure, identify what record mapping and null-versus-empty information the consuming task still needs. A child-buffer count alone cannot settle that requirement.

The related Arrow dictionary article addresses which category an index denotes. The SQL null article addresses predicate membership and aggregate inputs after values are interpreted. Here the prior decision is which child slots are elements of valid nested lists at all.

## Bring a separate equipment-connection need to the cabling inquiry

If the surrounding AI project also requires physical connections between compute and storage equipment, Leviathan's Structured Cabling & Fiber Installation service is the relevant route for that work. Use the related project inquiry to share approved endpoints, routes, material responsibilities and required labels and connection records. Arrow interpretation and application behavior remain with the appointed software providers.

References: [L1](https://www.leviathansystems.co/services/structured-cabling)

## Primary sources

- [AL1 — Apache Arrow 21.0.0: Columnar Format 1.5 — List layout, validity and physical buffers](https://arrow.apache.org/docs/21.0/format/Columnar.html#list-layout). Checked 2026-10-20. Variable-size List parent validity and child segments, offset invariants, primitive slot widths and padding. The worked layouts are stipulated examples; buffer subtotals exclude allocation and transport overhead.
- [AL2 — PyArrow 21.0.0: ListArray.values](https://arrow.apache.org/docs/21.0/python/generated/pyarrow.ListArray.html#pyarrow.ListArray.values). Checked 2026-10-20. Documents underlying child access versus logical element membership, including segments backing null parents and the effect of parent slicing offsets.
- [AL3 — Apache Arrow 21.0.0: Schema.fbs — List, LargeList and ListView types](https://raw.githubusercontent.com/apache/arrow/apache-arrow-21.0.0/format/Schema.fbs). Checked 2026-10-20. Version-tagged type definitions distinguish List from 64-bit-offset LargeList and the offset-plus-size ListView representation.
- [L1 — Leviathan Systems: Structured Cabling & Fiber Installation](https://www.leviathansystems.co/services/structured-cabling). Checked 2026-10-20. Agreed physical fiber and copper installation, endpoints, routes, labels and handoff records. Supports a separately defined equipment-connection inquiry.

## Related Leviathan resources

- [Arrow dictionaries and logical category values](/articles/arrow-dictionary-indices-categorical-input-values)
- [SQL nulls, predicate membership and aggregate populations](/articles/ai-dataset-sql-null-predicate-aggregate-population)
- [Structured Cabling & Fiber Installation](/services/structured-cabling)
- [Discuss a separate physical equipment-connection project](/contact?service=structured-cabling&from=%2Farticles%2Farrow-list-validity-offsets-visible-child-values)
- [Data Center Remediation Punch List: Record, Assign and Close Defects](/articles/data-center-remediation-punch-list)

## Does the wider project need physical equipment connections?

Share approved equipment endpoints, routes, material responsibilities and required connection records to discuss a separately defined physical cabling scope.

[Discuss the scope](/contact?service=structured-cabling&from=/articles/arrow-list-validity-offsets-visible-child-values)