Repository navigation
Description
Version of Awkward Array
2.13.0
Description and code to reproduce
ak.array_equal returns False for two union arrays with the same values when one side's index into a list-type child is a reordering of the other's and the list's content is a record (or another list). ak.almost_equal, and ak.array_equal with same_content_types=False, return True for the same pair. Reproduced on a clean install of 2.13.0 and of 2.12.0.
import awkward as ak
z = ak.Array([[{"x": 1}], 2, []])
p = z[[1, 2, 0]]
q = ak.Array(p.tolist())
print(p.tolist() == q.tolist())
print(ak.array_equal(p, q))
print(ak.array_equal(q, p))
print(ak.array_equal(p, q, same_content_types=False))
print(ak.almost_equal(p, q))True
False
False
True
True
The slice reorders p's index into the list child to [1, 0]; ak.Array(p.tolist()) rebuilds q with the index [0, 1]. (The two types also differ in content order: q numbers its union contents in order of first appearance, union[int64, var * {x: int64}], against p's union[var * {x: int64}, int64]. The comparison pairs contents by that same order, so this difference is not the cause.)
The same failure with two hand-built unions of identical form, both [[{}], []]:
import numpy as np
import awkward as ak
def union(index, offsets):
return ak.Array(
ak.contents.UnionArray(
ak.index.Index8(np.array([1, 1], dtype=np.int8)),
ak.index.Index64(np.array(index, dtype=np.int64)),
[
ak.contents.RecordArray([], [], 0),
ak.contents.ListOffsetArray(
ak.index.Index64(np.array(offsets, dtype=np.int64)),
ak.contents.RecordArray([], [], 1),
),
],
)
)
left = union([0, 1], [0, 1, 1])
right = union([1, 0], [0, 0, 1])
print(left.type)
print(left.tolist(), right.tolist())
print(left.layout.form == right.layout.form)
print(ak.array_equal(left, right), ak.array_equal(right, left))2 * union[{}, var * {}]
[[{}], []] [[{}], []]
True
False False
The list child's content decides the outcome. With the same tags and indices, ak.array_equal returns True when the list child is var * int64 and False for var * var * int64, var * string, and 1 * var * {}.
Found by hypothesis-awkward: the nightly Property Tests run 32689834692 of 2026-08-24 drew 1 * 5 * union[{}, var * {}] in test_roundtrip of tests/properties/operations/test_to_from_arrow.py. ak.from_arrow(ak.to_arrow(a)) returns an array with the same tolist() and the same form, but ak.array_equal(a, ak.from_arrow(ak.to_arrow(a)), equal_nan=True) is False. The draw's RegularArray uses five of the union's six elements, so to_packed renumbers the original's index to [0, 1, 2, 3, 4] over a projected child, while the reconstruction keeps the drawn order [0, 1, 2, 4, 3] over a child of exactly five elements.
Cause
ak.array_equal forwards same_content_types=True (its default) into ak_almost_equal._impl (ak_array_equal.py L47-L59). _impl packs both layouts (L96-L97), and its visitor compares the two node classes before anything else (L120-L123); the projection of indexed nodes that follows (L125-L129) does not run when the classes differ.
The union branch compares the children pairwise with left.project(i) and right.project(j) (L295-L297). UnionArray.project carries the child lazily, self._contents[index]._carry(nextcarry, True) (unionarray.py L771-L803), so a ListOffsetArray child comes back as a ListArray whose starts and stops follow the union's index (listoffsetarray.py L380-L391). When the tags reference only one content, as in the hand-built pair and the nightly draw, the earlier simplified(..., dropunused=True) (L138-L163) takes the same lazy carry in UnionArray.simplified (unionarray.py L276-L287).
The list-list branch then calls packed_list_content, which converts each side with to_ListOffsetArray64(False) (L116-L118). For a ListArray whose lists are contiguous (starts[1:] equal to stops[:-1]) the conversion keeps the content as it is (listarray.py L292-L317); otherwise it compacts the offsets and _broadcast_tooffsets64 carries the content lazily (listarray.py L485). A lazy carry of a RecordArray returns an IndexedArray (recordarray.py L552-L570), and a lazy carry of a ListOffsetArray returns a ListArray. The projected lists are non-contiguous on the side whose union index is out of order and contiguous on the other, so _impl itself turns two equal forms into IndexedArray against RecordArray (or ListArray against ListOffsetArray), and the visitor returns False at the class check. A NumpyArray is carried eagerly, which is why a var * int64 child compares equal.
The intermediate layouts for the first reproducer (project on the packed union, then to_ListOffsetArray64(False)):
p: project(0) -> ListArray starts=[1, 0] stops=[1, 1]
p: to_ListOffsetArray64(False) -> ListOffsetArray offsets=[0, 0, 1] content=IndexedArray
q: project(1) -> ListArray starts=[0, 0] stops=[0, 1]
q: to_ListOffsetArray64(False) -> ListOffsetArray offsets=[0, 0, 1] content=RecordArray
With same_content_types=False the class check is skipped, the IndexedArray is projected at L125-L129, and the comparison returns True.
History
same_content_types and the class check arrived with ak.array_equal in #3215 (2024-08-14, v2.6.8); the code comment reads "Most firstly, check same_content_types before any transformations". The to_packed calls and packed_list_content come from #2460 (2023-05-16, v2.2.1), whose description by Angus Hollands says it "Adds handling of IndexedArray nodes to ak.almost_equal (these come from carrying a RecordArray)"; that handling is the projection of indexed nodes at L125-L129, which the class check, added later, now precedes.
The union branch with dropunused=True comes from #3773 (2025-12-22, v2.8.12). #3909 (2026-03-12, v2.9.1), for the second item of #3888, added the project() after simplified for the case where the union node itself becomes an IndexedArray; in #3888 ikrommyd wrote that "The union array equality assertion is still problematic but that's not something that PR tackles". The class mismatch reported here occurs deeper, in the content of the union's list child, which that projection does not reach.
The existing union tests use the same index on both sides (tests/test_3773_almost_equal_unionarray.py), and the property tests for ak.array_equal cover only reflexivity and symmetry (tests/properties/operations/test_array_equal.py).
ðĪ Generated with Claude Code
Metadata
Metadata
Assignees
Labels
Type
Projects
- StatusShow more project fieldsIn progress
Activity
aashirvad08 commentedon Aug 31, 2026
picking this up
test: exclude layouts affected by #4316 from the Arrow `test_roundtriâĶ