Skip to content

fix: BitMaskedArray mask inversion and placeholder detection - #4092

Merged
ianna merged 10 commits into
mainfrom
henryiii/fix-bitmasked-python
Jul 1, 2026
Merged

ianna merged 10 commits into
mainfrom
henryiii/fix-bitmasked-python

Conversation

@henryiii

@henryiii henryiii commented Jun 10, 2026 •

Copy link
Copy Markdown
Member

🤖 AI text below 🤖

Summary

Three silent bugs fixed across BitMaskedArray, ByteMaskedArray, and UnionArray:

1. BitMaskedArray.to_BitMaskedArray — bitmask inversion (data corruption)
src/awkward/contents/bitmaskedarray.py:443 used nplike.logical_not(self._mask.data) — a boolean byte-level NOT — to flip valid_when when lsb_order is unchanged. This corrupts packed bitmasks: e.g. 0b10011010 becomes [False] rather than the correct bitwise inverse 0b01100101. Changed to ~self._mask.data (bitwise invert), matching the lsb_order-change branch at line 459.

2. BitMaskedArray._unique — spurious out._content unwrap
src/awkward/contents/bitmaskedarray.py:659-668 had a conditional that returned out._content when negaxis is not None, stripping the option-type wrapper from the result. The sibling ByteMaskedArray._unique (bytemaskedarray.py) returns out unchanged. Removed the branch to match ByteMaskedArray semantics.

3. _is_getitem_at_placeholder checks Index wrapper instead of buffer
Three methods checked isinstance(self._mask/tags/index, PlaceholderArray) against the Index wrapper object, which is never a PlaceholderArray. Siblings ListOffsetArray (listoffsetarray.py:314) and IndexedArray (indexedarray.py:294) correctly check .data. Fixed in:

  • src/awkward/contents/bitmaskedarray.py:499-502
  • src/awkward/contents/bytemaskedarray.py:380-383
  • src/awkward/contents/unionarray.py:587-591

Test plan

  • Regression tests in tests/test_4092_bitmasked_fixes.py: one test per bug fixed (Fix 1: inversion, Fix 2: unique consistency)
  • Existing BitMasked tests pass: test_3033_flatten_bitmaskedarray.py, test_0127_tomask_operation.py, test_1260_simplify_masked_option_types.py
  • All changes linted with prek -a --quiet (clean)

AI-assisted contribution: fixes identified and implemented with Claude Code (Anthropic), from an automated multi-agent review.

henryiii added a commit that referenced this pull request Jun 10, 2026
Covers to_BitMaskedArray mask inversion, _unique consistency between
BitMasked and ByteMasked option types, and _is_getitem_at_placeholder
buffer check propagation.

Assisted-by: ClaudeCode:claude-sonnet-4-6
@codecov

codecov Bot commented Jun 10, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (main@da9dbfc). Learn more about missing BASE report.
✅ All tests successful. No failed tests found.

Additional details and impacted files
Files with missing lines Coverage Δ
src/awkward/contents/bitmaskedarray.py 77.45% <100.00%> (ø)
src/awkward/contents/bytemaskedarray.py 88.61% <100.00%> (ø)
src/awkward/contents/unionarray.py 87.17% <100.00%> (ø)

henryiii and others added 4 commits June 11, 2026 15:39
- Use bitwise `~` instead of `logical_not` in `to_BitMaskedArray` when
  inverting `valid_when` with the same `lsb_order`, matching the
  different-lsb_order branch and preventing silent data corruption.
- Remove erroneous `out._content` unwrap in `BitMaskedArray._unique` so
  it matches `ByteMaskedArray._unique` semantics (both now delegate to
  `IndexedOptionArray64._unique` and return the result unchanged).
- Fix `_is_getitem_at_placeholder` in `BitMaskedArray`, `ByteMaskedArray`,
  and `UnionArray` to check `.data` on the Index wrapper, consistent with
  `ListOffsetArray` and `IndexedArray` siblings.

Assisted-by: ClaudeCode:claude-sonnet-4-6
Covers to_BitMaskedArray mask inversion, _unique consistency between
BitMasked and ByteMasked option types, and _is_getitem_at_placeholder
buffer check propagation.

Assisted-by: ClaudeCode:claude-sonnet-4-6
Remove redundant round-trip, big-endian, and ByteMaskedArray tests that
don't cover changed code paths. Remove placeholder tests that pass even
without the fix. Rename file to use underscores per project convention.
Two focused tests remain: one per distinct bug fixed.

Assisted-by: ClaudeCode:claude-sonnet-4-6
@henryiii
henryiii force-pushed the henryiii/fix-bitmasked-python branch from a2418ab to 624fdb2 Compare June 11, 2026 19:44
@TaiSakuma TaiSakuma added the type/fix PR title type: fix (set automatically) label Jun 12, 2026

@ianna ianna left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@henryiii -- looks good! Thanks.

Comment thread src/awkward/contents/bitmaskedarray.py Resolved
@github-actions

github-actions Bot commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

The documentation preview is ready to be viewed at http://preview.awkward-array.org.s3-website.us-east-1.amazonaws.com/PR4092

@ianna
ianna merged commit c1eb3de into main Jul 1, 2026
38 checks passed
@ianna
ianna deleted the henryiii/fix-bitmasked-python branch July 1, 2026 21:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type/fix PR title type: fix (set automatically)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants