Skip to content

Nearly all reducers and sorting raise KeyError on float16/float128/complex256 #4259

Closed

Description

@TaiSakuma

Version of Awkward Array

2.12.0

Description and code to reproduce

float16 is a documented primitive — NumpyArray's docstring lists it ("floating point 16-bit, if your system's NumPy supports it"), ak.types.numpytype registers it, ak.from_numpy accepts it without warning, and ak.from_arrow maps Arrow halffloat to it, so Arrow and Parquet input reaches this failure without the user choosing the dtype. When a float16 leaf reaches a compiled kernel, the kernel-table lookup raises a KeyError whose message is the lookup key:

>>> import numpy as np
>>> import awkward as ak
>>> ak.sort(ak.Array(np.array([1.0, 2.0], dtype=np.float16)))
KeyError: ('awkward_sort', <class 'numpy.float16'>, <class 'numpy.float16'>, <class 'numpy.int64'>)
>>> a = ak.values_astype(ak.Array([[1.0, 2.0]]), np.float16)  # 1 * var * float16
>>> ak.sum(a, axis=1)
KeyError: ('awkward_reduce_sum', <class 'numpy.float16'>, <class 'numpy.float16'>, <class 'numpy.int64'>)

The KeyError is raised by the unguarded dictionary lookup in _backends/numpy.py: awkward_cpp.cpu_kernels.kernel[index].

Which operations fail depends on whether a kernel is reached (probed on 1 * var * float16 at axis=None, 0, 1, and -1):

operations axis=None integer axis
ak.sort, ak.argsort n/a KeyError
ak.all, ak.any, ak.prod, ak.count_nonzero, ak.argmin, ak.argmax KeyError KeyError
ak.sum, ak.min, ak.max, ak.mean, ak.std succeeds KeyError
ak.count succeeds succeeds

(ak.sort and ak.argsort reject axis=None for every dtype: TypeError: 'axis' must be an integer, not None. Lookup keys observed from other operations: awkward_reduce_prod_bool, awkward_reduce_argmin, awkward_reduce_max.)

ak.sum, ak.min, and ak.max succeed at axis=None because the whole-array reducers (AxisNoneSum, AxisNoneMin, AxisNoneMax) call the NumPy ufunc instead of a kernel; ak.mean and ak.std are built from those and from AxisNoneSumOfSquares. ak.count, the one exception, never dispatches on the value dtype. The derived operations follow their bases: ak.var, ak.ptp, ak.moment, ak.corr, ak.covar, ak.linear_fit, and the nan* variants raise the same KeyError at an integer axis.

float128 and complex256 fail the same lookup on the platforms where NumPy provides them (macOS arm64 has neither): continuous integration on Linux observed KeyError: ('awkward_reduce_prod_bool', <class 'numpy.bool'>, <class 'numpy.longdouble'>, <class 'numpy.int64'>) and the numpy.clongdouble counterpart from ak.all on 0 * float128 and 0 * complex256 arrays — the kernel keys carry NumPy's other names for the dtypes.

Cause

There are no float16 kernels: kernel-specification.yml (at the repository root, the source for both the CPU and CUDA kernel tables) contains no float16 entries — nor float128 or complex256 entries — so the GPU backend lacks them as well. The dtype is neither mapped onto an existing kernel nor refused with a documented error before the lookup, as datetime sums are (ValueError: cannot compute the sum (ak.sum) of dtype('<M8[s]')). The mapping route exists: KernelReducer._dtype_for_kernel views datetime64 and timedelta64 as int64, complex128 as float64, and complex64 as float32 before the kernel lookup, so those dtypes reuse existing kernels; float16 is absent from that mapping.

History

jpivarski's dtype roadmap in #392 (2020) recorded "float16: need to find a C++ library for it" (and the same for float128), while for complex64/complex128 it recorded "C++'s built-in std::complex can do this", and they were implemented in #652. float16 received neither kernels nor a mapping, and no issue tracks it (the Claude review in #4085 mentions float16 only in a dead-code note).

Found by hypothesis-awkward.

ðŸĪ– Generated with Claude Code

Activity

  1. added
    bug (unverified)The problem described would be a bug, but needs to be triaged
    on Jul 29, 2026
  2. changed the title [-]`ak.all`, `ak.sort`, `ak.argsort`, and every reducer except one raise `KeyError` on `float16` arrays[/-] [+]`ak.all`, `ak.sort`, `ak.argsort`, and every reducer except one raise `KeyError` on `float16` and `float128` arrays[/+] on Jul 30, 2026
  3. changed the title [-]`ak.all`, `ak.sort`, `ak.argsort`, and every reducer except one raise `KeyError` on `float16` and `float128` arrays[/-] [+]`ak.all`, `ak.sort`, `ak.argsort`, and every reducer except one raise `KeyError` on `float16`, `float128`, and `complex256` arrays[/+] on Jul 30, 2026
  4. self-assigned this
    on Jul 30, 2026
  5. moved this from Todo to In progress in Awkward array 2026 Roadmapon Jul 30, 2026
  6. ianna commented on Jul 30, 2026

    @ianna
    Member

    Hi @TaiSakuma -- I'd like to work on this issue if you don't mind.

  7. TaiSakuma commented on Jul 30, 2026

    @TaiSakuma
    MemberAuthor

    @ianna, thank you. Please go ahead.

  8. added
    bugThe problem described is something that must be fixed
    and removed
    bug (unverified)The problem described would be a bug, but needs to be triaged
    on Jul 30, 2026
  9. changed the title [-]`ak.all`, `ak.sort`, `ak.argsort`, and every reducer except one raise `KeyError` on `float16`, `float128`, and `complex256` arrays[/-] [+]Nearly all reducers and sorting raise `KeyError` on `float16`/`float128`/`complex256`[/+] on Aug 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugThe problem described is something that must be fixed

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions