struct (un)packing of half-precision may turn NaN into ±inf

Bug report

Bug description:

import struct
f = struct.unpack(">d", bytes.fromhex("7FF0000000080001"))[0]
print(f)  # nan

f2 = struct.unpack(">f", struct.pack(">f", f))[0]
print(f2)  # inf

Repeating this with FFF0000000080001 you'll see nan turn into -inf.

This only happens in Python 3.14+ (not 3.13). I'm pretty sure that this fix is causing it: #130317

Preserving the signaling/quieting bit is a great idea, I find. But since the mantissa is being truncated, one must ensure that the narrower width mantissa doesn't end up being all 0 bits as that represents inf instead of nan.

CPython versions tested on:

3.14

Operating systems tested on:

Linux

Linked PRs

Preserving the signaling/quieting bit is a great idea, I find.

It certainly does make sense for same floating-point format on round-trip.

But since the mantissa is being truncated, one must ensure that the narrower width mantissa doesn't end up being all 0 bits

I see following alternatives:

set some high bit of payload
make low-precision nan a quiet

(2) always happens in C for type-cast:

$ gcc a.c && ./a.out
No high-bit set:
+nan = 0 11111111111 0000000000000000000000000000000010000000000000000001
positive signaling nan with payload: 524289
type-cast to float:
+nan = 0 11111111 10000000000000000000000
positive quiet nan with payload: 0
High-bit set:
+nan = 0 11111111111 0000000010000000000000000000000000000000000000000001
positive signaling nan with payload: 8796093022209
type-cast to float:
+nan = 0 11111111 10000000100000000000000
positive quiet nan with payload: 16384

Details

#include <stdint.h>
#include <stdio.h>
void print_bits(const size_t size, void const * const ptr,
                const unsigned long long layout)
{
    unsigned char *b = (unsigned char*) ptr;
    unsigned char byte; 
    long long i, j;
    for (i = size - 1; i >= 0; i--) {
        for (j = 7; j >= 0; j--) {
            if (layout & 1ULL<<(8*i+j)) {
                printf(" ");
            }
            byte = (b[i] >> j) & 1;
            printf("%u", byte);
        }
    }
    printf("\n");
}
const unsigned long long float_layout = (1 << 30) + (1 << 22);
const unsigned long long double_layout = (1ULL << 62) + (1ULL << 51);
void print_float(const float *x)
{
    uint32_t v = *((uint32_t *)x);
    uint32_t man = (v - (v & (0xfffffUL << 23)));
    int sign = (v >> 31) & 1, bexp = ((v & (0xff << 23)) >> 23);
    int exp = bexp;
    if (exp && exp != 255) {
        exp -= 127;
    }
    printf("%+a = ", *x);
    print_bits(4, &v, float_layout);
    if (bexp != 255) {
        char s[10];
        sprintf(s, "%.7x", man ? ((1<<24)+man)<<1 : man);
        printf("%s0x%d.%sp%+d\n", sign ? "-" : "+",
               (bexp != 0), man ? s+1:"0", bexp ? exp : -127);
    }
    else {
        if (man) {
            printf("%s %s nan with payload: %u\n",
                   sign ? "negative" : "positive",
                   man & (1 << 22) ? "quiet" : "signaling",
                   man & ~(1 << 22));
        }
        else {
            printf("%sinfn", sign ? "-" : "+");
        }
    }
}
void print_double(const double *x)
{
    uint64_t v = *((uint64_t *)x);
    uint64_t man = (v - (v & (0xffffffffUL << 52)));
    int sign = (v >> 63) & 1, bexp = ((v & (0x7ffUL << 52)) >> 52);
    int exp = bexp;
    if (exp && exp != 2047) {
        exp -= 1023;
    }
    printf("%+a = ", *x);
    print_bits(8, &v, double_layout);
    if (bexp != 2047) {
        printf("%s0x%d.%.13lxp%+d\n", sign ? "-" : "+",
               (bexp != 0), man, bexp ? exp : -1023);
    }
    else {
        if (man) {
            printf("%s %s nan with payload: %lu\n",
                   sign ? "negative" : "positive",
                   man & (1UL << 51) ? "quiet" : "signaling",
                   man & ~(1UL << 51));
        }
        else {
            printf("%sinf\n", sign ? "-" : "+");
        }
    }
}
int main()
{
    uint64_t v = 0x7FF0000000080001UL;
    double d = *((double *)&v);
    float f = (float)d;
    printf("No high-bit set:\n");
    print_double(&d);
    printf("type-cast to float:\n");
    print_float(&f);
    v = 0x7FF0080000000001UL;
    d = *((double *)&v);
    f = (float)d;
    printf("High-bit set:\n");
    print_double(&d);
    printf("type-cast to float:\n");
    print_float(&f);
    return 0;
}

pythongh-141042: fix sNaN's (un)packing for mixed floating-point formats

(2) always happens in C for type-cast:

Is this guaranteed across compilers and platforms? I know for instance, that LLVM doesn't make many guarantees about how NaNs behave in this regard. https://llvm.org/docs/LangRef.html#floatnan

No, of course. See e.g. #133328.

Note that it's not even guaranteed by IEEE 754 that for a NaN, the first bit of the significand field is 0 for signaling and 1 for quiet - it could be the other way around (0 for quiet, 1 for signaling). IEEE 754 recommends the first way but doesn't require it. (The verb used is "should" rather than "shall".)

The relevant section of IEEE 754-2019 is §6.2.1 ("NaN encodings in binary interchange formats").

Fortunately, this was silently standardized by CPython long time ago. Maybe it should be documented, as we have at least one buildbot with reversed meaning of the first bit.

this was silently standardized by CPython long time ago

Can you say more about this? How can CPython standardise something that's determined by the hardware, and that differs from machine to machine? That doesn't seem possible - CPython can't dictate how the hardware behaves.

How can CPython standardise something that's determined by the hardware, and that differs from machine to machine?

Ah, sorry for a bad joke. This was in the context of (un)packing of various floating-point formats. While for floats/doubles we rely on hardware (on IEEE boxes), for the half-precision format its not. Here how it happens in v3.13:

cpython/Objects/floatobject.c

Lines 2079 to 2088 in 7a3b035

    
               else if (Py_IS_NAN(x)) { 
        
                   /* There are 2046 distinct half-precision NaNs (1022 signaling and 
        
                      1024 quiet), but there are only two quiet NaNs that don't arise by 
        
                      quieting a signaling NaN; we get those by setting the topmost bit 
        
                      of the fraction field and clearing all other fraction bits. We 
        
                      choose the one with the appropriate sign. */ 
        
                   sign = (copysign(1.0, x) == -1.0); 
        
                   e = 0x1f; 
        
                   bits = 512; 
        
               }

No doubts, here clearly stated that some very specific qNaN's (two) have 0b1000000000 mantissa.

PS: Recommended encoding was proposed by the IEEE 754-2008.

gh-141042: fix sNaN's packing for mixed floating-point formats (#141107)

pythongh-141042: fix sNaN's packing for mixed floating-point formats (p…

Thanks for your bug report.

Fixed in the main branch by the change 23d85a2. The backport to 3.14 will be merged after the CI tests pass.

[3.14] gh-141042: fix sNaN's packing for mixed floating-point formats (…

pythongh-141042: fix sNaN's packing for mixed floating-point formats (p…

Navigation Menu

Repository navigation

struct (un)packing of half-precision may turn NaN into ±inf #141042

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

	else if (Py_IS_NAN(x)) {
	/* There are 2046 distinct half-precision NaNs (1022 signaling and
	1024 quiet), but there are only two quiet NaNs that don't arise by
	quieting a signaling NaN; we get those by setting the topmost bit
	of the fraction field and clearing all other fraction bits. We
	choose the one with the appropriate sign. */
	sign = (copysign(1.0, x) == -1.0);
	e = 0x1f;
	bits = 512;
	}

struct (un)packing of half-precision may turn NaN into ±inf #141042

Description

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

Activity

skirpichev commented on Nov 5, 2025

robsdedude commented on Nov 6, 2025

skirpichev commented on Nov 6, 2025

mdickinson commented on Nov 6, 2025

skirpichev commented on Nov 6, 2025

mdickinson commented on Nov 6, 2025

skirpichev commented on Nov 6, 2025

vstinner commented on Nov 12, 2025

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions