Skip to content

Async Smile parser misses #312 NUL-padding: short property name collides with NUL-prefixed longer one #761

Description

@cowtowncoder

Describe the bug

The fix for #312 ("Short NUL-only keys incorrectly detected as duplicates", 2.14.0) was applied to the
blocking SmileParser but never to the non-blocking NonBlockingParserBase. The async parser still
zero-pads partial "quads" when looking names up in ByteQuadsCanonicalizer, so a short name collides
with a longer, NUL-prefixed one and the wrong name is returned.

Version information

Present on 2.21 through 3.x. Introduced when #312 was fixed in 2.14.0 — the blocking parser got
_padQuadForNulls() / _padLastQuad(), the async one did not.

Smile only; there is no non-blocking CBOR parser, so the CBOR half of #312 is unaffected.

To Reproduce

String n1 = new String(new char[] { 0, 0, 0, 'a' }); // "\0\0\0a"
String n2 = "a";

Map<String, Object> m = new LinkedHashMap<>();
m.put(n1, 1);
m.put(n2, 2);
byte[] doc = new SmileMapper().writeValueAsBytes(m);

// blocking parser: correct
try (JsonParser p = new SmileMapper().createParser(doc)) {
    p.nextToken();
    while (p.nextToken() == JsonToken.PROPERTY_NAME) {
        System.out.println(escaped(p.currentName()));
        p.nextToken();
    }
}

// non-blocking parser: second name is wrong
try (JsonParser p = new SmileMapper().reader().createNonBlockingByteArrayParser()) {
    ByteArrayFeeder f = (ByteArrayFeeder) p.nonBlockingInputFeeder();
    f.feedInput(doc, 0, doc.length);
    f.endOfInput();
    JsonToken t;
    while ((t = p.nextToken()) != null) {
        if (t == JsonToken.PROPERTY_NAME) {
            System.out.println(escaped(p.currentName()));
        }
    }
}

Output:

SYNC : [<NUL><NUL><NUL>a, a]
ASYNC: [<NUL><NUL><NUL>a, <NUL><NUL><NUL>a]

Expected behavior

Both parsers should report the two distinct property names, "\0\0\0a" and "a".

Root cause

NonBlockingParserBase._findDecodedFromSymbols() builds the quad for a 1-4 byte name by
accumulating bytes with 8-bit shifts, leaving the unused high bytes as zeros:

if (len < 5) {
    int q = inBuf[inPtr] & 0xFF;
    if (--len > 0) {
        q = (q << 8) + (inBuf[++inPtr] & 0xFF);
        ...
    }
    _quad1 = q;
    return _symbols.findName(q);
}

So "a" (1 byte) and "\0\0\0a" (4 bytes) both produce 0x00000061 and hit the same symbol table
entry — whichever name was interned first wins for both. SmileParser avoids this by padding the
unused high bytes with 1s instead (_padQuadForNulls, _padLastQuad).

The same zero-padding is used for the other partial quads in that class: the q2 tail in the
len < 9 branch, and the trailing 1-3 bytes in _findDecodedLonger(). NonBlockingByteArrayParser._finishLongPropertyName() has it too.

Suggested fix

Apply the same 1s-padding to every partial quad in the async parser, matching SmileParser.
Worth a test asserting the blocking and non-blocking parsers agree on names of every length
1..N, including ones with embedded and trailing NULs — the two currently diverge silently.

Targeting 2.21 since the bug is present there.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions