Describe the bug
The fix for #312 ("Short NUL-only keys incorrectly detected as duplicates", 2.14.0) was applied to the
blocking SmileParser but never to the non-blocking NonBlockingParserBase. The async parser still
zero-pads partial "quads" when looking names up in ByteQuadsCanonicalizer, so a short name collides
with a longer, NUL-prefixed one and the wrong name is returned.
Version information
Present on 2.21 through 3.x. Introduced when #312 was fixed in 2.14.0 — the blocking parser got
_padQuadForNulls() / _padLastQuad(), the async one did not.
Smile only; there is no non-blocking CBOR parser, so the CBOR half of #312 is unaffected.
To Reproduce
String n1 = new String(new char[] { 0, 0, 0, 'a' }); // "\0\0\0a"
String n2 = "a";
Map<String, Object> m = new LinkedHashMap<>();
m.put(n1, 1);
m.put(n2, 2);
byte[] doc = new SmileMapper().writeValueAsBytes(m);
// blocking parser: correct
try (JsonParser p = new SmileMapper().createParser(doc)) {
p.nextToken();
while (p.nextToken() == JsonToken.PROPERTY_NAME) {
System.out.println(escaped(p.currentName()));
p.nextToken();
}
}
// non-blocking parser: second name is wrong
try (JsonParser p = new SmileMapper().reader().createNonBlockingByteArrayParser()) {
ByteArrayFeeder f = (ByteArrayFeeder) p.nonBlockingInputFeeder();
f.feedInput(doc, 0, doc.length);
f.endOfInput();
JsonToken t;
while ((t = p.nextToken()) != null) {
if (t == JsonToken.PROPERTY_NAME) {
System.out.println(escaped(p.currentName()));
}
}
}
Output:
SYNC : [<NUL><NUL><NUL>a, a]
ASYNC: [<NUL><NUL><NUL>a, <NUL><NUL><NUL>a]
Expected behavior
Both parsers should report the two distinct property names, "\0\0\0a" and "a".
Root cause
NonBlockingParserBase._findDecodedFromSymbols() builds the quad for a 1-4 byte name by
accumulating bytes with 8-bit shifts, leaving the unused high bytes as zeros:
if (len < 5) {
int q = inBuf[inPtr] & 0xFF;
if (--len > 0) {
q = (q << 8) + (inBuf[++inPtr] & 0xFF);
...
}
_quad1 = q;
return _symbols.findName(q);
}
So "a" (1 byte) and "\0\0\0a" (4 bytes) both produce 0x00000061 and hit the same symbol table
entry — whichever name was interned first wins for both. SmileParser avoids this by padding the
unused high bytes with 1s instead (_padQuadForNulls, _padLastQuad).
The same zero-padding is used for the other partial quads in that class: the q2 tail in the
len < 9 branch, and the trailing 1-3 bytes in _findDecodedLonger(). NonBlockingByteArrayParser._finishLongPropertyName() has it too.
Suggested fix
Apply the same 1s-padding to every partial quad in the async parser, matching SmileParser.
Worth a test asserting the blocking and non-blocking parsers agree on names of every length
1..N, including ones with embedded and trailing NULs — the two currently diverge silently.
Targeting 2.21 since the bug is present there.
Describe the bug
The fix for #312 ("Short NUL-only keys incorrectly detected as duplicates", 2.14.0) was applied to the
blocking
SmileParserbut never to the non-blockingNonBlockingParserBase. The async parser stillzero-pads partial "quads" when looking names up in
ByteQuadsCanonicalizer, so a short name collideswith a longer, NUL-prefixed one and the wrong name is returned.
Version information
Present on
2.21through3.x. Introduced when #312 was fixed in 2.14.0 — the blocking parser got_padQuadForNulls()/_padLastQuad(), the async one did not.Smile only; there is no non-blocking CBOR parser, so the CBOR half of #312 is unaffected.
To Reproduce
Output:
Expected behavior
Both parsers should report the two distinct property names,
"\0\0\0a"and"a".Root cause
NonBlockingParserBase._findDecodedFromSymbols()builds the quad for a 1-4 byte name byaccumulating bytes with 8-bit shifts, leaving the unused high bytes as zeros:
So
"a"(1 byte) and"\0\0\0a"(4 bytes) both produce0x00000061and hit the same symbol tableentry — whichever name was interned first wins for both.
SmileParseravoids this by padding theunused high bytes with 1s instead (
_padQuadForNulls,_padLastQuad).The same zero-padding is used for the other partial quads in that class: the
q2tail in thelen < 9branch, and the trailing 1-3 bytes in_findDecodedLonger().NonBlockingByteArrayParser._finishLongPropertyName()has it too.Suggested fix
Apply the same 1s-padding to every partial quad in the async parser, matching
SmileParser.Worth a test asserting the blocking and non-blocking parsers agree on names of every length
1..N, including ones with embedded and trailing NULs — the two currently diverge silently.
Targeting
2.21since the bug is present there.