perf(ext/web): apply the base64url op design to standard base64 - #36422
Conversation
Collapse the parallel simdutf_base64url_* helper family into the standard helpers, parameterized by simdutf Base64Options. The loose/strict decode-into pair merges into one function that also carries the output-capacity assert (previously only on the url side), and the unified encode-to-V8-string keeps the url side's debug_assert placement (before set_len, not after). No behavior change.
op_base64_decode_into now reports invalid input as a -1 sentinel instead of throwing, mirroring op_base64url_decode_into: materializing an exception across the op boundary is expensive and the caller treats invalid input as a fallback signal, not an error. Both decode-into ops now share base64_decode_into_slice; the standard op keeps its strict pre-pass for clean padded input. base64Write's fallback goes straight to the cleaning path (base64CleanToBytes) instead of through base64ToBytes, whose eager decode attempt would always throw after a sentinel: the op and the wrapper share the same loose decode, so an input that produced -1 can never pass it. Deletes the orphaned forgivingBase64DecodeInto infra export (no consumers remained).
âĶcode base64Slice split at 4096 between op_base64_encode (#[string] return) and op_base64_encode_from_buffer. Re-measured on main during the denoland#36398 review, the from-buffer op (direct one-byte V8 string) wins at every size, so the split predates that encode path and is now pure overhead. base64Slice uses the from-buffer op unconditionally, matching base64urlSlice. That leaves op_base64_encode with no caller that isn't better served by op_base64_encode_from_buffer, so it is deleted: forgivingBase64Encode and the jupyter image display rewire to the from-buffer op. hash.digest("base64") rides along via forgivingBase64Encode.
op_base64_btoa returns via the shared base64_encode_to_v8_string helper instead of a Rust String, skipping one full copy of the output through the op glue. Error semantics are unchanged: the InvalidCharacterError mapping lives in the JS wrapper's ByteString-conversion catch, which this does not touch.
bartlomieju
left a comment
There was a problem hiding this comment.
Good follow-up to #36398. The collapse of the two helper families into one parameterized set is a clear net win â the base64/base64url pair had already drifted (the capacity assert! and the debug_assert_eq! placement only existed on the url side), and unifying them fixes that drift rather than just deduplicating it.
Things I verified rather than assumed:
- The capacity assert can't fire.
simdutf_base64_decode_intonow assertsoutput.len() >= maximal_binary_length_from_base64(input)on every path. All four call sites satisfy it: the strict pre-pass inop_base64_decode_intoand the fast path inbase64_decode_into_sliceboth guard ontarget.len() >= max_len, and the two stack-buffer sites guard onmax_len <= STACK_BUF_SIZEwith a full-sizeSTACK_BUF_SIZEslice. - The
base64Writefallback rewiring is sound. The old fallback went throughbase64ToBytes, whose first attempt isop_base64_decodeâsimdutf_base64_decode_to_vec(Default, Loose)â exactly the decode the op already ran to produce-1. It could only ever throw. Going straight tobase64CleanToBytesis the same observable behavior. The>U+00FFcase also lands identically: the op2 onebyte-string conversion throws,writtenstays-1, and cleaning strips the invalid chars â same as the old path's inner catch. op_base64_encodehas no remaining callers.forgivingBase64Encodeand both jupyter sites are rewired; everyforgivingBase64Encodecaller (websocket userinfo viacore.encode, filereaderDataUrl,hash.digest) passes a real TypedArray, soTypedArrayPrototypeGetByteLengthis safe there.forgiving_base64_encodekeeps its one out-of-crate consumer incli/ops/jupyter.rs, so it's correct to leave it.forgivingBase64DecodeIntoreally was orphaned â no consumers in tree.op_base64_btoareturning av8::Local<v8::String>doesn't move error semantics.InvalidCharacterErrorcomes from theByteStringconversion on the input, which is unchanged.
Pushed one trivial thing: ext/web/README.md still listed op_base64_encode in "Provided ops". (That list is also missing the four base64url ops added in #36398 â separate pre-existing gap, not yours to fix here.)
Benchmark table and the corpus/property-test verification are more than enough. CI green. LGTM.
|
Native support for base64 conversion is now available using |
|
Ran the numbers: for base64 these ops beat the native methods, so nothing changes here. But the native methods beat our hex paths 3-12x, so I'll follow up with a PR switching Buffer's hex over to them. As soon as this merges ðĪ |
Follow-up to #36398, delivering the two items promised there: the stale
<= 4096op split inbase64Sliceand the transfer of the sentinel/no-split findings to the standard base64 ops.Same-tree A/B, release-lite builds, macOS aarch64, medians of 5, ns/op:
buf.toString("base64"), 32 Bbuf.toString("base64"), 256 Bbuf.toString("base64"), 1 KiBbuf.toString("base64"), 4 KiBhash.digest("base64"), sha256Buffer.from(s, "base64"), junk chars, 346 chBuffer.from(s, "base64"), embedded=, 345 chbuf.write(s, "base64"), junk chars, 346 chbtoa, 96 Bbtoa, 4 KiBbtoa, 16 KiB (external-string path)Guard rows, all within noise (Âą5% across repeated runs): clean padded/unpadded decode (344 ch and 87,384 ch),
writewith clean input, sub-range and 64 KiB encode,atob, and a full re-run of the #36398 base64url table (every row unchanged).Encode at 32 B to 4 KiB now beats Node. The dirty-input rows stay behind Node because the cleaning semantics themselves remain in JS (
base64clean), as on the base64url side.What changed
Four commits, reviewable independently:
Unify the helper families (refactor, no behavior change).
simdutf_base64url_*collapses into the standard helpers, parameterized bysimdutf::Base64Options. Two hardening items from the perf(ext/web): implement base64url encode/decode as simdutf ops #36398 review ride along. The output-capacityassert!now covers every decode-into caller (it guards memory safety: simdutf may write up to the maximal decoded length before detecting an error, and the assert previously existed only on the url side). And thedebug_assert_eq!in the standard encode path moves aboveset_len, the same placement fix the url side got in review.-1sentinel forop_base64_decode_into. Same contract asop_base64url_decode_into: invalid input returns-1instead of materializing an exception across the op boundary (~4.4 Âĩs, measured in perf(ext/web): implement base64url encode/decode as simdutf ops #36398). The strict pre-pass stays, because clean padded input is the common case for the standard alphabet, unlike base64url (base64_std_decode_into_strict_and_loosepins this). One structural find:base64Write's fallback previously went throughbase64ToBytes, whose eager decode attempt would always throw after a sentinel. The op and the wrapper share the same loose decode, so an input that produced-1can never pass it. The fallback now goes straight to the cleaning path (base64CleanToBytes), which is where the 6.5-11.6x on the dirty rows comes from. The orphanedforgivingBase64DecodeIntoinfra export is deleted (no consumers).Drop the 4096 encode split, delete
op_base64_encode. Re-measured during the perf(ext/web): implement base64url encode/decode as simdutf ops #36398 review:op_base64_encode_from_buffer(direct one-byte V8 string) wins at every size, so the split predates that encode path. With the split gone,op_base64_encodehad no caller that isn't better served by the from-buffer op, so it is deleted rather than kept for the case it loses (the same reasoning as perf(ext/web): implement base64url encode/decode as simdutf ops #36398's digest fix).forgivingBase64Encodeand the jupyter image display rewire to the from-buffer op.hash.digest("base64")rides along.btoabuilds its result as a V8 one-byte string directly via the now-shared helper, skipping one full copy of the output through the op glue. Error semantics untouched: theInvalidCharacterErrormapping lives in the JS wrapper's ByteString-conversion catch. This commit is independent and droppable if it raises any concern.Behavior
None intended. Verified:
len % 4 == 1residues, high-Unicode strings, sizes straddling the old 4096 split and the 8192 stack buffer) run throughBuffer.from,buf.writeat offsets,buf.toStringsub-ranges,Buffer.byteLength,atob, andbtoais byte-identical to main and to Node 26.5.0.toString("base64")matches thebtoareference, decode round-trips,atobround-trips. The perf(ext/web): implement base64url encode/decode as simdutf ops #36398 base64url corpus and property test also re-run unchanged against this branch.Testing
base64_decode_into_slicebranches for the standard alphabet, and ashould_panicpin on the capacity assert.buffer_test.ts: std dirty-input matrix (junk, embedded=, url alphabet, >U+00FF chars through the catch path), truncating writes through both the op and the cleaning fallback, and base64 onBufferviews with non-zerobyteOffset(strict and loose write paths, sub-rangetoString, surrounding bytes checked). That last one is the test class the perf(ext/web): implement base64url encode/decode as simdutf ops #36398 review asked for.html/webappapis/atob(760 assertions) andFileAPI/reading-data-section/filereader_readAsDataURLgreen locally.unit_node::buffer_test,unit_node::crypto::crypto_hash_test,unit::text_encoding_test,unit::filereader_test,unit::websocket_testgreen.tools/lint.jsandtools/format.jsclean.Left for follow-ups
base64clean) stays in JS. It only runs on invalid input.atob) does not apply:atobmust return a binary string and already reuses the input allocation on the small path.I used Claude Code to help investigate and write this change.