[bobcallaway]

Motivation

CPU + heap-alloc profiles captured from a production rekor-server showed the JSON unmarshal of ProposedEntry on the CreateLogEntry handler dominating request cost:

• UnmarshalProposedEntry — 31 % of total CPU (cumulative)
• UnmarshalProposedEntry — 60 % of total alloc_space
• (*DSSE).UnmarshalJSON alone — 27.8 % of allocs (~594 MB in a 10 s trace)
• json.Decoder.refill — 42 % of alloc_space, driven by repeated 4 KB-growing buffers on multi-hundred-KB bodies

Root cause is the polymorphic-unmarshal pattern go-swagger emits for a discriminated base type: the request body is decoded three times on the hot path:

• once via io.ReadAll + a sniff for kind
• once as the concrete type via the runtime Consumer, and
• a third time inside the per-kind generated UnmarshalJSON

Change

Replace the polymorphic sniff-then-consumer-decode with a single JSON pass into a shim, then construct the concrete value directly from the shim.

This was done via overriding the swagger code-gen template rather than patching the generated code directly.

Because every kind's Spec field is a type X any alias, the resulting Spec (a map[string]any tree with UseNumber() semantics) is byte-for-byte identical to what the per-kind generated UnmarshalJSON produced.

Performance Improvements from attached benchmarks

Small real payloads (~700 B – 1 KB)

Kind	Time (before → after)	Bytes/op	Allocs/op
dsse	15.0 µs → 3.64 µs (−75.7%)	15.1 KiB → 4.8 KiB (−68.5%)	72 → 31 (−56.9%)
hashedrekord	11.6 µs → 3.07 µs (−73.7%)	13.8 KiB → 4.8 KiB (−65.2%)	83 → 43 (−48.2%)
intoto	16.7 µs → 4.15 µs (−75.2%)	15.5 KiB → 5.2 KiB (−66.7%)	79 → 38 (−51.9%)

Large real signed payloads — time (sec/op)

Case	Before → After
dsse / 256 KiB	6.71 ms → 1.52 ms (−77.4%)
dsse / 1 MiB	25.94 ms → 5.93 ms (−77.2%)
dsse / 32 MiB	807 ms → 187 ms (−76.9%)
intoto_v002 / 256 KiB	6.55 ms → 1.38 ms (−78.9%)
intoto_v002 / 1 MiB	25.40 ms → 5.35 ms (−78.9%)
intoto_v002 / 32 MiB	789 ms → 170 ms (−78.5%)

Large real signed payloads — allocated bytes (B/op)

Case	Before → After
dsse / 256 KiB	7.3 MiB → 1.9 MiB (−74.1%)
dsse / 1 MiB	29.7 MiB → 7.6 MiB (−74.5%)
dsse / 32 MiB	926 MiB → 242 MiB (−73.9%)
intoto_v002 / 256 KiB	6.9 MiB → 1.4 MiB (−78.9%)
intoto_v002 / 1 MiB	27.9 MiB → 5.8 MiB (−79.3%)
intoto_v002 / 32 MiB	869 MiB → 185 MiB (−78.7%)

Large real signed payloads — allocation count (allocs/op)

Case	Before → After
dsse / 256 KiB	125 → 39 (−68.8%)
dsse / 1 MiB	139 → 41 (−70.5%)
dsse / 32 MiB	174 → 46 (−73.6%)
intoto_v002 / 256 KiB	163 → 71 (−56.4%)
intoto_v002 / 1 MiB	177 → 73 (−58.8%)
intoto_v002 / 32 MiB	212 → 78 (−63.2%)

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 27.53%. Comparing base (488eb97) to head (8d90da0).
⚠️ Report is 719 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff             @@
##             main    #2880       +/-   ##
===========================================
- Coverage   66.46%   27.53%   -38.93%     
===========================================
  Files          92      191       +99     
  Lines        9258    20282    +11024     
===========================================
- Hits         6153     5585      -568     
- Misses       2359    13831    +11472     
- Partials      746      866      +120     

Flag	Coverage Δ
e2etests	49.49% <ø> (+1.93%)	⬆️
unittests	18.57% <100.00%> (-29.12%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.