Repository navigation
Conversation
`accepted_type` and `accepted_header` in `dmr/_compiled/negotiation.py` ignored the `q` weight in two ways. `q=0` means "not acceptable" in RFC 9110, but neither function dropped those media types. `Accept: application/json;q=0` was answered with a `200` and a json body instead of a `406`. Django gets this right, `HttpRequest.accepted_types` filters `quality != 0`, and our own `media_by_precedence` does the same for provided types. `_MediaTypeHeader.priority` also ran `int(100 * float(qparam))` guarded only by `except ValueError`. `Accept: text/html;q=inf,application/json` therefore raised `OverflowError` out of the view and returned a `500`. Quality is now parsed once in `_MediaTypeHeader.__init__` by the new `_parse_quality`, which discards malformed and out of range weights the same way `django.http.request.MediaType.quality` does, so `inf` and `nan` never reach the int conversion. Both public functions skip media types with `quality == 0`.
2aa200b to
b46acc2
Compare
Merging this PR will improve performance by 15.81%
|
| Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|
| ⚡ | test_negotiation_compiled |
357.1 µs | 304.4 µs | +17.31% |
| ⚡ | test_negotiation_raw |
526 µs | 460 µs | +14.33% |
Tip
Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.
Comparing MaxFreedomPollard:fix/accept-header-quality-weights (8ec279f) with master (aafc384)
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #1407 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 475 475
Lines 22204 22238 +34
Branches 685 687 +2
=========================================
+ Hits 22204 22238 +34 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
sobolevn
left a comment
There was a problem hiding this comment.
Thank you! Seems like a valid bug, but we need to:
- Fix the CI
- Maybe think about way to optimize this? 🤔
`HttpRequest.accepted_types` only drops media types with `q=0` since Django 5.2, older versions report them even though nothing is acceptable. The `406` message we build from it is therefore different per version, so the test failed on the `Django~=5.0.0` and `Django~=5.1.0` jobs of `test-extras`, where the message was `accepted types [<MediaType: application/json; q=0>]`. The expected list is now picked by `django.VERSION`, the whole response body is still asserted exactly on every supported version.
Parsing the `q` weight in `_MediaTypeHeader.__init__` added a dict lookup to every media type we build, which CodSpeed reported as a regression in the `accepted_type` and `accepted_header` benchmarks. Media types without parameters now take a fast path. There is nothing to parse for them, so they skip `_parse_content_header`, the `q` lookup and the unescaping regex, which used to run on every media type, including the vast majority that have no escaped quotes at all. The regex now only runs when the value really contains one. Measured with `timeit` on the pure Python implementation over the same case lists the benchmarks use, best of 5 runs of 15 repeats of 2000 iterations: `accepted_type` 15.14us before this branch and 16.13us with it, 12.25us now. `accepted_header` 14.87us, 17.04us and 12.60us. So both are about 1.2x faster than they were before the fix.
…-quality-weights # Conflicts: # CHANGELOG.md
Silences WPS342 in the lint job. The string values are unchanged.
…-quality-weights # Conflicts: # CHANGELOG.md
|
CI should be green now. The lint failures were Master had also moved on since your merge, so I resolved the |
sobolevn
left a comment
There was a problem hiding this comment.
Looks like we miss one line coverage. Is it still in use?
Resolved the `CHANGELOG.md` conflict, keeping the reworded `wemake-services#1402` and `wemake-services#1332` lines from master alongside the three `wemake-services#1407` entries.
The fast path for media types without parameters means
`_parse_content_header` now runs only when a media type contains a `;`
or an escaped quote. A `;` always leaves `accept.find(';')` non
negative, so the `pos == -1` branch is now reached only by a media type
that carries an escaped quote and no parameters, and no test sent one.
That input is still reachable, clients control the `Accept` header, and
the branch is what unescapes such a value before matching. So the case
is tested rather than removed.
I have made things!
Two bugs in
dmr/_compiled/negotiation.py, both about theqweight in theAcceptheader.q=0means "not acceptable" in RFC 9110, butaccepted_typenever dropped those media types, soAccept: application/json;q=0was answered with200and a json body instead of406. Django does drop them:HttpRequest.accepted_typesfiltersquality != 0, anddmr.internal.negotiation.media_by_precedencealready does the same for the types we provide.accepted_headerhad the same hole, and its docstring calls it a faster alternative toHttpRequest.accepts, which returnsFalseforq=0._MediaTypeHeader.prioritycomputedint(100 * float(qparam))in atrythat only caughtValueError(line 159 on master).float('inf')parses fine andint(inf)raisesOverflowError, soAccept: text/html;q=inf,application/jsonraised out of the view and returned500. Any client can send that header.Quality is now parsed once in
_MediaTypeHeader.__init__by a new_parse_quality, which discards malformed and out of range weights and treats them asq=1, the same rule asdjango.http.request.MediaType.quality, soinfandnannever reach the int conversion. Both public functions then skip media types withquality == 0. The,fast path inaccepted_typeis untouched, and the weight is only parsed when aqparameter is actually present.Verification on macOS, Python 3.14.3, Django 5.2.17:
I skipped
tests/test_integration/test_throttlinglocally because it needs a Redis container. Coverage ofdmr/_compiled/is 100% statements and branches with the new cases.The new tests fail on master: 20 parametrized cases in
test_negotiate_compiled.py(compiled and pure), plustest_zero_quality_accept_headerandtest_out_of_range_quality_accept_header, one of them with theOverflowErrorabove.I also built the mypyc extension and ran the unit suite against it, since this module is compiled:
The
q=0change is user visible, so move that changelog line to "Breaking changes" if you would rather have it there.AI Policy
Checklist
Related issues
No existing issue, I found this while reading the negotiation code.