Skip to content

Fix q weights in Accept header parsing - #1407

Merged
sobolevn merged 9 commits into
wemake-services:masterfrom
MaxFreedomPollard:fix/accept-header-quality-weights
Sep 10, 2026
Merged

sobolevn merged 9 commits into
wemake-services:masterfrom
MaxFreedomPollard:fix/accept-header-quality-weights

Conversation

@MaxFreedomPollard

Copy link
Copy Markdown
Contributor

I have made things!

Two bugs in dmr/_compiled/negotiation.py, both about the q weight in the Accept header.

q=0 means "not acceptable" in RFC 9110, but accepted_type never dropped those media types, so Accept: application/json;q=0 was answered with 200 and a json body instead of 406. Django does drop them: HttpRequest.accepted_types filters quality != 0, and dmr.internal.negotiation.media_by_precedence already does the same for the types we provide. accepted_header had the same hole, and its docstring calls it a faster alternative to HttpRequest.accepts, which returns False for q=0.

_MediaTypeHeader.priority computed int(100 * float(qparam)) in a try that only caught ValueError (line 159 on master). float('inf') parses fine and int(inf) raises OverflowError, so Accept: text/html;q=inf,application/json raised out of the view and returned 500. Any client can send that header.

Quality is now parsed once in _MediaTypeHeader.__init__ by a new _parse_quality, which discards malformed and out of range weights and treats them as q=1, the same rule as django.http.request.MediaType.quality, so inf and nan never reach the int conversion. Both public functions then skip media types with quality == 0. The , fast path in accepted_type is untouched, and the weight is only parsed when a q parameter is actually present.

Verification on macOS, Python 3.14.3, Django 5.2.17:

just lint         # ruff, ruff format, flake8 + wemake, slotscheck, import-linter: clean
just type-check   # mypy 668 files, pyright 0 errors, pyrefly 0 diagnostics, ty: clean
pytest -n 2 --inline-snapshot=disable --ignore=tests/test_integration/test_throttling
# 3168 passed, 10 skipped, 53 subtests passed

I skipped tests/test_integration/test_throttling locally because it needs a Redis container. Coverage of dmr/_compiled/ is 100% statements and branches with the new cases.

The new tests fail on master: 20 parametrized cases in test_negotiate_compiled.py (compiled and pure), plus test_zero_quality_accept_header and test_out_of_range_quality_accept_header, one of them with the OverflowError above.

I also built the mypyc extension and ran the unit suite against it, since this module is compiled:

HATCH_BUILD_HOOKS_ENABLE=1 uv build --wheel   # cp314 macosx arm64 wheel
pytest tests/test_unit -o addopts='' -o timeout=30   # 2234 passed, 1 skipped

The q=0 change is user visible, so move that changelog line to "Breaking changes" if you would rather have it there.

AI Policy

  • I have read and agree to the AI Policy, removed any "Co-Authored-By" lines attributing coding agents, and manually reviewed the final result

Checklist

  • I have double checked that there are no unrelated changes in this pull request (old patches, accidental config files, etc)
  • I have created at least one test case for the changes I have made
  • I have updated the documentation for the changes I have made

Related issues

No existing issue, I found this while reading the negotiation code.

`accepted_type` and `accepted_header` in `dmr/_compiled/negotiation.py`
ignored the `q` weight in two ways.

`q=0` means "not acceptable" in RFC 9110, but neither function dropped
those media types. `Accept: application/json;q=0` was answered with a
`200` and a json body instead of a `406`. Django gets this right,
`HttpRequest.accepted_types` filters `quality != 0`, and our own
`media_by_precedence` does the same for provided types.

`_MediaTypeHeader.priority` also ran `int(100 * float(qparam))` guarded
only by `except ValueError`. `Accept: text/html;q=inf,application/json`
therefore raised `OverflowError` out of the view and returned a `500`.

Quality is now parsed once in `_MediaTypeHeader.__init__` by the new
`_parse_quality`, which discards malformed and out of range weights the
same way `django.http.request.MediaType.quality` does, so `inf` and
`nan` never reach the int conversion. Both public functions skip media
types with `quality == 0`.
@MaxFreedomPollard
MaxFreedomPollard force-pushed the fix/accept-header-quality-weights branch from 2aa200b to b46acc2 Compare September 6, 2026 06:38
@codspeed

codspeed Bot commented Sep 6, 2026 •

Copy link
Copy Markdown

Merging this PR will improve performance by 15.81%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 2 improved benchmarks
✅ 14 untouched benchmarks

Performance Changes

Benchmark BASE HEAD Efficiency
⚡ test_negotiation_compiled 357.1 µs 304.4 µs +17.31%
⚡ test_negotiation_raw 526 µs 460 µs +14.33%

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.


Comparing MaxFreedomPollard:fix/accept-header-quality-weights (8ec279f) with master (aafc384)

Open in CodSpeed

@codecov

codecov Bot commented Sep 6, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (aafc384) to head (8ec279f).

Additional details and impacted files
@@            Coverage Diff            @@
##            master     #1407   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files          475       475           
  Lines        22204     22238   +34     
  Branches       685       687    +2     
=========================================
+ Hits         22204     22238   +34     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@sobolevn sobolevn left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! Seems like a valid bug, but we need to:

  • Fix the CI
  • Maybe think about way to optimize this? 🤔

MaxFreedomPollard and others added 6 commits September 9, 2026 01:09
`HttpRequest.accepted_types` only drops media types with `q=0` since
Django 5.2, older versions report them even though nothing is
acceptable. The `406` message we build from it is therefore different
per version, so the test failed on the `Django~=5.0.0` and
`Django~=5.1.0` jobs of `test-extras`, where the message was
`accepted types [<MediaType: application/json; q=0>]`.

The expected list is now picked by `django.VERSION`, the whole response
body is still asserted exactly on every supported version.
Parsing the `q` weight in `_MediaTypeHeader.__init__` added a dict
lookup to every media type we build, which CodSpeed reported as a
regression in the `accepted_type` and `accepted_header` benchmarks.

Media types without parameters now take a fast path. There is nothing
to parse for them, so they skip `_parse_content_header`, the `q` lookup
and the unescaping regex, which used to run on every media type,
including the vast majority that have no escaped quotes at all. The
regex now only runs when the value really contains one.

Measured with `timeit` on the pure Python implementation over the same
case lists the benchmarks use, best of 5 runs of 15 repeats of 2000
iterations: `accepted_type` 15.14us before this branch and 16.13us with
it, 12.25us now. `accepted_header` 14.87us, 17.04us and 12.60us. So
both are about 1.2x faster than they were before the fix.
…-quality-weights

# Conflicts:
#	CHANGELOG.md
Silences WPS342 in the lint job. The string values are unchanged.
…-quality-weights

# Conflicts:
#	CHANGELOG.md
@MaxFreedomPollard

Copy link
Copy Markdown
Contributor Author

CI should be green now. The lint failures were WPS342 on the escaped-quote literals in negotiation.py and the compiled negotiation tests; they're raw strings now, same values.

Master had also moved on since your merge, so I resolved the CHANGELOG.md conflict, keeping your #1417 and #1332 entries alongside the two here.

@sobolevn sobolevn left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like we miss one line coverage. Is it still in use?

Resolved the `CHANGELOG.md` conflict, keeping the reworded `wemake-services#1402` and
`wemake-services#1332` lines from master alongside the three `wemake-services#1407` entries.
The fast path for media types without parameters means
`_parse_content_header` now runs only when a media type contains a `;`
or an escaped quote. A `;` always leaves `accept.find(';')` non
negative, so the `pos == -1` branch is now reached only by a media type
that carries an escaped quote and no parameters, and no test sent one.

That input is still reachable, clients control the `Accept` header, and
the branch is what unescapes such a value before matching. So the case
is tested rather than removed.

@sobolevn sobolevn left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, great work!

@sobolevn
sobolevn merged commit 3927bad into wemake-services:master Sep 10, 2026
29 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants