Repository navigation
FFmpeg decoder: send complete access units without the parser - #1921
Conversation
libheif pushes one complete access unit per push_data() call, so running av_parser_parse2() over it only parsed every NAL unit a second time before the decoder did. This also removes the partial-consumption path, which resized input_data by a byte count.
…he input (GHSA-hqpw-h8p6-22q9) FFmpeg's VVC decoder copies the colour description of the VUI into the codec context only since FFmpeg 8.0 (libavcodec 62). In FFmpeg 7.x, this is done by the VVC parser. Since the parser is not used anymore (#1921), VVC images were reported with an unspecified colour profile and limited range: the colours were wrong for files without 'colr' box, and files with a 'colr' box gave the warning "colr box and bitstream colour signalling disagree". Run the parser again for VVC when built against libavcodec < 62, only to let it fill the codec context. The access unit is still sent to the decoder as one packet. All other decoders export the colour signalling themselves. Also append AV_INPUT_BUFFER_PADDING_SIZE zero bytes to each input packet, as FFmpeg requires for its input buffers. Without the padding, the parser read up to 6 bytes past the end of the buffer (GHSA-hqpw-h8p6-22q9).
|
Thanks, merged. I added one follow-up commit (eda8f2d). FFmpeg's native VVC decoder exports the colour description of the VUI only since FFmpeg 8.0. In FFmpeg 7.x, the VVC parser is the only code that fills these fields, so without the parser all VVC images were reported with an unspecified colour profile. The parser is now kept for VVC when building against libavcodec < 62, only for this side effect. The access unit still goes to the decoder as one packet, as in your change. The input packets now also get the I compared the output before and after on about 470 files with FFmpeg 6.1.1, 7.1.1 and 8.0: identical for all codecs. On Linux I see a smaller gain than you measured: about 5 % with FFmpeg 6.1.1 for a 48-tile iPhone photo and none with 7.1.1, where the HEVC parser became much cheaper. |
Found while benchmarking Telegram Desktop's migration from libheif + libde265 to libheif + FFmpeg (see also #1920).
libheif pushes one complete access unit per
push_data()call, so runningav_parser_parse2()over it only parses every NAL unit a second time before the decoder does. On phone photos (grids of 512×512 tiles) that was a large share of the plugin's time: on a 48-tile iPhone HEIC, 1.28 ms per tile went to the parser against 2.66 ms of actual decoding, plus about 0.2 ms per tile closing the parser.This sends each access unit to the decoder as one packet and drops the parser. It also removes the partial-consumption path, which resized
input_data(the packet queue) by a byte count.x64 Release, FFmpeg 6.1.6, 48-tile 4032×3024 iPhone photo: 235 → 166 ms single-threaded, 105 → 79 ms with libheif's default tile threading. Decoded output is identical on the conformance files, KDE's HEIC autotests, phone photos and libheif's fuzzing corpus.
🤖 Generated with Claude Code