[john-preston]

This came up while migrating Telegram Desktop from libheif + libde265 to libheif + FFmpeg. Both commits add pixel formats that FFmpeg returns and the plugin rejected as unsupported, while libde265 decodes the same files.

Planar GBR. FFmpeg's HEVC decoder returns AV_PIX_FMT_GBRP/GBRP10/GBRP12 instead of YUV 4:4:4 when a 4:4:4 stream signals matrix_coefficients = 0 (see libavcodec/hevc_ps.c). The G, B, R planes map directly onto Y, Cb, Cr, which is how libheif represents the identity matrix, and the plugin already takes the nclx profile from the stream, so it only needs to accept these formats.

9-bit. FFmpeg returns GRAY9 and YUV420P9/422P9/444P9 for 9-bit streams (and GBRP9 for 9-bit AVC with the identity matrix).

Tested with FFmpeg 6.1.6: 13 files from KDE kimageformats' autotests/read/heic (rgb, rgba, crop-*) that failed before, and x265-encoded 8/10/12-bit 4:4:4 files with matrix 0, now decode pixel-identically to libde265's output. The 9-bit mappings follow the same pattern; I had no 9-bit sample to test with.

🤖 Generated with Claude Code

[farindk]

Thanks for the PR. I merged it unchanged.

Since you had no 9 bit sample, I generated test files and checked the new
formats against reference data:

• AVC with 9 bits in 4:2:0, 4:2:2, 4:4:4 and GBR decodes bit-exact.
• GBR from AVC works at 8, 9, 10 and 12 bits.
• HEVC with the identity matrix at 8, 10 and 12 bits gives the same output as
  libde265, as you found.
• HEVC with 9 bits works too, including monochrome (GRAY9).
• AV1 with the identity matrix arrives as planar GBR as well and matches dav1d.

I added a few commits on top:

• 52ce85a adds GBRP14. AVC codes up to 14 bits in the High 4:4:4 Predictive
  profile.
• 3873d81 fixes a side effect for JPEG. The new GBRP case applies to every
  codec, and FFmpeg's JPEG decoder returns GBRP for a JPEG that was coded
  without colour transform. JPEG has no colour signalling that the plugin
  could pass on, so such an image was decoded with wrong colours when the file
  had no colr box. The plugin now attaches the identity matrix in this case.
• cce01a5 adds YUVJ422P and YUVJ444P. This gap existed before your PR:
  full range AVC in 4:2:2 and 4:4:4, and every JPEG with these chroma formats,
  were rejected.
• 21dc3ea documents the bit depths in the README and in the plugin.

Two things that may matter for your move from libde265 to FFmpeg:

• FFmpeg's HEVC decoder handles 8, 9, 10 and 12 bits only. Streams with 11 bits
  or with more than 12 bits fail in avcodec_send_packet(). libde265 decodes
  all of them.
• FFmpeg maps the identity matrix to planar GBR for HEVC at 8, 10 and 12 bits.
  At 9 bits it returns yuv444p9 with colour space gbr, so GBRP9 comes from
  AVC only. The planes are the same either way.

I left out GBRP16 on purpose. FFmpeg returns it for JPEG with 12 bits coded
as RGB, but with the planes in the order R, G, B. It reorders them for the
8 bit format only.

The test files are in https://github.com/farindk/heif-testcorpus:

• corpus/hevc-high-bitdepth:
  HEVC with 9, 11, 15 and 16 bits, encoded with HM.
• sources/avc-high-bitdepth:
  AVC with 9 and 14 bits.
• sources/jpeg-rgb-and-high-bitdepth:
  JPEG coded as RGB, with 12 bits, and lossless with 16 bits.

The HEVC images are in the repository. The AVC and JPEG images are generated
by python3 build-corpus.py. Every image comes with a raw file of the expected
decoded planes, and the README of each folder lists what FFmpeg returns for it.