heif-testcorpus

A HEIF/AVIF test corpus for testing HEIF implementations such as libheif, assembled by a reproducible build script from several image sources.

A source is either a set of third-party files that is downloaded at build time, or images created for this corpus, which are checked in as binary files or generated by a script. Third-party files are not redistributed in this repository: such a source pins its origin (repository, commit, sha256 of every file), and the build script downloads the files, verifies them, and applies scripted corrections to files that are known to be invalid. Each correction carries the hash of the file it expects and the hash of the file it produces, so the generated corpus is bit-identical on every machine.

The third-party sources are the MPEG HEIF conformance file candidates published by Nokia and the HEIF files that the MPEG File Format Conformance repository adds beyond those. Our own images cover coding formats, bit depths and conformance windows that the common encoders do not produce.

corpus/<name>/         the corpus, one folder per part:
                       - committed folders hold our own images, with a README.md
                         for provenance and licence
                       - produced folders are written by sources/<name>/build.py,
                         contain a SHA256SUMS file, and ignore themselves in git
                         through a generated .gitignore
build-corpus.py        builds every produced folder (run this)
sources/<name>/        recipe for one produced folder
    build.py           downloads, verifies and corrects third-party files, or
                       generates images, and writes them to corpus/<name>/
    README.md          provenance, licence, known upstream defects
    manifest.txt       third-party sources: pinned upstream commit and sha256 per file
    patches/*.py       third-party sources: one module per correction
corpuslib/             shared helpers: box editing, HEVC NAL inspection, sample tables,
                       reusable corrections, verified download, patch loading, the
                       manifest-driven build step shared by third-party sources, the
                       build step shared by generating sources, and writers for
                       generated images (test picture, AVC intra pictures, JPEG,
                       HEVC configuration record, HEIF files)
.cache/<name>/         unmodified upstream downloads (git-ignored)

Sources currently included:

Committed image folders:

Requirements: Python 3.8 or newer, no third-party packages. Network access is needed on the first run only; the downloads are cached in .cache/.

python3 build-corpus.py

This writes every produced folder to corpus/<source>/. The committed folders with our own images are complete right after cloning and are not touched by the build. Useful options:

python3 build-corpus.py --out /some/dir        # different output directory
python3 build-corpus.py --source NAME          # build one source only
python3 build-corpus.py --offline              # never download, use the cache
python3 build-corpus.py --list                 # list the available sources
python3 build-corpus.py --verbose              # show downloads and patches

Every source's build.py can also be run on its own, see --help.

1. Report the defect upstream first and reference the report in the module.
2. Add sources/<name>/patches/<file>_<topic>.py defining FILE, DESCRIPTION, UPSTREAM_ISSUE, INPUT_SHA256, OUTPUT_SHA256 and apply(data) -> bytes. Set OUTPUT_SHA256 = None, run the build once, verify the produced file with an independent decoder, then pin the printed hash.
3. Add the correction to the source's README.

corpuslib/isobmff.py provides the box editing needed for container-level fixes: appending a box to a container and updating all enclosing sizes and the iloc, stco and co64 file offsets behind the edit. Defects in the coded bitstream itself cannot be corrected this way; list those in the source's README as known defects instead.

Create corpus/<name>/, put the image files in it and add a README.md stating where the images come from, what they test and under which licence they are published. Commit the folder. There is no build step, and the name must not collide with a folder under sources/.

Create sources/<name>/ with a build.py that accepts --out, --cache, --offline and --verbose, and writes the files of the source plus a SHA256SUMS file to --out. build-corpus.py picks it up automatically. build.py must call corpuslib.output.prepare_output_dir() on --out: it writes the self-ignoring .gitignore marker and refuses to overwrite a folder that was not produced by an earlier build, which protects committed images.

• Third-party files: list the pinned origin's files with their sha256 in manifest.txt, put corrections into patches/, and let build.py call corpuslib.manifestsource.run() with the upstream URL scheme; it downloads, verifies, corrects and writes the files. sources/nokiatech-heif-conformance/ and sources/mpeggroup-fileformatconformance/ are the templates.
• Generated images: let build.py call corpuslib.generatedsource.run() with a function that returns the generated files and with the pinned sha256 of every file. The generator must be deterministic so that the corpus stays bit-identical; a file that differs from its pinned hash fails the build. sources/avc-high-bitdepth/ and sources/jpeg-rgb-and-high-bitdepth/ are the templates.

Either way the images end up under corpus/<name>/, which is the only place consumers need to look.

The scripts in this repository are released under the MIT licence, see LICENSE. Images created for this corpus carry the licence stated in the README of their folder under corpus/. Third-party image files remain the property of their respective publishers and are subject to their terms; they are downloaded for local testing and are not redistributed here.