Skip to content

ZSTD_compressSequencesAndLiterals() does not enforce its documented litCapacity >= litSize + 8 contract — the implemented check is 8 bytes weaker than the API documentation, its own error message, and the introducing commit #4779

Open

Description

@bbingoCN

Describe the bug

ZSTD_compressSequencesAndLiterals() does not enforce its own documented
litBufCapacity >= litSize + 8 contract: the implemented check in
lib/compress/zstd_compress.c:8128 only tests litCapacity < litSize,
i.e. it is 8 bytes weaker than the contract stated in three independent
places:

  1. API documentation — lib/zstd.h line 1700:

    @litBufCapacity must be at least 8 bytes larger than @litSize.

  2. The introducing commit — b7a9e69d ("added parameter litCapacity
    to ZSTD_compressSequencesAndLiterals() to enforce the litCapacity >=
    litSize+8 condition
    .")
  3. The check's own error message —

    "literals buffer is not large enough: must be at least 8 bytes larger
    than litSize
    (risk of read out-of-bound)"

/* lib/compress/zstd_compress.c:8126-8130 */
if (litCapacity < litSize) {   /* <-- should be: litCapacity < litSize + 8 */
    RETURN_ERROR(workSpace_tooSmall,
        "literals buffer is not large enough: "
        "must be at least 8 bytes larger than litSize (risk of read out-of-bound)");
}

Calls that violate the documented contract by up to 8 bytes
(litSize <= litCapacity < litSize + 8) are silently accepted and produce
valid frames, instead of being rejected with the documented
ZSTD_error_workSpace_tooSmall.

To Reproduce

Steps to reproduce the behavior:

  1. Clone zstd and build the static library:
git clone https://github.com/facebook/zstd && cd zstd
make -C lib libzstd.a    # or: make -C lib
  1. Save the following as reproducer.c in the repo root:
/* reproducer.c — demonstrates that ZSTD_compressSequencesAndLiterals()
 * accepts litCapacity values that violate its documented
 * "at least 8 bytes larger than litSize" contract. */
#define ZSTD_STATIC_LINKING_ONLY
#include "zstd.h"
#include <stdio.h>

int main(void)
{
    /* one explicit-delimiter block: 284 literals + a 16-byte match at
     * offset 8, then 16 trailing literals carried by the delimiter */
    ZSTD_Sequence seqs[2] = {
        { /*offset=*/8, /*litLength=*/284, /*matchLength=*/16, /*rep=*/0 },
        { /*offset=*/0, /*litLength=*/ 16, /*matchLength=*/ 0, /*rep=*/0 },
    };
    unsigned char literals[300 + 8];        /* +8 so the probe itself is memory-safe */
    unsigned char dst[1024];
    const size_t litSize = 300, srcSize = 316;
    size_t i, r;
    ZSTD_CCtx* cctx = ZSTD_createCCtx();

    for (i = 0; i < litSize; i++) literals[i] = (unsigned char)('a' + (i % 26));

    ZSTD_CCtx_setParameter(cctx, ZSTD_c_compressionLevel, 3);
    ZSTD_CCtx_setParameter(cctx, ZSTD_c_blockDelimiters, ZSTD_sf_explicitBlockDelimiters);

    /* documented minimum is litSize + 8; pass litSize + 7 on purpose */
    r = ZSTD_compressSequencesAndLiterals(cctx, dst, sizeof(dst),
                                          seqs, 2, literals, litSize,
                                          litSize + 7, srcSize);
    if (ZSTD_isError(r))
        printf("litCapacity = litSize+7 : rejected (%s) -> contract enforced\n",
               ZSTD_getErrorName(r));
    else
        printf("litCapacity = litSize+7 : ACCEPTED (%zu-byte frame) -> "
               "documented contract NOT enforced\n", r);

    ZSTD_freeCCtx(cctx);
    return 0;
}
  1. Compile and run:
gcc -O2 -Ilib reproducer.c lib/libzstd.a -o reproducer -lpthread
./reproducer
  1. Scroll up on the log to the reproducer's output line:
litCapacity = litSize+7 : ACCEPTED (223-byte frame) -> documented contract NOT enforced
  1. See error: there is no error — the contract-violating call is
    accepted and produces a valid, round-tripping 223-byte frame. The same
    happens with litCapacity = litSize (violating by the full 8 bytes).
    With the one-line fix below, both calls are rejected with
    ZSTD_error_workSpace_tooSmall while a compliant
    litCapacity = litSize + 8 call keeps working.

Expected behavior

ZSTD_compressSequencesAndLiterals() should reject litCapacity values
below litSize + 8 with ZSTD_error_workSpace_tooSmall, exactly as its
documentation (lib/zstd.h:1700), its introducing commit (b7a9e69d) and
its own error message all promise:

--- a/lib/compress/zstd_compress.c
+++ b/lib/compress/zstd_compress.c
@@ -8125,7 +8125,7 @@ ZSTD_compressSequencesAndLiterals(ZSTD_CCtx* cctx,
     /* Transparent initialization stage, same as compressStream2() */
     DEBUGLOG(4, "ZSTD_compressSequencesAndLiterals (dstCapacity=%zu)", dstCapacity);
     assert(cctx != NULL);
-    if (litCapacity < litSize) {
+    if (litCapacity < litSize + 8) {
         RETURN_ERROR(workSpace_tooSmall, "literals buffer is not large enough: must be at least 8 bytes larger than litSize (risk of read out-of-bound)");
     }
     FORWARD_IF_ERROR(ZSTD_CCtx_init_compressStream2(cctx, ZSTD_e_end, decompressedSize), "CCtx initialization failed");

Screenshots and charts

Not applicable — the reproducer output above shows the behavior
(ACCEPTED ... contract NOT enforced on current dev; after the one-line
fix the same call prints rejected (workSpace buffer is not large enough) -> contract enforced).

Desktop (please complete the following information):

  • OS: Linux (x86-64)
  • Version: Ubuntu 20.04
  • Compiler: gcc 9.4.0
  • Flags: -O2 for the reproducer; the fuller PoC (see Additional context) was additionally run with -g -O1 -fsanitize=address,undefined -fno-sanitize-recover=all
  • Other relevant hardware specs: n/a (no hardware dependency)
  • Build system: make -C lib (static library lib/libzstd.a); library sources compiled directly in the PoC verification as well

Additional context

  • Affected version: current dev (verified byte-identical at
    lib/compress/zstd_compress.c:8128 on dev @ d9c0c7e2, 2026-09-09).
    I could not find an existing issue or PR covering this mismatch (closest:
    open PR Reject a too large block in ZSTD_compressSequencesAndLiterals() #4741 fixes a different validation gap in the same function).
  • Full verification PoC (attached): a three-phase test that (1) confirms a
    compliant litCapacity = litSize + 8 call succeeds, (2) demonstrates both
    litSize + 7 and litSize violations are accepted on the unpatched
    build and rejected with ZSTD_error_workSpace_tooSmall after the
    one-line fix, and (3) runs an ASan sweep of 360 configurations
    (30 literal sizes x 3 data patterns x 4 levels, literals buffer of
    exactly litSize bytes) probing for reads inside the documented-but-
    unenforced margin [litSize, litSize+8).
  • Honest impact statement: the sweep produced zero ASan reports —
    literal reads are currently bounded by litSize in the existing encoder
    paths, so this is a contract-enforcement / defense-in-depth gap rather
    than a demonstrated out-of-bounds read. The +8 slack exists because the
    literal encoders historically used 8-byte-wide loads near the tail; the
    check is the only guard between the caller and
    ZSTD_entropyCompressSeqStore_internal(), and today it enforces none of
    the slack that the documentation, the error message and commit
    b7a9e69d all promise.
  • Suggested title for the issue: "ZSTD_compressSequencesAndLiterals() does
    not enforce the documented litBufCapacity >= litSize + 8 contract (check
    is missing the + 8)".

Activity

  1. KSAGlory commented on Sep 11, 2026

    @KSAGlory

    Hi, I'd like to work on this. I'll enforce the documented 8-byte literal-buffer margin with an overflow-safe capacity check and add regression coverage for the boundary conditions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions