Skip to content

Fix pickle deserialization vulnerability in RAG cache #447

Closed

Description

@kovtcharov

Problem

src/gaia/rag/sdk.py:1620 uses pickle.load() on cached data without any validation:

cached_data = pickle.load(f)

A corrupted or tampered cache file can cause crashes or arbitrary code execution. There is no size limit check, no format validation, and no hash verification before deserialization.

Impact

  • Corrupted .pkl files crash the RAG system permanently
  • No integrity verification of cached data
  • Potential code execution via crafted pickle files (local attack surface)

Proposed Fix

  1. Add format version header (GAIA_CACHE_V1) to cache files
  2. Store and verify SHA-256 checksum before unpickling
  3. Validate file size before loading (reject unreasonably large caches)
  4. Wrap pickle.load() in robust try/except with specific exception types (EOFError, pickle.UnpicklingError, KeyError)
  5. On any load failure: delete corrupted cache file, log warning, re-index
  6. Long-term: plan migration to JSON metadata + numpy embeddings

Files

  • src/gaia/rag/sdk.py (lines 1615-1630)

Acceptance Criteria

  • Corrupted cache files are detected and deleted automatically
  • Cache load failures trigger clean re-index, not crash
  • Unit test for corrupted cache recovery

Activity

  1. added this to the v0.17.0 milestone on Mar 8, 2026
  2. added
    bugSomething isn't working
    p0high priority
    ragRAG system changes
    securitySecurity-sensitive changes
    robustnessReliability, error handling, and hardening
    on Mar 8, 2026
  3. albiol2004 commented on Apr 8, 2026

    @albiol2004
    Contributor

    Hi, I'd like to work on this. I have a fix ready that adds a format version header, SHA-256 checksum verification, file size limits, and auto-recovery with cache file deletion. I'll open a PR shortly

  4. added a commit that references this issue on Apr 10, 2026
    cd81adc
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingp0high priorityragRAG system changesrobustnessReliability, error handling, and hardeningsecuritySecurity-sensitive changes

Type

No type

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions