Skip to content

fix(vector_stores/turbopuffer): apply all filter operators instead of dropping them - #6564

Merged
blazingretinol merged 2 commits into
mem0ai:mainfrom
yashs33244:fix/turbopuffer-filters
Sep 25, 2026
Merged

blazingretinol merged 2 commits into
mem0ai:mainfrom
yashs33244:fix/turbopuffer-filters

Conversation

@yashs33244

Copy link
Copy Markdown
Contributor

Linked Issue

Closes #6562

Description

_convert_filters in the Turbopuffer vector store only handled gte and lte for operator dicts. Any other operator mem0 supports (gt, lt, ne, eq, in, nin) matched neither branch, so a filter like {"age": {"gt": 18}} was silently dropped and the query ran unfiltered, returning too-broad results with no error.

This maps the full mem0 operator set to the Turbopuffer filter tokens (verified against the SDKs Filter type in turbopuffer/types/custom.py: eq->Eq, ne->NotEq, gt->Gt, gte->Gte, lt->Lt, lte->Lte, in->In, nin->NotIn) and raises a clear ValueError on an unknown operator instead of dropping it. Existing scalar-equality and gte/lte behavior is unchanged.

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Refactor (no functional changes)
  • Documentation update

Breaking Changes

N/A

Test Coverage

  • I added/updated unit tests
  • I added/updated integration tests
  • I tested manually (describe below)
  • No tests needed (explain why)

Added regression tests in tests/vector_stores/test_turbopuffer.py: every operator maps to the right token, {"gt": 18} is no longer dropped, multiple operators on one field combine with And, and an unknown operator raises ValueError. Verified the new tests fail on the unpatched source.

Checklist

  • My code follows the projects style guidelines
  • I have performed a self-review of my code
  • I have added tests that prove my fix works
  • New and existing tests pass locally
  • I have updated documentation if needed

â€Ķ dropping them

_convert_filters only handled gte and lte for operator dicts, so filters
like {"age": {"gt": 18}} matched neither branch and were silently
dropped, returning unfiltered results. Map the full mem0 operator set
(eq, ne, gt, gte, lt, lte, in, nin) to the Turbopuffer filter tokens and
raise ValueError on an unknown operator instead of dropping it.
@github-actions github-actions Bot added sdk-python Python SDK specific vector-store Vector store backends (Qdrant, PGVector, Redis, etc.) labels Jul 24, 2026
@yashs33244
yashs33244 marked this pull request as ready for review July 24, 2026 15:55
Copilot AI review requested due to automatic review settings July 24, 2026 15:55

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@yashs33244

Copy link
Copy Markdown
Contributor Author

Open ~19 days, green CI, unreviewed 🙏 Turbopuffer _convert_filters now applies all operators (gt/lt/ne/in/nin) instead of silently dropping them; unknown ops raise. 4 tests. @kartik-mem0 @deshraj would appreciate a review.

@blazingretinol
blazingretinol merged commit ccd216c into mem0ai:main Sep 25, 2026
48 of 49 checks passed

@blazingretinol blazingretinol left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested

Correctly fixes the reported operator-drop bug (now merged), but the same _convert_filters still mistranslates "*" wildcard and bare-array filters into literal Eq matches that return nothing.

Worth a fast-follow:

  1. mem0/vector_stores/turbopuffer.py:167 - guard value == "*" (match-any) and bare-list-as-in, same as every other Python backend and the merged TS sibling #6578

Optional: contains/icontains now raise instead of silently dropping (line 135); also, open PR #6878 edits this same method and will need a small rebase against this merge.

How I verified
  • Problem is real: mem0/vector_stores/turbopuffer.py pre-PR (lines ~146-149) dropped any operator besides gte/lte; db._convert_filters({"age": {"gt": 18}}) returned None, matching issue #6562
  • Tests / red-green: PYTHONPATH=. pytest tests/vector_stores/test_turbopuffer.py -v: 75 passed on head; reverting turbopuffer.py to pre-PR main failed the 9 new operator tests, 10 pre-existing passed; restoring returned to 75 passed
  • CI: real checks (tests, lint, build, license/cla) green; Vercel preview-deploy red, not merge-blocking
  • Security: cleared, filter values pass through as opaque tuples to the typed SDK client, no injection surface
  • Bloat (ponytail): lean already, table-driven operator map is the right shape
  • Not verified: live Turbopuffer API (hosted-only, no sandbox lane); unit-level mocked-client tests only

Solid, well-tested fix for the issue it names. Since this already merged, a short follow-up PR closing the wildcard/array gap in the same function would finish the job.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

sdk-python Python SDK specific vector-store Vector store backends (Qdrant, PGVector, Redis, etc.)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Turbopuffer vector store silently drops filter operators other than gte/lte

3 participants