Repository navigation
fix(vector_stores/turbopuffer): apply all filter operators instead of dropping them - #6564
Conversation
âĶ dropping them
_convert_filters only handled gte and lte for operator dicts, so filters
like {"age": {"gt": 18}} matched neither branch and were silently
dropped, returning unfiltered results. Map the full mem0 operator set
(eq, ne, gt, gte, lt, lte, in, nin) to the Turbopuffer filter tokens and
raise ValueError on an unknown operator instead of dropping it.
|
Open ~19 days, green CI, unreviewed ð Turbopuffer |
blazingretinol
left a comment
There was a problem hiding this comment.
Changes requested
Correctly fixes the reported operator-drop bug (now merged), but the same _convert_filters still mistranslates "*" wildcard and bare-array filters into literal Eq matches that return nothing.
Worth a fast-follow:
mem0/vector_stores/turbopuffer.py:167- guardvalue == "*"(match-any) and bare-list-as-in, same as every other Python backend and the merged TS sibling #6578
Optional: contains/icontains now raise instead of silently dropping (line 135); also, open PR #6878 edits this same method and will need a small rebase against this merge.
How I verified
- Problem is real:
mem0/vector_stores/turbopuffer.pypre-PR (lines ~146-149) dropped any operator besidesgte/lte;db._convert_filters({"age": {"gt": 18}})returnedNone, matching issue #6562 - Tests / red-green:
PYTHONPATH=. pytest tests/vector_stores/test_turbopuffer.py -v: 75 passed on head; revertingturbopuffer.pyto pre-PRmainfailed the 9 new operator tests, 10 pre-existing passed; restoring returned to 75 passed - CI: real checks (tests, lint, build, license/cla) green; Vercel preview-deploy red, not merge-blocking
- Security: cleared, filter values pass through as opaque tuples to the typed SDK client, no injection surface
- Bloat (ponytail): lean already, table-driven operator map is the right shape
- Not verified: live Turbopuffer API (hosted-only, no sandbox lane); unit-level mocked-client tests only
Solid, well-tested fix for the issue it names. Since this already merged, a short follow-up PR closing the wildcard/array gap in the same function would finish the job.
Linked Issue
Closes #6562
Description
_convert_filtersin the Turbopuffer vector store only handledgteandltefor operator dicts. Any other operator mem0 supports (gt,lt,ne,eq,in,nin) matched neither branch, so a filter like{"age": {"gt": 18}}was silently dropped and the query ran unfiltered, returning too-broad results with no error.This maps the full mem0 operator set to the Turbopuffer filter tokens (verified against the SDKs
Filtertype inturbopuffer/types/custom.py:eq->Eq, ne->NotEq, gt->Gt, gte->Gte, lt->Lt, lte->Lte, in->In, nin->NotIn) and raises a clearValueErroron an unknown operator instead of dropping it. Existing scalar-equality andgte/ltebehavior is unchanged.Type of Change
Breaking Changes
N/A
Test Coverage
Added regression tests in
tests/vector_stores/test_turbopuffer.py: every operator maps to the right token,{"gt": 18}is no longer dropped, multiple operators on one field combine withAnd, and an unknown operator raisesValueError. Verified the new tests fail on the unpatched source.Checklist