Documents index with 0 chunks — RAG pipeline silently broken, agent reports system error #590

Problem

Documents are accepted by the upload endpoint and registered in the Document Library, but are never actually processed into chunks. The result is chunk_count: 0 on every indexed document, RAG returns no results, and the agent surfaces a system error when attempting to use the document.

Steps to Reproduce

1. Open the Document Library (upload button in chat)
2. Drop or browse to any supported file (tested: 10mb.pdf, Executives-guide-to-AI.pdf, .md files on macOS)
3. Click Index
4. Re-open the Document Library

Expected Behaviour

• Document appears with chunk_count > 0
• Asking the agent about the document returns relevant content from it

Actual Behaviour

• Document Library shows: <filename> · 0 chunks · just now
• Asking the agent about the document results in: "the document indexing tool isn't working properly on my end right now (system error)"
• Agent cannot retrieve any content from the document

Evidence

Document Library after indexing two files:

Executives-guide-to-AI.pdf   351.4 KB · 0 chunks · just now
10mb.pdf                      10 MB   · 0 chunks · 2m ago

API confirmation (GET /api/documents/<id>/status):

{ "indexing_status": "complete", "chunk_count": 0, "is_active": false }

Note: indexing_status reports "complete" even though no chunks were produced and is_active is false. The status field is misleading — this should surface as a failure, not a success.

Suspected Cause

Likely a regression in the fd-based file processing path introduced in PR #564. The file is opened, hashed, and the database record is written — but the text extraction and chunking step does not appear to be receiving the file content from the open file descriptor. The document record is created but the pipeline stops before producing any searchable chunks.

Impact

• RAG is completely non-functional — the core document Q&A feature does not work
• Affects all file types and all platforms tested (Windows, macOS)
• The agent actively reports a system error to the user when trying to use indexed files
• indexing_status: "complete" with chunk_count: 0 is a misleading false positive that hides the failure