fix(search): gate verify-on-read file results on vector-index tag membership

Verify-on-read only checked file *accessibility* (file_accessible_by_id),
never tag membership, so a file removed from the `vector-index` tag (but
still readable) kept surfacing in semantic search, and stale points only
got evicted when they happened to rank in a search's top-K.

Rework `_verify_files` to gate on current `vector-index` tag membership via
a single batch `find_files_by_tag(tag, mime_type_filter="application/pdf")`
REPORT per search (plus a one-shot EXCLUDED_TAGS lookup for exclusion-wins
parity) — exactly what the scanner indexes. A file is kept iff it is in that
set, so untagged / deleted / excluded files drop out immediately and the
existing eviction wiring reclaims their Qdrant points. The gate is strict
for all file results, own and shared. Mirrors the batch-fetch-and-intersect
shape of `_verify_news_items` (one semaphore slot, fail-open on fetch error,
malformed-id keep).

- Promote the tag name to a `vector_sync_pdf_tag` Settings field (dynaconf
  env mapping VECTOR_SYNC_PDF_TAG) used by both scanner and verifier;
  drop the scanner's direct os.getenv.
- Expose `find_files_by_tag` on NextcloudClientProtocol.
- Rewrite the file-verifier unit tests (tagged/untagged/deleted/excluded/
  fail-open/non-numeric); update the ACL + verify-on-read integration tests
  to seed tagged PDFs.
- Amend ADR-019 and the configuration.md verify-on-read latency budget.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Chris Coutinho
2026-06-02 21:14:44 +02:00
co-authored by Claude Opus 4.8
parent 7e4b83dc94
commit d4dbf01b0a
9 changed files with 358 additions and 220 deletions
+1 -2
View File
@@ -4,7 +4,6 @@ Periodically scans enabled users' content and queues changed documents for proce
"""
import logging
import os
import random
import time
from dataclasses import dataclass
@@ -396,7 +395,7 @@ async def scan_user_documents(
# PDF descendants (Depth: infinity SEARCH), so a tag on a
# folder applies to every PDF beneath it.
settings = get_settings()
tag_name = os.getenv("VECTOR_SYNC_PDF_TAG", "vector-index")
tag_name = settings.vector_sync_pdf_tag
tagged_files = await nc_client.find_files_by_tag(
tag_name, mime_type_filter="application/pdf"
)