fix(search): gate verify-on-read file results on vector-index tag membership

Verify-on-read only checked file *accessibility* (file_accessible_by_id),
never tag membership, so a file removed from the `vector-index` tag (but
still readable) kept surfacing in semantic search, and stale points only
got evicted when they happened to rank in a search's top-K.

Rework `_verify_files` to gate on current `vector-index` tag membership via
a single batch `find_files_by_tag(tag, mime_type_filter="application/pdf")`
REPORT per search (plus a one-shot EXCLUDED_TAGS lookup for exclusion-wins
parity) — exactly what the scanner indexes. A file is kept iff it is in that
set, so untagged / deleted / excluded files drop out immediately and the
existing eviction wiring reclaims their Qdrant points. The gate is strict
for all file results, own and shared. Mirrors the batch-fetch-and-intersect
shape of `_verify_news_items` (one semaphore slot, fail-open on fetch error,
malformed-id keep).

- Promote the tag name to a `vector_sync_pdf_tag` Settings field (dynaconf
  env mapping VECTOR_SYNC_PDF_TAG) used by both scanner and verifier;
  drop the scanner's direct os.getenv.
- Expose `find_files_by_tag` on NextcloudClientProtocol.
- Rewrite the file-verifier unit tests (tagged/untagged/deleted/excluded/
  fail-open/non-numeric); update the ACL + verify-on-read integration tests
  to seed tagged PDFs.
- Amend ADR-019 and the configuration.md verify-on-read latency budget.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Chris Coutinho
2026-06-02 21:14:44 +02:00
co-authored by Claude Opus 4.8
parent 7e4b83dc94
commit d4dbf01b0a
9 changed files with 358 additions and 220 deletions
@@ -75,6 +75,14 @@ class NextcloudClientProtocol(Protocol):
"""News client for accessing news item documents."""
...
# Top-level client helper (not a sub-client) used by verify-on-read to
# gate file results on current vector-index tag membership.
async def find_files_by_tag(
self, tag_name: str, mime_type_filter: str | None = None
) -> list[dict]:
"""Return files carrying ``tag_name`` (folders expanded by MIME)."""
...
async def get_indexed_doc_types(
user_id: str, accessible_owners: list[str] | None = None