fix(search): address PR #834 re-review (403/404 coverage + docs)

Resolve the blocking + important findings from the claude bot's re-review:

- test (blocking): pin the file verifier's fail-open contract for definitive
  403/404 on the tag REPORT, not just transient 503/429. A disabled systemtags
  endpoint commonly 403s; unlike the per-access verifiers (where 403/404 = drop),
  the batch file verifier must keep all results since the whole set hinges on one
  REPORT. Adds _http_error(403)/_http_error(404) to
  test_verify_files_tag_fetch_failure_keeps_all and documents the asymmetry.
- docs (important): migration caveat — if vector-index was created as
  user_visible=False (manual occ tag:add, or pre-release), an owner's tag won't
  surface in a recipient's REPORT and shared-file results are silently dropped
  after upgrade. Note that the MCP server's get_or_create_tag defaults to
  user_visible=True, and how to verify/fix an existing tag.
- docs (important): note the file verifier's latency scales with both the
  Depth:infinity folder expansion and the EXCLUDED_TAGS lookup (~2 WebDAV calls
  per excluded tag, fanned out under one slot); suggest lowering
  VERIFICATION_CONCURRENCY for large excluded-tag lists / deeply tagged trees.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Chris Coutinho
2026-06-02 23:52:16 +02:00
co-authored by Claude Opus 4.8
parent 8cae7d1708
commit 7033c64393
2 changed files with 38 additions and 7 deletions
+21 -4
View File
@@ -675,10 +675,27 @@ aware of:
an excluded folder) drops out of results immediately rather than waiting for
the scanner sweep. The REPORT expands tagged folders via a `Depth: infinity`
SEARCH, so deployments that tag whole directory trees pay that walk once per
search; configure `VECTOR_SYNC_PDF_TAG` to change the tag name. **Shared
files**: a file an owner tagged and shared with the searcher only survives
verification if the owner's (userVisible) tag surfaces in the *searcher's*
tag REPORT.
search; configure `VECTOR_SYNC_PDF_TAG` to change the tag name. The `file`
verifier's latency therefore scales with **both** the `Depth: infinity` folder
expansion **and** the `EXCLUDED_TAGS` lookup: that lookup fans out ~2 WebDAV
calls (1 PROPFIND + 1 REPORT) *per excluded tag*, concurrently, while holding
a single verification slot — so a deployment with a long `EXCLUDED_TAGS` list
and/or deeply tagged trees issues many parallel Nextcloud requests per search.
Operators in that situation may want to **lower `VERIFICATION_CONCURRENCY`** so
the file verifier's internal fan-out does not overwhelm the backend.
- **Shared files**: a file an owner tagged and shared with the searcher only
survives verification if the owner's **`userVisible`** tag surfaces in the
*searcher's* tag REPORT. The MCP server's own tag-creation path
(`WebDAVClient.get_or_create_tag`) defaults to `user_visible=True`, so tags it
creates are fine. **Migration caveat**: if the `vector-index` tag was created
some other way — manually via `occ tag:add … --user-visible=false`, or in a
deployment predating this release — it may be `user_visible=False` (the
Nextcloud default for system-managed tags). In that case an owner's tag will
**not** surface in a recipient's systemtag REPORT, so every shared-file result
is *silently dropped* for recipients after upgrading — no error, just a
narrower result set. Verify the tag's visibility (Administration → *Collaborative
tags*, or `occ tag:list`) and, if it is not user-visible, recreate it as
user-visible so shared search keeps working.
- **Eviction**: when verification finds a definitive miss (a 404 / 403, or — for
files — absence from the tag set), the corresponding Qdrant points are deleted
in the background on a lifespan-owned task group — fire-and-forget, does