feat(search): verify-on-read for semantic search results (ADR-019)

The vector index lags Nextcloud (5-min webhook cron + scanner interval),
producing ghost records for deleted/unshared documents until the next
reconciliation. Verify each unique document against Nextcloud at query
time, drop inaccessible results, and lazily evict the corresponding
Qdrant points.

Per-doc_type batch verifiers: notes/files/deck cards run concurrently
per id; news items use a single fetch + intersect to avoid the per-item
fetch-all amplification. Transient errors fail open (keep result, log
warning) — only definitive 4xx drops. Multiple chunks of the same doc
collapse to one verification call.

Wired into nc_semantic_search before the limit trim and before context
expansion. nc_semantic_search_answer's per-note re-fetch retained as a
sub-second race guard since verification now happens upstream.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Chris Coutinho
2026-05-01 07:45:01 +02:00
co-authored by Claude Opus 4.7
parent 0c2d3e1086
commit d90e793d19
8 changed files with 1379 additions and 24 deletions
+61
View File
@@ -0,0 +1,61 @@
"""Lazy eviction of stale documents from the vector index.
Used by the verify-on-read path (ADR-019) to remove points for documents that
have been deleted or unshared in Nextcloud but not yet reconciled by the
webhook/scanner sync loop. Eviction is fire-and-forget from the search hot
path; failures are logged but never propagated, since the next query will
simply re-verify and re-attempt.
"""
import logging
from qdrant_client.models import FieldCondition, Filter, MatchValue
from nextcloud_mcp_server.config import get_settings
from nextcloud_mcp_server.vector.qdrant_client import get_qdrant_client
logger = logging.getLogger(__name__)
async def delete_document_points(
doc_id: str | int,
doc_type: str,
user_id: str,
) -> None:
"""Remove all Qdrant points for a single document.
Deletes both real chunk points and any leftover placeholder points for the
given (user_id, doc_id, doc_type) tuple. Safe to call when the document is
not present — Qdrant returns successfully with zero points affected.
Args:
doc_id: Document ID (int for notes/files/cards/news, str otherwise)
doc_type: Document type (note, file, deck_card, news_item)
user_id: Owner of the points being evicted
Raises:
Exception: If the underlying Qdrant client raises. Callers in the
search hot path should catch and log; eviction failures must not
block search responses.
"""
qdrant_client = await get_qdrant_client()
settings = get_settings()
await qdrant_client.delete(
collection_name=settings.get_collection_name(),
points_selector=Filter(
must=[
FieldCondition(key="user_id", match=MatchValue(value=user_id)),
FieldCondition(key="doc_id", match=MatchValue(value=doc_id)),
FieldCondition(key="doc_type", match=MatchValue(value=doc_type)),
]
),
)
logger.info(
"Evicted Qdrant points for %s_%s (user=%s); "
"document was inaccessible at verification time",
doc_type,
doc_id,
user_id,
)