fix(vector): guard dead-letter on etag, harden marker filter

Addresses round-2 review on PR #920:
- Only dead-letter a terminal failure when the file has an etag to
  content-address the marker; without one, fall back to the legacy per-user
  placeholder mark (an etagless marker is unmatchable). + test.
- _dead_letter_filter now also matches is_placeholder=True (redundant with
  dead_letter=True but lets Qdrant use the is_placeholder payload index).
- TODO(deck-349) documenting the dead-lettered-then-deleted orphan-marker leak
  (out of scope; needs a marker sweep or TTL field) per reviewer.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Chris Coutinho
2026-06-17 19:25:04 +02:00
co-authored by Claude Opus 4.8
parent cd348b3233
commit d720071942
4 changed files with 59 additions and 8 deletions
+17 -1
View File
@@ -25,6 +25,16 @@ filter, plus ``dead_letter=True`` so the orphan-placeholder sweep and the scanne
can tell it apart from a volatile in-flight placeholder. Mirrors the fail-safe
philosophy of ``sharing_state``: a Qdrant error never aborts ingest — a failed
lookup degrades to "process normally", a failed write is logged, not raised.
TODO(deck-349): a marker for a file that is dead-lettered and *then* deleted from
Nextcloud can be orphaned. The processor's delete path clears it, but the
scanner's grace-period deletion tracking only sees a file via its real indexed
points (filtered by ``user_id``); a dead-lettered file has only this
user-agnostic marker, so its disappearance enqueues no delete task and the marker
is never reached. Not a correctness issue (search excludes ``is_placeholder=True``
and the etag check means a stale marker never blocks new content), but it
accumulates. A dedicated marker sweep or a TTL payload field would close it —
tracked as a follow-up, not done here.
"""
import logging
@@ -57,11 +67,17 @@ def _generate_dead_letter_id(doc_type: str, doc_id: str) -> str:
def _dead_letter_filter(doc_id: str, doc_type: str) -> Filter:
"""Match the dead-letter marker for one document (tenant-wide, no user_id)."""
"""Match the dead-letter marker for one document (tenant-wide, no user_id).
Includes ``is_placeholder=True`` (redundant with ``dead_letter=True``, which
nothing else sets) so Qdrant can lean on the existing ``is_placeholder``
payload index.
"""
return Filter(
must=[
FieldCondition(key="doc_id", match=MatchValue(value=doc_id)),
FieldCondition(key="doc_type", match=MatchValue(value=doc_type)),
FieldCondition(key="is_placeholder", match=MatchValue(value=True)),
FieldCondition(key=DEAD_LETTER_KEY, match=MatchValue(value=True)),
]
)