docs(vector): note tiers_sig extensibility, warn on dead-letter placeholder cleanup failure
Round-3 review nits on PR #920 (none blocking): - escalation_tiers_signature: TODO noting future settings that can rescue a previously-terminal document (a toggleable llm tier, a raised oversize cap) should be folded into the signature so raising them auto-retries dead-letters. - Terminal-path placeholder cleanup: a delete failure here is real Qdrant I/O, not control-flow -- log at warning (was debug) for observability. Non-fatal (the durable marker is already written). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
d720071942
commit
a7f7461716
@@ -45,6 +45,13 @@ def escalation_tiers_signature(settings: Any) -> str:
|
|||||||
pin (``document_tier1_engine``). Enabling OCR changes the signature, so the
|
pin (``document_tier1_engine``). Enabling OCR changes the signature, so the
|
||||||
pathological-but-OCR-recoverable documents dead-lettered while OCR was off are
|
pathological-but-OCR-recoverable documents dead-lettered while OCR was off are
|
||||||
re-attempted automatically.
|
re-attempted automatically.
|
||||||
|
|
||||||
|
TODO: when a future setting can make a previously-terminal document parseable,
|
||||||
|
fold it in here so raising it auto-retries existing dead-letters. Two known
|
||||||
|
candidates: a new escalation tier becoming toggleable (e.g. the reserved
|
||||||
|
``llm`` rung in ``TIER_LADDER``), and a raised oversize cap (an oversize PDF is
|
||||||
|
always-terminal, so without the cap in this signature it stays dead-lettered
|
||||||
|
until its etag changes even after an operator allows bigger files).
|
||||||
"""
|
"""
|
||||||
return (
|
return (
|
||||||
f"ocr={int(bool(settings.document_ocr_enabled))};"
|
f"ocr={int(bool(settings.document_ocr_enabled))};"
|
||||||
|
|||||||
@@ -1108,7 +1108,11 @@ async def _index_document(
|
|||||||
user_id=doc_task.user_id,
|
user_id=doc_task.user_id,
|
||||||
)
|
)
|
||||||
except Exception:
|
except Exception:
|
||||||
logger.debug(
|
# A real Qdrant I/O failure (not control-flow): warn so
|
||||||
|
# it's observable. Non-fatal -- the durable dead-letter
|
||||||
|
# marker is already written, so the leftover volatile
|
||||||
|
# placeholder is merely redundant.
|
||||||
|
logger.warning(
|
||||||
"Could not delete placeholder for dead-lettered %s",
|
"Could not delete placeholder for dead-lettered %s",
|
||||||
doc_task.doc_id,
|
doc_task.doc_id,
|
||||||
exc_info=True,
|
exc_info=True,
|
||||||
|
|||||||
Reference in New Issue
Block a user