feat(ingest): per-tier escalation via procrastinate queue-hop
Split external (procrastinate) document processing into per-tier queues so a
document is attempted at most once per tier and requeued to the next tier's
queue on a low-quality parse, using procrastinate's native retry.
- escalation.py: TIER_LADDER (fast->structured->ocr) + EscalateError signal
- registry: process_tier (one tier) + evaluate_escalation post-parse gate
(reuses classify_from_text) + next_available_tier; shared _classify_result
and _oversize_result with the inline pipeline
- processor: process_document(tier=...) runs one tier and raises EscalateError
before embed (junk text never indexed); inline memory path unchanged
- queue/procrastinate: ingest-fast|structured|ocr queues; TieredEscalationStrategy
(queue-hop on EscalateError, bounded same-tier transient retry); queue-aware
task; producer defers to ingest-fast; per-queue counts + all-queue reclaim
- cli: worker --tier {fast,structured,ocr}
- billing: pages_ocr usage event + pipeline_tier metadata (paid OCR billed apart)
- observability: astrolabe_ingest_queue_depth{queue,status} gauge + per-queue
counts in nc_get_vector_sync_status / management status endpoint
- config: INGEST_ESCALATION_ENABLED (default true), INGEST_TRANSIENT_MAX_ATTEMPTS
INGEST_ESCALATION_ENABLED=false and INGEST_QUEUE=memory preserve prior behaviour.
Deck #323.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
6fab0e2ae3
commit
9676bb3106
@@ -29,6 +29,7 @@ from qdrant_client.models import FieldCondition, Filter, MatchValue
|
||||
|
||||
from nextcloud_mcp_server.config import get_settings
|
||||
from nextcloud_mcp_server.observability.metrics import (
|
||||
update_ingest_queue_depth,
|
||||
update_vector_sync_indexed_chunks,
|
||||
update_vector_sync_indexed_documents,
|
||||
update_vector_sync_pending_documents,
|
||||
@@ -96,6 +97,8 @@ async def publish_vector_sync_metrics(
|
||||
# Keep the legacy gauge meaningful on every consumer path, not just the
|
||||
# single-user one — existing dashboards/alerts reference it.
|
||||
update_vector_sync_queue_size(pending.pending)
|
||||
# Per-tier-queue depth (Deck #323): None on the memory backend (no-op).
|
||||
update_ingest_queue_depth(pending.job_counts_by_queue)
|
||||
except Exception as exc: # noqa: BLE001 — metrics must not break ingest
|
||||
logger.warning("Failed to publish pending-documents gauge: %s", exc)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user