Commit Graph
3 Commits
Author SHA1 Message Date
Chris CoutinhoandClaude Opus 4.8 ce53e21ead fix(ingest): zero queue-depth gauge on all-queues-drained (review round 4)
- metrics: update_ingest_queue_depth guarded on `not by_queue`, which conflated
  None (memory backend no-op) with {} (postgres, ALL queues drained). When every
  queue drains at once, get_ingest_job_counts_by_queue returns {} and the
  pre-zero loop was skipped, leaving a stale ghost backlog in the gauge. Guard on
  `by_queue is None` only; add an all-drained regression test.
- procrastinate: note that INGEST_TRANSIENT_MAX_ATTEMPTS is snapshotted at
  blueprint-build time (restart to pick up changes).

Deck #323.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 14:10:01 +02:00
Chris CoutinhoandClaude Opus 4.8 44f72839ed fix(ingest): address review round 3 + SonarCloud reliability gate
- tests: make the transient-backoff progression assertion load-independent by
  bracketing the get_retry_decision call with before/after timestamps instead of
  measuring against a second datetime.now() (no freezegun dependency).
- tests: use pytest.approx for the ingest-queue-depth gauge assertions —
  SonarCloud python:S1244 (float == ) was a MAJOR reliability finding that
  tripped the new_reliability_rating quality gate.
- processor: tighten the EscalateError lazy-bind comment (file processing already
  imports the document stack via get_registry; the gating only spares the
  delete / text-doc paths and module-load time).

Deck #323.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 14:00:20 +02:00
Chris CoutinhoandClaude Opus 4.8 e7c0c23486 fix(ingest): address review round 2 (stale gauge + hygiene)
- metrics: update_ingest_queue_depth now pre-zeroes every managed ingest queue
  before applying live counts, so a queue that drains to empty (and drops out of
  procrastinate's list_queues_async) reads 0 instead of sticking at its last
  non-zero value (ghost backlog in Grafana/alerts). Adds a regression test.
- procrastinate: comment that _is_transient_infra_error treats all qdrant errors
  as transient deliberately (bounded same-tier retry; over-broad is acceptable).
- escalation: note next_tier is the building block; production routing uses
  ProcessorRegistry.next_available_tier.
- tests: add evaluate_escalation fast+ocr-only low-confidence -> ocr case.

Deck #323.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 13:45:01 +02:00