fix(ingest): address review round 2 (stale gauge + hygiene)
- metrics: update_ingest_queue_depth now pre-zeroes every managed ingest queue before applying live counts, so a queue that drains to empty (and drops out of procrastinate's list_queues_async) reads 0 instead of sticking at its last non-zero value (ghost backlog in Grafana/alerts). Adds a regression test. - procrastinate: comment that _is_transient_infra_error treats all qdrant errors as transient deliberately (bounded same-tier retry; over-broad is acceptable). - escalation: note next_tier is the building block; production routing uses ProcessorRegistry.next_available_tier. - tests: add evaluate_escalation fast+ocr-only low-confidence -> ocr case. Deck #323. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
35f8204a16
commit
e7c0c23486
@@ -400,3 +400,23 @@ def test_evaluate_escalation_zero_page_does_not_escalate(monkeypatch):
|
||||
processor="fast",
|
||||
)
|
||||
assert r.evaluate_escalation(res, b"%PDF", "fast", _Settings(ocr=True)) is None
|
||||
|
||||
|
||||
def test_evaluate_escalation_lowconf_to_ocr_when_no_structured(monkeypatch):
|
||||
"""fast+ocr only: a low-confidence parse routes straight to ocr (skips the
|
||||
unregistered structured rung), not to None."""
|
||||
monkeypatch.setattr(reg_mod, "record_document_classification", MagicMock())
|
||||
junk = "z" * 40 # non-empty but junk -> recommended ocr, total_chars > 0
|
||||
r = _registry((_Fake("fast", "fast"), 20), (_Fake("ocr", "ocr"), 5))
|
||||
res = ProcessingResult(
|
||||
text=junk,
|
||||
metadata={
|
||||
"page_count": 1,
|
||||
"page_boundaries": [
|
||||
{"page": 1, "start_offset": 0, "end_offset": len(junk)}
|
||||
],
|
||||
},
|
||||
processor="fast",
|
||||
)
|
||||
decision = r.evaluate_escalation(res, b"%PDF", "fast", _Settings(ocr=True))
|
||||
assert decision == ("ocr", "low_confidence")
|
||||
|
||||
Reference in New Issue
Block a user