fix(ingest): address review round 1 (reclaim queue + tests)
- Register the periodic stalled-job reclaim on a dedicated ingest-maintenance queue that every worker drains (any --tier), so reclaim still fires when the fast fleet is scaled to zero and only ocr workers run. procrastinate's periodic-defer dedup keeps it single-run across drainers. - escalation: mark `unsupported`/`forced` reason labels as reserved (not raised). - processor: note that options/progress_callback are intentionally not threaded through _parse_pdf_tier yet (symmetric with the inline path). - tests: assert TieredEscalationStrategy backoff progression (4/8/16/…/300s); cover get_ingest_pending per-queue aggregation + the legacy job_counts fallback; add an external-path zero-page no-escalation case; use the canonical INGEST_QUEUE_FAST instead of the back-compat alias. Deck #323. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
9676bb3106
commit
35f8204a16
@@ -388,3 +388,15 @@ def test_evaluate_escalation_terminal_when_no_higher_tier(monkeypatch):
|
||||
processor="fast",
|
||||
)
|
||||
assert r.evaluate_escalation(res, b"%PDF", "fast", _Settings(ocr=True)) is None
|
||||
|
||||
|
||||
def test_evaluate_escalation_zero_page_does_not_escalate(monkeypatch):
|
||||
"""A zero-page (empty/corrupt) PDF never escalates on the external path."""
|
||||
monkeypatch.setattr(reg_mod, "record_document_classification", MagicMock())
|
||||
r = _registry((_Fake("fast", "fast"), 20), (_Fake("ocr", "ocr"), 5))
|
||||
res = ProcessingResult(
|
||||
text="",
|
||||
metadata={"page_count": 0, "page_boundaries": []},
|
||||
processor="fast",
|
||||
)
|
||||
assert r.evaluate_escalation(res, b"%PDF", "fast", _Settings(ocr=True)) is None
|
||||
|
||||
Reference in New Issue
Block a user