fix(review): _enum_fields validation, gate classify_from_text flags, OCR warnings

Address PR #858 review round 3:

- document_tier1_engine / document_ocr_provider now validate + normalize via
  Settings.__post_init__ _enum_fields (the repo's canonical opt-in-enum pattern;
  case-insensitive) instead of dynaconf Validators. A typo now raises ValueError
  at load and "Gateway" normalizes to "gateway".
- classify_from_text gates no_text_layer/bad_text_layer on ocr_frac >=
  OCR_PAGE_FRACTION, matching classify_pdf -- a "fast"-routed doc with a few junk
  pages no longer emits a misleading flag (keeps the shadow vs hot-path
  classification metrics consistent).
- build_ocr_backend warns when an EXPLICIT provider is misconfigured
  (gateway without EMBEDDING_GATEWAY_URL, mistral without MISTRAL_API_KEY)
  instead of silently returning None.
- Pypdfium2FastProcessor.health_check probes the import; documented why
  OcrProcessor.health_check is unconditionally True (lazy per-tenant backends).
- Removed the leftover per-boundary / per-chunk debug logging loops.

Tests: enum normalization + rejection for the two new settings.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Chris Coutinho
2026-06-05 02:35:26 +02:00
co-authored by Claude Opus 4.8
parent f1272dfe84
commit bc957b5bc7
6 changed files with 37 additions and 25 deletions
-18
View File
@@ -595,14 +595,6 @@ async def _index_document(
len(page_boundaries),
len(content),
)
# Log first 3 page boundaries for debugging
for boundary in page_boundaries[:3]:
logger.debug(
" Page %s: offsets [%s:%s]",
boundary["page"],
boundary["start_offset"],
boundary["end_offset"],
)
# Verify last boundary matches text length
if page_boundaries:
last_boundary = page_boundaries[-1]
@@ -659,16 +651,6 @@ async def _index_document(
file_path,
)
# Log first 3 chunks to see their page assignments
for i, chunk in enumerate(chunks[:3]):
logger.debug(
" Chunk %s: page=%s, offsets=[%s:%s]",
i,
chunk.page_number,
chunk.start_offset,
chunk.end_offset,
)
# Warning if NO page numbers were assigned
if assigned_count == 0:
logger.warning(