fix(review): _enum_fields validation, gate classify_from_text flags, OCR warnings
Address PR #858 review round 3: - document_tier1_engine / document_ocr_provider now validate + normalize via Settings.__post_init__ _enum_fields (the repo's canonical opt-in-enum pattern; case-insensitive) instead of dynaconf Validators. A typo now raises ValueError at load and "Gateway" normalizes to "gateway". - classify_from_text gates no_text_layer/bad_text_layer on ocr_frac >= OCR_PAGE_FRACTION, matching classify_pdf -- a "fast"-routed doc with a few junk pages no longer emits a misleading flag (keeps the shadow vs hot-path classification metrics consistent). - build_ocr_backend warns when an EXPLICIT provider is misconfigured (gateway without EMBEDDING_GATEWAY_URL, mistral without MISTRAL_API_KEY) instead of silently returning None. - Pypdfium2FastProcessor.health_check probes the import; documented why OcrProcessor.health_check is unconditionally True (lazy per-tenant backends). - Removed the leftover per-boundary / per-chunk debug logging loops. Tests: enum normalization + rejection for the two new settings. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
f1272dfe84
commit
bc957b5bc7
@@ -595,14 +595,6 @@ async def _index_document(
|
||||
len(page_boundaries),
|
||||
len(content),
|
||||
)
|
||||
# Log first 3 page boundaries for debugging
|
||||
for boundary in page_boundaries[:3]:
|
||||
logger.debug(
|
||||
" Page %s: offsets [%s:%s]",
|
||||
boundary["page"],
|
||||
boundary["start_offset"],
|
||||
boundary["end_offset"],
|
||||
)
|
||||
# Verify last boundary matches text length
|
||||
if page_boundaries:
|
||||
last_boundary = page_boundaries[-1]
|
||||
@@ -659,16 +651,6 @@ async def _index_document(
|
||||
file_path,
|
||||
)
|
||||
|
||||
# Log first 3 chunks to see their page assignments
|
||||
for i, chunk in enumerate(chunks[:3]):
|
||||
logger.debug(
|
||||
" Chunk %s: page=%s, offsets=[%s:%s]",
|
||||
i,
|
||||
chunk.page_number,
|
||||
chunk.start_offset,
|
||||
chunk.end_offset,
|
||||
)
|
||||
|
||||
# Warning if NO page numbers were assigned
|
||||
if assigned_count == 0:
|
||||
logger.warning(
|
||||
|
||||
Reference in New Issue
Block a user