fix(review): _enum_fields validation, gate classify_from_text flags, OCR warnings
Address PR #858 review round 3: - document_tier1_engine / document_ocr_provider now validate + normalize via Settings.__post_init__ _enum_fields (the repo's canonical opt-in-enum pattern; case-insensitive) instead of dynaconf Validators. A typo now raises ValueError at load and "Gateway" normalizes to "gateway". - classify_from_text gates no_text_layer/bad_text_layer on ocr_frac >= OCR_PAGE_FRACTION, matching classify_pdf -- a "fast"-routed doc with a few junk pages no longer emits a misleading flag (keeps the shadow vs hot-path classification metrics consistent). - build_ocr_backend warns when an EXPLICIT provider is misconfigured (gateway without EMBEDDING_GATEWAY_URL, mistral without MISTRAL_API_KEY) instead of silently returning None. - Pypdfium2FastProcessor.health_check probes the import; documented why OcrProcessor.health_check is unconditionally True (lazy per-tenant backends). - Removed the leftover per-boundary / per-chunk debug logging loops. Tests: enum normalization + rejection for the two new settings. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
f1272dfe84
commit
bc957b5bc7
@@ -301,12 +301,9 @@ _dynaconf = Dynaconf(
|
||||
Validator("DOCUMENT_CHUNK_OVERLAP", gte=0),
|
||||
# Non-empty strings
|
||||
Validator("VECTOR_SYNC_PDF_TAG", len_min=1),
|
||||
# Enum constraints
|
||||
# Enum constraints (document_* enums are validated + normalized in
|
||||
# __post_init__ via _enum_fields instead, for case-insensitive input).
|
||||
Validator("LOG_FORMAT", is_in=["text", "json"]),
|
||||
Validator("DOCUMENT_TIER1_ENGINE", is_in=["pypdfium2", "pymupdf"]),
|
||||
Validator(
|
||||
"DOCUMENT_OCR_PROVIDER", is_in=["auto", "gateway", "mistral", "none"]
|
||||
),
|
||||
Validator(
|
||||
"LOG_LEVEL",
|
||||
is_in=["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"],
|
||||
@@ -887,6 +884,8 @@ class Settings:
|
||||
"embedding_provider": {"autodetect", "gateway"},
|
||||
"mcp_role": {"api", "worker", "all"},
|
||||
"collection_metadata_source": {"qdrant", "api"},
|
||||
"document_tier1_engine": {"pypdfium2", "pymupdf"},
|
||||
"document_ocr_provider": {"auto", "gateway", "mistral", "none"},
|
||||
}
|
||||
for _field, _allowed in _enum_fields.items():
|
||||
_val = (getattr(self, _field) or "").strip().lower()
|
||||
|
||||
Reference in New Issue
Block a user