fix(review): sample last page, document flags-vs-routing, add flag-path tests
Address PR #855 review (all non-blocking): - classifier: _sample_indices now always includes the first AND last page (the old evenly-spaced sample missed the tail, e.g. last sampled index 95 on a 100-page doc -- a scanned tail could be missed). - classifier + metrics: document that flags are diagnostic and fire independently of routing (image_heavy on ANY page vs the ocr route needing a page FRACTION), so flag{image_heavy} is expected to exceed classified{ocr}. - classifier: clarify the text-quality whitespace comment (caps at 12%) and note the image double-count approximation (min() caps coverage). - tests: add the scanned (no text layer) and bad_text_layer (junk text over an image) flag paths, and a test pinning first/last-page sampling. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
044c1da750
commit
0347e96679
@@ -285,6 +285,9 @@ document_classified_total = Counter(
|
||||
)
|
||||
|
||||
document_classifier_flag_total = Counter(
|
||||
# Diagnostic flags, independent of the routing verdict: image_heavy fires if
|
||||
# ANY page is image-heavy whereas the ocr route needs a fraction of pages,
|
||||
# so flag{image_heavy} is expected to exceed classified{recommended_tier=ocr}.
|
||||
"astrolabe_document_classifier_flag_total",
|
||||
"Tier-0 classifier flags raised on documents",
|
||||
["flag"], # image_heavy | scanned | bad_text_layer
|
||||
|
||||
Reference in New Issue
Block a user