fix(review): sample last page, document flags-vs-routing, add flag-path tests

Address PR #855 review (all non-blocking):

- classifier: _sample_indices now always includes the first AND last page (the
  old evenly-spaced sample missed the tail, e.g. last sampled index 95 on a
  100-page doc -- a scanned tail could be missed).
- classifier + metrics: document that flags are diagnostic and fire
  independently of routing (image_heavy on ANY page vs the ocr route needing a
  page FRACTION), so flag{image_heavy} is expected to exceed classified{ocr}.
- classifier: clarify the text-quality whitespace comment (caps at 12%) and note
  the image double-count approximation (min() caps coverage).
- tests: add the scanned (no text layer) and bad_text_layer (junk text over an
  image) flag paths, and a test pinning first/last-page sampling.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Chris Coutinho
2026-06-05 00:12:26 +02:00
co-authored by Claude Opus 4.8
parent 044c1da750
commit 0347e96679
3 changed files with 62 additions and 4 deletions
@@ -285,6 +285,9 @@ document_classified_total = Counter(
)
document_classifier_flag_total = Counter(
# Diagnostic flags, independent of the routing verdict: image_heavy fires if
# ANY page is image-heavy whereas the ocr route needs a fraction of pages,
# so flag{image_heavy} is expected to exceed classified{recommended_tier=ocr}.
"astrolabe_document_classifier_flag_total",
"Tier-0 classifier flags raised on documents",
["flag"], # image_heavy | scanned | bad_text_layer