test(vector): unit-test page-aware routing; clarify fallback comment

Address claude-review round 2 on PR #868:
- Extract the use_page_aware branching into a pure `should_use_page_aware`
  helper and cover the (doc_type, page_boundaries, page_aware_setting) matrix
  in tests/unit/test_processor_routing.py (file+boundaries+enabled, empty
  list, None, non-file doc types, disabled setting).
- Clarify the PageAwareChunker.chunk_text no-boundaries comment: the processor
  pre-filters via should_use_page_aware, so that branch is a direct-call safety
  net, not a production indexing path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Chris Coutinho
2026-06-06 14:02:05 +02:00
co-authored by Claude Opus 4.8
parent 8d20339b3a
commit bb4ef809c2
3 changed files with 90 additions and 9 deletions
@@ -167,8 +167,11 @@ class PageAwareChunker:
if not content:
return [ChunkWithPosition(text="", start_offset=0, end_offset=0)]
# No page info (e.g. a non-PDF that reached this path) — degrade to the
# char-based behaviour so the caller still gets sensible chunks.
# No page info — degrade to char-based behaviour so the class is safe to
# call directly. The vector-sync processor pre-filters this case
# (``should_use_page_aware`` requires a truthy boundary list), so in
# production this branch is only reached by direct callers/tests, not
# the indexing path.
if not page_boundaries:
docs = await anyio.to_thread.run_sync( # type: ignore[attr-defined]
self.splitter.create_documents,