test(vector): unit-test page-aware routing; clarify fallback comment
Address claude-review round 2 on PR #868: - Extract the use_page_aware branching into a pure `should_use_page_aware` helper and cover the (doc_type, page_boundaries, page_aware_setting) matrix in tests/unit/test_processor_routing.py (file+boundaries+enabled, empty list, None, non-file doc types, disabled setting). - Clarify the PageAwareChunker.chunk_text no-boundaries comment: the processor pre-filters via should_use_page_aware, so that branch is a direct-call safety net, not a production indexing path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
8d20339b3a
commit
bb4ef809c2
@@ -167,8 +167,11 @@ class PageAwareChunker:
|
||||
if not content:
|
||||
return [ChunkWithPosition(text="", start_offset=0, end_offset=0)]
|
||||
|
||||
# No page info (e.g. a non-PDF that reached this path) — degrade to the
|
||||
# char-based behaviour so the caller still gets sensible chunks.
|
||||
# No page info — degrade to char-based behaviour so the class is safe to
|
||||
# call directly. The vector-sync processor pre-filters this case
|
||||
# (``should_use_page_aware`` requires a truthy boundary list), so in
|
||||
# production this branch is only reached by direct callers/tests, not
|
||||
# the indexing path.
|
||||
if not page_boundaries:
|
||||
docs = await anyio.to_thread.run_sync( # type: ignore[attr-defined]
|
||||
self.splitter.create_documents,
|
||||
|
||||
Reference in New Issue
Block a user