Chris CoutinhoandClaude Opus 4.7
ee402ea00e
Per-chunk PDF page renders (~150–700 KB base64 PNG each) were the dominant disk consumer in production, repeatedly tripping `No space left on device: WAL buffer size exceeds available disk space` on welcomed-malamute Qdrant. Replace the inline highlighted_page_image / highlighted_page_number / highlight_count fields with a small `chunk_bbox` field: list[(x0, y0, x1, y1)] of normalized [0, 1] floats, ~32 bytes per chunk. Astrolabe (the only known consumer) renders the highlight client-side as a percentage-positioned overlay on top of the existing /api/v1/pdf-preview render-on-demand path (cbcoutinho/astrolabe#76). - pdf_highlighter: new compute_chunk_bboxes_batch() that reuses the existing _find_chunk_bbox text-search path, skipping all pixmap/PIL/PNG work. - processor: store chunk_bbox + chunk_bbox_page in the Qdrant payload, drop highlighted_page_image + friends, drop the base64 import. - visualization /api/v1/chunk-context and auth/viz_routes: read chunk_bbox instead of highlighted_page_image. - vector/__init__: stop eagerly re-exporting `processor`/`scanner` — fixes a pre-existing circular import (search.algorithms -> vector.placeholder -> vector/__init__ -> processor -> scanner -> server.semantic -> search.bm25_hybrid -> search.algorithms partial). Test suite that was broken on master (test_bm25_hybrid.py et al.) now collects and passes. - scripts/purge_page_images.py: ad-hoc, idempotent migration that delete_payload's the legacy keys from existing points. No reindex required; legacy chunks render the page with no overlay. Pairs with cbcoutinho/astrolabe#76. Frontend handles missing chunk_bbox gracefully, so this can land in either order. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
18 lines
600 B
Python
18 lines
600 B
Python
"""Vector database and background sync package.
|
|
|
|
`processor` and `scanner` are intentionally NOT re-exported from this
|
|
package init: they transitively import `server.semantic` ->
|
|
`search.bm25_hybrid`, which forms an import cycle with
|
|
`search.algorithms` -> `vector.placeholder` -> `vector/__init__`.
|
|
Consumers that need those symbols import them from their submodules
|
|
directly (e.g. `from nextcloud_mcp_server.vector.processor import ...`).
|
|
"""
|
|
|
|
from .document_chunker import DocumentChunker
|
|
from .qdrant_client import get_qdrant_client
|
|
|
|
__all__ = [
|
|
"get_qdrant_client",
|
|
"DocumentChunker",
|
|
]
|