fix(vector): address PR review round 15 — concurrency, pagination, stale coercion

Defer publication of `_qdrant_client` until after the in-lock backfill +
payload-index migration awaits complete. The fast-path check at the top of
`get_qdrant_client` reads the singleton without holding the init lock, so
publishing the constructed-but-unmigrated client let concurrent fast-path
callers fire filtered searches before `_ensure_payload_indexes` ran —
producing HTTP 400 ("Index required but not found") on Qdrant Cloud strict
mode. Local `provisional` is now used for every await inside the lock; the
global is assigned exactly once, last.

Replace the five hand-rolled `scroll(..., limit=10000)` calls in
`vector/scanner.py` (notes / files / news / deck-cards deletion tracking,
plus the timestamp scroll) with a single paginated `_scroll_all_points`
helper. The previous single-page cap silently dropped deletion-tracking
points beyond the first 10 k for any user past that threshold. Pagination
follows Qdrant's documented contract (loop until `next_page_offset is
None`) with a fixed per-page `_DELETION_TRACKING_PAGE_SIZE = 1024`.

Extract `_create_one_payload_index` from `_ensure_payload_indexes` to drop
its cognitive complexity below the SonarQube limit (17 → ≤ 15) without
losing the per-field error-containment rationale; every comment is
preserved verbatim on the helper.

Drop the stale `SearchResult.id` `int | str` comment and the redundant
`str(d)` coercion in `_verify_news_items` — the contract has been
str-only since the producer-side stringification landed earlier in this
PR.

Fix eight `doc_id=<int>` test calls in `test_chunk_context_offset_gate.py`
that violated the `doc_id: str` signature of `get_chunk_with_context`,
plus align `_make_result` in `test_verification.py` to coerce `id=str(...)`
matching the production contract — and update 30+ assertions from int
sets (`{1, 2, 3}`) to str sets (`{"1", "2", "3"}`) so the tests now model
the post-PR `SearchResult.id: str` reality end-to-end. Previously these
were masked by the `str(d)` coercion now removed from production.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Chris Coutinho
2026-05-10 13:49:17 +02:00
co-authored by Claude Opus 4.7
parent c5020d9629
commit 68506f96c5
5 changed files with 220 additions and 143 deletions
+3 -3
View File
@@ -389,9 +389,9 @@ async def _verify_news_items(
# above for why this is narrower than the API-response failure path.
accessible: set[str] = set()
for d in doc_ids:
# SearchResult.id is `int | str` (D1: forward-compat widening). Coerce
# to str so the validator's regex applies consistently to both shapes.
if not is_valid_nextcloud_doc_id(str(d)):
# SearchResult.id is always str (Qdrant payload doc_id is keyword-
# indexed; producers stringify on write). Pass through verbatim.
if not is_valid_nextcloud_doc_id(d):
logger.warning(
"Malformed news_item doc_id %r in verifier; keeping (cannot verify)",
d,