fix(vector): address PR review round 3 — sentinel guard, skip indexed fields, narrow types

- Add a fixed-UUID sentinel point written after a successful doc_id
  backfill so subsequent restarts retrieve it and short-circuit the
  O(N) scroll. Sentinel has no user_id/doc_id/doc_type payload so
  production search filters never see it.
- Pre-fetch payload_schema in _ensure_keyword_payload_indexes and
  silently skip fields that are already indexed; the "Created KEYWORD
  payload index" INFO log fires only on actual creation.
- Narrow stale `int | str` doc_id annotations to `str` across
  search/verification.py (BatchVerifier return type, per-verifier
  accessible sets, by_type / accessible_by_type / inaccessible
  collections); drop the now-redundant `type(d).__name__` prefix in
  the dropped-docs log.
- Align the backfill log message with the PR description's
  "Running doc_id backfill" promise; add a caller cross-reference to
  the wait=True comment.
- Fix _get_file_path_from_qdrant docstring (file_id is str, not numeric).
- Convert legacy `id=1` to `id="1"` in test_search_result.py to match
  the SearchResult.id: str annotation.

Three new unit tests cover sentinel-found, sentinel-written, and
skip-existing-index branches; existing backfill tests pass dimension
and explicit retrieve.return_value=[] for the no-sentinel path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Chris Coutinho
2026-05-08 22:59:46 +02:00
co-authored by Claude Opus 4.7
parent b5b4025bb4
commit 92b2d50cd7
5 changed files with 223 additions and 47 deletions
+8 -8
View File
@@ -23,7 +23,7 @@ def _make_point(point_id, payload, score=0.5):
def test_search_result_rrf_score_in_range():
"""Test SearchResult accepts RRF scores in [0.0, 1.0] range."""
result = SearchResult(
id=1,
id="1",
doc_type="note",
title="Test Note",
excerpt="Test excerpt",
@@ -37,7 +37,7 @@ def test_search_result_rrf_score_in_range():
def test_search_result_rrf_score_at_lower_bound():
"""Test SearchResult accepts RRF score at lower bound (0.0)."""
result = SearchResult(
id=1,
id="1",
doc_type="note",
title="Test Note",
excerpt="Test excerpt",
@@ -51,7 +51,7 @@ def test_search_result_rrf_score_at_lower_bound():
def test_search_result_rrf_score_at_upper_bound():
"""Test SearchResult accepts RRF score at upper bound (1.0)."""
result = SearchResult(
id=1,
id="1",
doc_type="note",
title="Test Note",
excerpt="Test excerpt",
@@ -71,7 +71,7 @@ def test_search_result_dbsf_score_above_one():
"""
# Typical DBSF score when both systems agree
result = SearchResult(
id=1,
id="1",
doc_type="note",
title="Highly Relevant Note",
excerpt="Contains keywords and is semantically similar",
@@ -88,7 +88,7 @@ def test_search_result_dbsf_score_edge_case():
Maximum DBSF score with 2 systems: 1.0 (dense) + 1.0 (sparse) = 2.0
"""
result = SearchResult(
id=1,
id="1",
doc_type="note",
title="Perfect Match",
excerpt="Perfect semantic and keyword match",
@@ -103,7 +103,7 @@ def test_search_result_negative_score_raises_error():
"""Test SearchResult rejects negative scores."""
with pytest.raises(ValueError) as exc_info:
SearchResult(
id=1,
id="1",
doc_type="note",
title="Test Note",
excerpt="Test excerpt",
@@ -118,7 +118,7 @@ def test_search_result_negative_score_raises_error():
def test_search_result_with_metadata():
"""Test SearchResult with optional metadata field."""
result = SearchResult(
id=1,
id="1",
doc_type="note",
title="Test Note",
excerpt="Test excerpt",
@@ -136,7 +136,7 @@ def test_search_result_with_metadata():
def test_search_result_with_chunk_offsets():
"""Test SearchResult with chunk offset information."""
result = SearchResult(
id=1,
id="1",
doc_type="note",
title="Test Note",
excerpt="matching chunk text",