refactor(search): address PR #750 round 7 review feedback

Round 7 raised 5 issues; this round addresses all of them and fixes
the underlying causes (not just the comments) where applicable so
they don't get re-flagged in future passes.

Critical:
- verified_count description in SemanticSearchResponse said "unique
  documents" but the value is len(verified_results), a chunk count.
  Description rewritten to accurately document chunk-level granularity
  AND explicitly call out the asymmetry with dropped_count (which
  counts unique (doc_id, doc_type) pairs).

- _verify_files false-eviction risk: the round-6 doc-only fix was
  re-flagged. Address at the source — widen WebDAVClient.get_file_info
  to raise HTTPStatusError on 404 (matching the rest of the client
  convention) and reserve None for the genuinely ambiguous
  malformed-PROPFIND case. _verify_files now keeps the result on None
  (cannot tell whether the file exists) and evicts only on a
  definitive HTTPStatusError 404. Tests updated; new test added for
  the malformed-XML keep-result path.

Non-critical:
- News verifier semaphore lifetime now explicitly documented: one
  slot held for one deduplicated fetch per search is the correct
  backpressure behaviour.

- Cross-reference comments in _verify_notes / _verify_deck_cards no
  longer claim "Mirrors X" pointing at functions defined later in
  the file; now use direction-neutral "parallel implementation in".

- accessible_by_type is mutated by concurrent run_verifier tasks; a
  comment explains why this is race-free under anyio's cooperative
  multitasking (distinct keys per task, no await between read and
  write) so a future reader doesn't add a redundant lock.

- Knock-on: tests/integration/test_rag.py wraps get_file_info in a
  try/except for the new contract.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Chris Coutinho
2026-05-01 21:15:39 +02:00
co-authored by Claude Opus 4.7
parent e8df6003c5
commit 3e981e647a
6 changed files with 123 additions and 57 deletions
+26 -3
View File
@@ -317,9 +317,11 @@ async def test_verify_files_uses_path_from_metadata(mocker):
@pytest.mark.unit
async def test_verify_files_404_via_get_file_info_drops(mocker):
"""get_file_info returns None on 404 — that's a definitive drop."""
webdav_client = SimpleNamespace(get_file_info=mocker.AsyncMock(return_value=None))
async def test_verify_files_404_drops(mocker):
"""get_file_info raising HTTPStatusError(404) is a definitive drop."""
webdav_client = SimpleNamespace(
get_file_info=mocker.AsyncMock(side_effect=_http_error(404))
)
client = SimpleNamespace(webdav=webdav_client, username="alice")
result = await _verify_files(
@@ -331,6 +333,27 @@ async def test_verify_files_404_via_get_file_info_drops(mocker):
assert result == set()
@pytest.mark.unit
async def test_verify_files_malformed_propfind_keeps_result(mocker):
"""get_file_info returning None means malformed PROPFIND — keep the result.
Per the contract change in webdav.py: ``None`` is now reserved for the
ambiguous "malformed XML" case. Real 404s raise HTTPStatusError. The
file verifier must NOT evict on the ambiguous case (we cannot tell
whether the file exists), only log a warning and keep the result.
"""
webdav_client = SimpleNamespace(get_file_info=mocker.AsyncMock(return_value=None))
client = SimpleNamespace(webdav=webdav_client, username="alice")
result = await _verify_files(
client,
[_make_result(123, doc_type="file", metadata={"path": "brittle.txt"})],
_sem(),
)
assert result == {123}, "ambiguous None must keep result, not evict"
@pytest.mark.unit
async def test_verify_files_403_drops(mocker):
"""get_file_info raising HTTPStatusError(403) is a definitive drop."""