refactor(search): address PR #750 round 7 review feedback
Round 7 raised 5 issues; this round addresses all of them and fixes the underlying causes (not just the comments) where applicable so they don't get re-flagged in future passes. Critical: - verified_count description in SemanticSearchResponse said "unique documents" but the value is len(verified_results), a chunk count. Description rewritten to accurately document chunk-level granularity AND explicitly call out the asymmetry with dropped_count (which counts unique (doc_id, doc_type) pairs). - _verify_files false-eviction risk: the round-6 doc-only fix was re-flagged. Address at the source — widen WebDAVClient.get_file_info to raise HTTPStatusError on 404 (matching the rest of the client convention) and reserve None for the genuinely ambiguous malformed-PROPFIND case. _verify_files now keeps the result on None (cannot tell whether the file exists) and evicts only on a definitive HTTPStatusError 404. Tests updated; new test added for the malformed-XML keep-result path. Non-critical: - News verifier semaphore lifetime now explicitly documented: one slot held for one deduplicated fetch per search is the correct backpressure behaviour. - Cross-reference comments in _verify_notes / _verify_deck_cards no longer claim "Mirrors X" pointing at functions defined later in the file; now use direction-neutral "parallel implementation in". - accessible_by_type is mutated by concurrent run_verifier tasks; a comment explains why this is race-free under anyio's cooperative multitasking (distinct keys per task, no await between read and write) so a future reader doesn't add a redundant lock. - Knock-on: tests/integration/test_rag.py wraps get_file_info in a try/except for the new contract. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.7
parent
e8df6003c5
commit
3e981e647a
@@ -317,9 +317,11 @@ async def test_verify_files_uses_path_from_metadata(mocker):
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
async def test_verify_files_404_via_get_file_info_drops(mocker):
|
||||
"""get_file_info returns None on 404 — that's a definitive drop."""
|
||||
webdav_client = SimpleNamespace(get_file_info=mocker.AsyncMock(return_value=None))
|
||||
async def test_verify_files_404_drops(mocker):
|
||||
"""get_file_info raising HTTPStatusError(404) is a definitive drop."""
|
||||
webdav_client = SimpleNamespace(
|
||||
get_file_info=mocker.AsyncMock(side_effect=_http_error(404))
|
||||
)
|
||||
client = SimpleNamespace(webdav=webdav_client, username="alice")
|
||||
|
||||
result = await _verify_files(
|
||||
@@ -331,6 +333,27 @@ async def test_verify_files_404_via_get_file_info_drops(mocker):
|
||||
assert result == set()
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
async def test_verify_files_malformed_propfind_keeps_result(mocker):
|
||||
"""get_file_info returning None means malformed PROPFIND — keep the result.
|
||||
|
||||
Per the contract change in webdav.py: ``None`` is now reserved for the
|
||||
ambiguous "malformed XML" case. Real 404s raise HTTPStatusError. The
|
||||
file verifier must NOT evict on the ambiguous case (we cannot tell
|
||||
whether the file exists), only log a warning and keep the result.
|
||||
"""
|
||||
webdav_client = SimpleNamespace(get_file_info=mocker.AsyncMock(return_value=None))
|
||||
client = SimpleNamespace(webdav=webdav_client, username="alice")
|
||||
|
||||
result = await _verify_files(
|
||||
client,
|
||||
[_make_result(123, doc_type="file", metadata={"path": "brittle.txt"})],
|
||||
_sem(),
|
||||
)
|
||||
|
||||
assert result == {123}, "ambiguous None must keep result, not evict"
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
async def test_verify_files_403_drops(mocker):
|
||||
"""get_file_info raising HTTPStatusError(403) is a definitive drop."""
|
||||
|
||||
Reference in New Issue
Block a user