perf(search): skip exclusion lookup on empty tag set; fix semaphore comment
Address the two important findings from the claude bot's latest re-review: - _verify_files: skip get_excluded_file_paths entirely when the tag REPORT returns no files. An empty `tagged` yields an empty `tagged_ids` regardless of exclusions, so the lookup's 2xlen(EXCLUDED_TAGS) WebDAV fan-out is wasted work in the common "this tag matched nothing" case. The per-result loop still runs, so malformed doc_ids are still kept (fail-open) — pinned by a new test (test_verify_files_empty_tag_set_skips_exclusion_lookup), which also asserts the exclusion lookup is never awaited. - Rewrite the semaphore comment: it claimed "the slot bounds them", but the slot only caps concurrent *searches* — get_excluded_file_paths internally spawns a task group issuing 2xlen(EXCLUDED_TAGS) concurrent WebDAV calls, so live Nextcloud connections can exceed VERIFICATION_CONCURRENCY. Comment now says so and points at configuration.md. The third 🟡 (sequential dir expansion in find_files_by_tag) is pre-existing and flagged by the reviewer as a follow-up, not part of this PR. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
7033c64393
commit
b736bf199b
@@ -504,6 +504,30 @@ async def test_verify_files_deleted_drops(mocker):
|
||||
assert result == set()
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
async def test_verify_files_empty_tag_set_skips_exclusion_lookup(mocker):
|
||||
"""When the tag REPORT returns no files, the EXCLUDED_TAGS lookup is skipped
|
||||
entirely: an empty tagged set drops every valid-id result regardless of
|
||||
exclusions, so the lookup's 2xN WebDAV fan-out is wasted work. Malformed
|
||||
doc_ids are still kept (fail-open), exactly as on the non-empty path."""
|
||||
excluded = _patch_excluded(mocker, {"Secret"})
|
||||
client = _file_client(mocker, tagged=[])
|
||||
|
||||
result = await _verify_files(
|
||||
client,
|
||||
[
|
||||
_make_result(123, doc_type="file"),
|
||||
_make_result("not-a-file-id", doc_type="file"),
|
||||
],
|
||||
_sem(),
|
||||
)
|
||||
|
||||
# Valid id absent from the (empty) tagged set → dropped; malformed id kept.
|
||||
assert result == {"not-a-file-id"}
|
||||
# The optimization: no exclusion fan-out when there is nothing to filter.
|
||||
excluded.assert_not_awaited()
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
async def test_verify_files_excluded_path_drops(mocker):
|
||||
"""A tagged file under an EXCLUDED_TAGS folder must not surface — exclusion
|
||||
|
||||
Reference in New Issue
Block a user