test(integration): address round-7 review — keep RAG assertion live, fix races
- test_no_results_for_unrelated_query: replace pytest.skip with `unrelated = ... or 0.0` and fall through. The physics query almost always returns nothing on this corpus, so the skip meant the comparison (and the manual-is-indexed check) never ran. Treating no-results as score 0.0 keeps the test live and vacuously satisfies `0.0 <= relevant`. - test_sampling: the three limit/threshold/max-tokens tests now gate on a representative created note being searchable (search_term + note_id) instead of a bare idle signal that can fire before the new notes are enqueued. - _get_with_retry: only sleep between attempts, not before giving up. - _search_helpers: log the id/doc_type schema-drift mismatch at WARNING (CI runs --log-cli-level=WARN) so it surfaces instead of hiding behind a timeout. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
4c7c627e51
commit
ca313e7271
@@ -432,13 +432,16 @@ async def test_no_results_for_unrelated_query(nc_mcp_client, indexed_manual_pdf)
|
||||
query happened to retrieve any chunk at all). Comparing against a relevant
|
||||
query on the same corpus is self-calibrating and stable.
|
||||
"""
|
||||
unrelated = await _top_score(
|
||||
nc_mcp_client, "quantum entanglement hadron collider particle physics"
|
||||
# No results for the nonsense query is the ideal outcome — treat as score
|
||||
# 0.0 and fall through, so the comparison (and the manual-is-indexed check
|
||||
# below) still runs instead of the test silently skipping every time the
|
||||
# physics query finds nothing.
|
||||
unrelated = (
|
||||
await _top_score(
|
||||
nc_mcp_client, "quantum entanglement hadron collider particle physics"
|
||||
)
|
||||
or 0.0
|
||||
)
|
||||
if unrelated is None:
|
||||
# No results for the nonsense query is the ideal outcome; skip (rather
|
||||
# than a silent pass) so the test report shows the path was taken.
|
||||
pytest.skip("No results for nonsense physics query — the ideal outcome")
|
||||
|
||||
relevant = await _top_score(
|
||||
nc_mcp_client, "how do I enable two-factor authentication"
|
||||
|
||||
Reference in New Issue
Block a user