test(integration): address round-7 review — keep RAG assertion live, fix races

- test_no_results_for_unrelated_query: replace pytest.skip with `unrelated =
  ... or 0.0` and fall through. The physics query almost always returns nothing
  on this corpus, so the skip meant the comparison (and the manual-is-indexed
  check) never ran. Treating no-results as score 0.0 keeps the test live and
  vacuously satisfies `0.0 <= relevant`.
- test_sampling: the three limit/threshold/max-tokens tests now gate on a
  representative created note being searchable (search_term + note_id) instead
  of a bare idle signal that can fire before the new notes are enqueued.
- _get_with_retry: only sleep between attempts, not before giving up.
- _search_helpers: log the id/doc_type schema-drift mismatch at WARNING (CI runs
  --log-cli-level=WARN) so it surfaces instead of hiding behind a timeout.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Chris Coutinho
2026-06-17 23:19:41 +02:00
co-authored by Claude Opus 4.8
parent 4c7c627e51
commit ca313e7271
4 changed files with 29 additions and 16 deletions
+14 -6
View File
@@ -274,8 +274,12 @@ async def test_semantic_search_answer_with_limit(nc_mcp_client, temporary_note_f
category="Development",
)
# Wait for vector indexing to complete
await wait_for_vector_sync(nc_mcp_client)
# Wait until the batch is indexed — gate on the last note being searchable
# rather than a bare idle signal, which can fire before the new notes are
# even enqueued.
await wait_for_vector_sync(
nc_mcp_client, search_term="async context managers", note_id=_note3["id"]
)
call_result = await nc_mcp_client.call_tool(
"nc_semantic_search_answer",
@@ -315,8 +319,10 @@ async def test_semantic_search_answer_score_threshold(
category="Test",
)
# Wait for vector indexing to complete
await wait_for_vector_sync(nc_mcp_client)
# Gate on the new note being searchable (not a bare idle signal).
await wait_for_vector_sync(
nc_mcp_client, search_term="widget manufacturing", note_id=_note["id"]
)
# Query with exact match
call_result = await nc_mcp_client.call_tool(
@@ -362,8 +368,10 @@ async def test_semantic_search_answer_max_tokens(nc_mcp_client, temporary_note_f
category="Test",
)
# Wait for vector indexing to complete
await wait_for_vector_sync(nc_mcp_client)
# Gate on the new note being searchable (not a bare idle signal).
await wait_for_vector_sync(
nc_mcp_client, search_term="Long Document content", note_id=_note["id"]
)
call_result = await nc_mcp_client.call_tool(
"nc_semantic_search_answer",