refactor: Optimize Nextcloud access verification with centralized filtering

Move access verification from individual search algorithms to final output
stage, eliminating redundant API calls and improving performance.

## Changes

**New:**
- `search/verification.py`: Centralized verification using anyio task groups
  - Deduplicates results by (doc_id, doc_type) before verification
  - Verifies all unique documents in parallel using structured concurrency
  - Filters out inaccessible documents in single pass

**Modified Search Algorithms:**
- `search/semantic.py`: Removed _deduplicate_and_verify() and _verify_document_access()
- `search/keyword.py`: Removed _verify_access() and parallel verification
- `search/fuzzy.py`: Removed _verify_access() and parallel verification
- `search/hybrid.py`: Removed nextcloud_client parameter passing

All algorithms now return unverified results from Qdrant payload.

**Modified Output Stages:**
- `server/semantic.py`: Added verify_search_results() call after search
- `auth/viz_routes.py`: Added verify_search_results() call after search

Both endpoints now verify access once at final stage with deduplication.

## Performance Impact

**Before:**
- Hybrid mode (limit=10): 30 API calls (10 per algorithm × 3 algorithms)
- Single algorithm: 10-20 API calls (with verification buffer)

**After:**
- Hybrid mode (limit=10): 10 API calls (deduplicated verification)
- Single algorithm: 10 API calls (deduplicated verification)

**Performance Gain:** 3x reduction in API calls for hybrid search

## Architecture Benefits

- **Separation of concerns**: Algorithms handle scoring, output stage handles security
- **Deduplication**: Each document verified exactly once
- **Parallel execution**: All verifications run concurrently via anyio task groups
- **Consistency**: Same verification logic across MCP tools and viz endpoints

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Chris Coutinho
2025-11-15 06:21:06 +01:00
co-authored by Claude
parent ed0825e661
commit 42376483ab
7 changed files with 224 additions and 406 deletions
+8 -24
View File
@@ -5,11 +5,7 @@ import logging
from collections import defaultdict
from typing import Any
from nextcloud_mcp_server.search.algorithms import (
NextcloudClientProtocol,
SearchAlgorithm,
SearchResult,
)
from nextcloud_mcp_server.search.algorithms import SearchAlgorithm, SearchResult
from nextcloud_mcp_server.search.fuzzy import FuzzySearchAlgorithm
from nextcloud_mcp_server.search.keyword import KeywordSearchAlgorithm
from nextcloud_mcp_server.search.semantic import SemanticSearchAlgorithm
@@ -85,24 +81,22 @@ class HybridSearchAlgorithm(SearchAlgorithm):
user_id: str,
limit: int = 10,
doc_type: str | None = None,
nextcloud_client: NextcloudClientProtocol | None = None,
**kwargs: Any,
) -> list[SearchResult]:
"""Execute hybrid search using RRF to combine algorithms.
Returns unverified results from combined algorithms. Access verification
should be performed separately at the final output stage.
Args:
query: Search query
user_id: User ID for filtering
limit: Maximum results to return
doc_type: Optional document type filter
nextcloud_client: NextcloudClient for document access
**kwargs: Additional parameters passed to sub-algorithms
Returns:
List of SearchResult objects ranked by RRF combined score
Raises:
ValueError: If nextcloud_client not provided (needed for keyword/fuzzy)
List of unverified SearchResult objects ranked by RRF combined score
"""
logger.info(
f"Hybrid search: query='{query}', user={user_id}, limit={limit}, "
@@ -116,29 +110,19 @@ class HybridSearchAlgorithm(SearchAlgorithm):
if self.semantic_weight > 0:
tasks.append(
self.semantic.search(
query, user_id, limit * 2, doc_type, nextcloud_client, **kwargs
)
self.semantic.search(query, user_id, limit * 2, doc_type, **kwargs)
)
algo_names.append("semantic")
if self.keyword_weight > 0:
if not nextcloud_client:
raise ValueError("Hybrid search with keyword requires nextcloud_client")
tasks.append(
self.keyword.search(
query, user_id, limit * 2, doc_type, nextcloud_client, **kwargs
)
self.keyword.search(query, user_id, limit * 2, doc_type, **kwargs)
)
algo_names.append("keyword")
if self.fuzzy_weight > 0:
if not nextcloud_client:
raise ValueError("Hybrid search with fuzzy requires nextcloud_client")
tasks.append(
self.fuzzy.search(
query, user_id, limit * 2, doc_type, nextcloud_client, **kwargs
)
self.fuzzy.search(query, user_id, limit * 2, doc_type, **kwargs)
)
algo_names.append("fuzzy")