refactor: Optimize Nextcloud access verification with centralized filtering
Move access verification from individual search algorithms to final output stage, eliminating redundant API calls and improving performance. ## Changes **New:** - `search/verification.py`: Centralized verification using anyio task groups - Deduplicates results by (doc_id, doc_type) before verification - Verifies all unique documents in parallel using structured concurrency - Filters out inaccessible documents in single pass **Modified Search Algorithms:** - `search/semantic.py`: Removed _deduplicate_and_verify() and _verify_document_access() - `search/keyword.py`: Removed _verify_access() and parallel verification - `search/fuzzy.py`: Removed _verify_access() and parallel verification - `search/hybrid.py`: Removed nextcloud_client parameter passing All algorithms now return unverified results from Qdrant payload. **Modified Output Stages:** - `server/semantic.py`: Added verify_search_results() call after search - `auth/viz_routes.py`: Added verify_search_results() call after search Both endpoints now verify access once at final stage with deduplication. ## Performance Impact **Before:** - Hybrid mode (limit=10): 30 API calls (10 per algorithm × 3 algorithms) - Single algorithm: 10-20 API calls (with verification buffer) **After:** - Hybrid mode (limit=10): 10 API calls (deduplicated verification) - Single algorithm: 10 API calls (deduplicated verification) **Performance Gain:** 3x reduction in API calls for hybrid search ## Architecture Benefits - **Separation of concerns**: Algorithms handle scoring, output stage handles security - **Deduplication**: Each document verified exactly once - **Parallel execution**: All verifications run concurrently via anyio task groups - **Consistency**: Same verification logic across MCP tools and viz endpoints 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -5,11 +5,7 @@ import logging
|
||||
from collections import defaultdict
|
||||
from typing import Any
|
||||
|
||||
from nextcloud_mcp_server.search.algorithms import (
|
||||
NextcloudClientProtocol,
|
||||
SearchAlgorithm,
|
||||
SearchResult,
|
||||
)
|
||||
from nextcloud_mcp_server.search.algorithms import SearchAlgorithm, SearchResult
|
||||
from nextcloud_mcp_server.search.fuzzy import FuzzySearchAlgorithm
|
||||
from nextcloud_mcp_server.search.keyword import KeywordSearchAlgorithm
|
||||
from nextcloud_mcp_server.search.semantic import SemanticSearchAlgorithm
|
||||
@@ -85,24 +81,22 @@ class HybridSearchAlgorithm(SearchAlgorithm):
|
||||
user_id: str,
|
||||
limit: int = 10,
|
||||
doc_type: str | None = None,
|
||||
nextcloud_client: NextcloudClientProtocol | None = None,
|
||||
**kwargs: Any,
|
||||
) -> list[SearchResult]:
|
||||
"""Execute hybrid search using RRF to combine algorithms.
|
||||
|
||||
Returns unverified results from combined algorithms. Access verification
|
||||
should be performed separately at the final output stage.
|
||||
|
||||
Args:
|
||||
query: Search query
|
||||
user_id: User ID for filtering
|
||||
limit: Maximum results to return
|
||||
doc_type: Optional document type filter
|
||||
nextcloud_client: NextcloudClient for document access
|
||||
**kwargs: Additional parameters passed to sub-algorithms
|
||||
|
||||
Returns:
|
||||
List of SearchResult objects ranked by RRF combined score
|
||||
|
||||
Raises:
|
||||
ValueError: If nextcloud_client not provided (needed for keyword/fuzzy)
|
||||
List of unverified SearchResult objects ranked by RRF combined score
|
||||
"""
|
||||
logger.info(
|
||||
f"Hybrid search: query='{query}', user={user_id}, limit={limit}, "
|
||||
@@ -116,29 +110,19 @@ class HybridSearchAlgorithm(SearchAlgorithm):
|
||||
|
||||
if self.semantic_weight > 0:
|
||||
tasks.append(
|
||||
self.semantic.search(
|
||||
query, user_id, limit * 2, doc_type, nextcloud_client, **kwargs
|
||||
)
|
||||
self.semantic.search(query, user_id, limit * 2, doc_type, **kwargs)
|
||||
)
|
||||
algo_names.append("semantic")
|
||||
|
||||
if self.keyword_weight > 0:
|
||||
if not nextcloud_client:
|
||||
raise ValueError("Hybrid search with keyword requires nextcloud_client")
|
||||
tasks.append(
|
||||
self.keyword.search(
|
||||
query, user_id, limit * 2, doc_type, nextcloud_client, **kwargs
|
||||
)
|
||||
self.keyword.search(query, user_id, limit * 2, doc_type, **kwargs)
|
||||
)
|
||||
algo_names.append("keyword")
|
||||
|
||||
if self.fuzzy_weight > 0:
|
||||
if not nextcloud_client:
|
||||
raise ValueError("Hybrid search with fuzzy requires nextcloud_client")
|
||||
tasks.append(
|
||||
self.fuzzy.search(
|
||||
query, user_id, limit * 2, doc_type, nextcloud_client, **kwargs
|
||||
)
|
||||
self.fuzzy.search(query, user_id, limit * 2, doc_type, **kwargs)
|
||||
)
|
||||
algo_names.append("fuzzy")
|
||||
|
||||
|
||||
Reference in New Issue
Block a user