Improve user comprehension by scaling RRF scores to match the intuitive
0-1 range used by other search algorithms.
## Problem
RRF (Reciprocal Rank Fusion) scores had a drastically different scale
than semantic/keyword/fuzzy scores:
- Semantic similarity: 0.0 to 1.0 (typical: 0.5-0.9)
- RRF scores: 0.0 to ~0.016 (typical: 0.005-0.015)
This caused user confusion - a score of 0.0078 looked terrible but was
actually excellent (near theoretical maximum).
## Solution
Normalize RRF scores using the formula:
`normalized_score = rrf_score * (rrf_k + 1) / total_weight`
Where:
- rrf_k = 60 (RRF constant)
- total_weight = sum of algorithm weights (default: 1.0)
**Example transformation:**
- Before: 0.0078 (confusing)
- After: 0.477 (intuitive)
## Changes
**nextcloud_mcp_server/search/hybrid.py:**
- Store total_weight as instance variable (line 63)
- Calculate normalization factor in _reciprocal_rank_fusion() (line 209)
- Apply normalization to all RRF scores (line 217)
- Preserve raw RRF score in metadata for debugging (line 222)
## Impact
**User Experience:**
- Hybrid search scores now comparable with semantic/keyword/fuzzy
- Score of 0.5 indicates good match across all algorithms
- Consistent scale improves score threshold usability
**Backward Compatibility:**
- Raw RRF scores preserved in metadata["rrf_score_raw"]
- Result ordering unchanged (normalization is linear transformation)
- Breaking change: Existing score thresholds need adjustment
**Performance:**
- Negligible overhead (single multiplication per result)
## Testing
Verified with nc_semantic_search and nc_semantic_search_answer:
- Hybrid scores now 0.47-0.7 range (was 0.003-0.011)
- Semantic scores unchanged (0.75)
- Result ordering preserved
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Move access verification from individual search algorithms to final output
stage, eliminating redundant API calls and improving performance.
## Changes
**New:**
- `search/verification.py`: Centralized verification using anyio task groups
- Deduplicates results by (doc_id, doc_type) before verification
- Verifies all unique documents in parallel using structured concurrency
- Filters out inaccessible documents in single pass
**Modified Search Algorithms:**
- `search/semantic.py`: Removed _deduplicate_and_verify() and _verify_document_access()
- `search/keyword.py`: Removed _verify_access() and parallel verification
- `search/fuzzy.py`: Removed _verify_access() and parallel verification
- `search/hybrid.py`: Removed nextcloud_client parameter passing
All algorithms now return unverified results from Qdrant payload.
**Modified Output Stages:**
- `server/semantic.py`: Added verify_search_results() call after search
- `auth/viz_routes.py`: Added verify_search_results() call after search
Both endpoints now verify access once at final stage with deduplication.
## Performance Impact
**Before:**
- Hybrid mode (limit=10): 30 API calls (10 per algorithm × 3 algorithms)
- Single algorithm: 10-20 API calls (with verification buffer)
**After:**
- Hybrid mode (limit=10): 10 API calls (deduplicated verification)
- Single algorithm: 10 API calls (deduplicated verification)
**Performance Gain:** 3x reduction in API calls for hybrid search
## Architecture Benefits
- **Separation of concerns**: Algorithms handle scoring, output stage handles security
- **Deduplication**: Each document verified exactly once
- **Parallel execution**: All verifications run concurrently via anyio task groups
- **Consistency**: Same verification logic across MCP tools and viz endpoints
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Vector Visualization Improvements:
- Add interactive vector viz tab with Alpine.js and Plotly.js to user info page
- Refactor viz route CSS for better scoping and maintainability
- Remove unused nextcloud_host variable
Performance Optimizations:
- Parallelize access verification in fuzzy and keyword search algorithms
- Use asyncio.gather() to verify multiple documents concurrently
- Add exception handling with return_exceptions=True for resilience
Dependencies:
- Update third_party/oidc submodule to include RFC 9728 resource_url support
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Skip tracing for /app/vector-sync/status to reduce noise from HTMX polling.
Metrics collection continues for this endpoint.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Move Webhooks tab to the right (User Info | Vector Sync | Vector Viz | Webhooks)
- Use request.user.display_name instead of session for viz routes
- Fixes session middleware error when accessing via iframe
- Add /app/vector-viz endpoint for interactive search testing
- Implement server-side PCA dimensionality reduction (768-dim → 2D)
- Support multi-select document type filter for cross-app search
- Support all search algorithms: semantic, keyword, fuzzy, hybrid
- Display 2D scatter plot of vector embeddings using Plotly
- Show search results with scores and document types
- Register viz routes in app.py
- Add custom PCA implementation using numpy eigendecomposition
- Replace sklearn.decomposition.PCA with custom implementation
- Maintains same API (fit, transform, fit_transform)
- Supports explained_variance_ratio_ for variance analysis
- Removes scikit-learn dependency from project
- Add type hints and assertion for type safety
BREAKING CHANGE: Search algorithms now require Qdrant to be populated.
Vector sync must be enabled and documents indexed for search to work.
- Keyword and fuzzy search now query Qdrant scroll API for title/excerpt
- Remove inefficient Nextcloud API fetching pattern
- Add optional Nextcloud verification for security
- Deduplicate by (doc_id, doc_type) tuple, keeping chunk_index=0
- Align with document processor pattern that already stores text in Qdrant
Implements NextcloudClientProtocol for multi-document type search following
user requirement that document types are not 1:1 with apps (e.g., Notes app
specializes in markdown, while Files/WebDAV handles multiple file types).
Key Changes:
- NextcloudClientProtocol: Generic protocol with app-specific client properties
- get_indexed_doc_types(): Query Qdrant for actually-indexed document types
- Document dispatch: All algorithms check Qdrant before attempting access
- Cross-type deduplication: Use (doc_id, doc_type) tuples in hybrid RRF
Search Algorithm Updates:
- Semantic: Added _verify_document_access() with dispatch to appropriate client
- Deduplication by (doc_id, doc_type) tuple
- Only "note" verification implemented, others return None with info log
- Keyword: Added _fetch_documents() dispatch method
- Queries Qdrant for available types before fetching
- Supports cross-app search when doc_type=None
- Fuzzy: Same pattern as keyword search
- Hybrid: Already uses (doc_id, doc_type) for deduplication (no changes needed)
Future-Proof Design:
- File/calendar verification stubs in place
- Clear logging when unsupported types found
- Easy to extend when processor indexes new document types
Currently Supported:
- "note" documents fully implemented and tested
- Other types gracefully handled (logged but skipped)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Implements ADR-012 by adding multi-algorithm support to the MCP tool.
Key changes:
- Added algorithm parameter: "semantic"|"keyword"|"fuzzy"|"hybrid" (default: "hybrid")
- Added weight parameters for hybrid mode configuration
- Replaced direct Qdrant/embedding calls with search module abstractions
- Updated docstring to describe all four algorithms
- Simplified implementation: ~50 lines vs ~150 lines (67% reduction)
- Better error handling for missing vector sync
Algorithm selection:
- semantic: Pure vector similarity (requires VECTOR_SYNC_ENABLED=true)
- keyword: Token-based matching with weighted title/content scoring
- fuzzy: Character overlap for typo tolerance
- hybrid: RRF fusion with configurable weights (default: 0.5/0.3/0.2)
Backward compatibility:
- Tool name unchanged (nc_semantic_search)
- New parameters have sensible defaults
- Existing clients get hybrid search automatically (better than pure semantic)
- search_method field in response reflects actual algorithm used
Weight validation:
- Performed in HybridSearchAlgorithm constructor
- Must sum to ≤1.0 and all non-negative
- At least one weight must be > 0
- Clear error messages on validation failure
Next: Update viz pane to use same algorithms
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Updates ADR-012 to clarify that all search and filtering operations
must happen server-side, not in the browser.
Key changes:
- Enhanced viz pane data flow showing server-side processing
- Added performance benefits section (384x bandwidth reduction)
- Detailed server-side filtering approach:
* Query execution via search/algorithms.py
* User ID filtering (multi-tenant security)
* Document type filtering
* PCA reduction (768-dim → 2D) on server
* Only 2D coordinates + metadata sent to client
- Updated Phase 3 implementation plan:
* Remove ALL client-side search logic
* Implement /app/vector-viz server endpoint
* htmx form submission for queries
* Performance optimizations (caching, streaming)
This ensures:
- Minimal bandwidth usage (only 2 floats per doc vs 768)
- Client handles only visualization, not computation
- Can visualize 10,000+ documents without client lag
- Raw vectors never leave server (security)
- Same search logic as MCP tool (consistency)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Enhances ADR-012 with detailed architecture visualization and UI mockup
for the vector visualization pane.
Added sections:
- Architecture diagram showing MCP tool and viz pane integration
- Data flow diagrams for both MCP requests and viz pane interactions
- Detailed UI mockup with ASCII art showing:
* Search configuration controls
* Algorithm selector with weight sliders
* Interactive 2D scatter plot (Plotly.js)
* Results panel with scores
* Performance comparison table
- Technology stack details (htmx, Alpine.js, Plotly.js, Tailwind CSS)
The diagrams illustrate how the viz pane and MCP tool share the same
search algorithm implementations from search/algorithms.py, ensuring
consistency between user testing interface and programmatic API.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Changes:
- Remove streamable-http transport override from mcp service in docker-compose.yml
- Service now uses CLI default SSE transport on /sse endpoint
- Add create_mcp_client_session_sse() helper for SSE connections
- Update nc_mcp_client fixture to use SSE transport
- Fix unpacking for SSE client (yields 2 values vs 3 for streamable-http)
Testing:
- All 4 smoke tests pass with SSE transport
- 32/34 affected tests pass (2 skipped for vector sync)
- OAuth services remain on streamable-http (unchanged)
Note: SSE transport is being deprecated in favor of streamable-http.
This enables minimal validation testing before deprecation.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
After comprehensive research, the hybrid OAuth + AppAPI architecture is NOT
being implemented due to fundamental architectural incompatibilities.
Key updates:
- Status: Proposed → Not Planned
- Added validation from Nextcloud Context Agent project
- Context Agent (official NC ExApp with MCP) faces IDENTICAL limitations
- Proves constraints are architectural, not implementation-specific
Context Agent findings:
- ExApp with MCP server endpoint (~28 tools exposed)
- Uses Task Processing API for confirmations (NOT MCP elicitation)
- Works around AppAPI proxy limitations by changing protocol
- MCP endpoint is secondary feature with documented constraints
- Primary use: In-app Assistant integration, not external MCP clients
Critical features impossible through AppAPI proxy:
- ❌ MCP sampling (eliminates RAG/LLM features)
- ❌ MCP elicitation (user prompts)
- ❌ Real-time progress updates
- ❌ Bidirectional streaming
- Validated by Context Agent facing same limitations
Decision rationale:
- MCP requires multi-turn nested interactions
- AppAPI provides stateless request/response proxy only
- No implementation effort can bridge this fundamental gap
- Would require complete AppAPI redesign (WebSocket, message routing)
- Even official Nextcloud projects work around these limitations
Alternative considered for future:
- Register as Task Processing provider (different product)
- Use Nextcloud Assistant UI (not external MCP clients)
- Accept different capabilities (no sampling, custom flows)
OAuth mode remains sole solution for external MCP client integration.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Previously, an empty query string to nc_notes_search_notes would return
zero results due to an early return when no query tokens were present.
This was counterintuitive - users expect an empty query to list all
notes, not return nothing.
Changes:
- Modified NotesSearchController.search_notes() to return all notes
when query is empty
- Added documentation to clarify this behavior
- Empty query results have _score: None (no relevance scoring)
- Non-empty query results continue to have relevance scores
Fixes behavior where listing all notes was impossible via the search tool.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Fixes#296
The application code was looking for OIDC_CLIENT_ID and OIDC_CLIENT_SECRET
(without NEXTCLOUD_ prefix), but the Helm chart, documentation, and CLI
all use NEXTCLOUD_OIDC_CLIENT_ID and NEXTCLOUD_OIDC_CLIENT_SECRET.
This mismatch caused OAuth deployments via Helm to fail with crashloops
because the credentials weren't being found.
Changes:
- app.py: Use NEXTCLOUD_OIDC_CLIENT_ID/SECRET in setup_oauth_config()
- config.py: Use NEXTCLOUD_OIDC_CLIENT_ID/SECRET in get_settings()
- Updated documentation comments and error messages
This aligns with the documented naming convention where all Nextcloud-related
environment variables use the NEXTCLOUD_ prefix.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Created @instrument_tool decorator for automatic MCP tool metrics collection.
Applied to all 7 tools in notes.py.
Changes:
- observability/metrics.py:
* New instrument_tool() decorator for automatic timing and error tracking
* Compatible with @mcp.tool() and @require_scopes() decorators
* Records tool_name, duration, and success/error status
- server/notes.py:
* Applied @instrument_tool to all 7 tool functions
* nc_notes_create_note, nc_notes_update_note, nc_notes_append_content
* nc_notes_search_notes, nc_notes_get_note, nc_notes_get_attachment
* nc_notes_delete_note
These metrics will populate the MCP Tool Calls dashboard panels.
Part of PR #295 - Complete metrics instrumentation (Phase 5)
Remaining: 86 tools across 8 server files
This ADR documents the architectural decision to support both OAuth and
AppAPI (ExApp) deployment modes in a single codebase with 90%+ code sharing.
Key additions:
- Comprehensive analysis of AppAPI limitations and challenges
- Feature parity matrix comparing OAuth vs AppAPI modes
- Resolution of critical open questions via research:
* Non-browser client authentication (app passwords/OAuth)
* Streaming transport compatibility (buffered, not real-time)
* Callbacks/webhooks (MCP notifications not possible in AppAPI)
- Detailed implementation plan with 4 phases (10 days)
- Mode-aware architecture with abstraction layer
Critical findings:
- AppAPI mode does NOT support MCP sampling (RAG features)
- No real-time progress updates (use Nextcloud notifications)
- Buffered streaming only (Streamable HTTP works, WebSocket doesn't)
- Requires app password support in AppAPI proxy
Deployment mode selection:
- OAuth: Multi-tenant, external clients, sampling/RAG, real-time updates
- AppAPI: Single-tenant, simplified install, native UI, admin-controlled
Related to investigation of ~/Software/app_api/ and ~/Software/nc_py_api/
for AppAPI integration patterns.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Fixes Kubernetes label validation error when deploying dashboard ConfigMap.
Problem:
- Kubernetes labels cannot contain spaces (validation regex: [A-Za-z0-9][-A-Za-z0-9_.]*[A-Za-z0-9])
- Previous implementation had grafana_folder: "Nextcloud MCP" as a label
- Deployment failed with: "Invalid value: 'Nextcloud MCP'"
Solution:
- Move grafana_folder from labels to annotations (annotations allow spaces)
- Keep grafana_dashboard="1" as label for ConfigMap discovery
- Grafana sidecar reads folder name from folderAnnotation parameter
Changes:
- dashboard-configmap.yaml: Move grafana_folder to annotations section
- dashboards/README.md: Fix kubectl commands to use annotations
- values.yaml: Update comments to clarify annotation usage
This follows the standard kube-prometheus-stack pattern where:
- Labels are used for ConfigMap discovery (strict validation)
- Annotations are used for metadata like folder names (relaxed validation)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This fixes dimension mismatch errors when using embedding models with
non-standard dimensions (e.g., qwen3-embedding:4b produces 2560-dim
vectors instead of the hardcoded 768).
Changes:
- OllamaEmbeddingProvider: Detect dimensions dynamically by generating
test embedding instead of hardcoding to 768
- qdrant_client: Call dimension detection before collection creation
- app.py: Initialize Qdrant collection before starting background tasks
in streamable-http transport path
- tests: Fix integration tests to properly mock EmbeddingService wrapper
Fixes dimension mismatch error:
"could not broadcast input array from shape (2560,) into shape (768,)"
All integration tests passing (6/6).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Fixes layout issues on the webhooks admin tab:
- Add min-height to container to fill viewport consistently
- Use CSS Grid to overlay tab panes without jumpiness
- Add smooth htmx fade transitions for content swaps
- Adjust vector sync polling interval from 3s to 10s
- Add .playwright-mcp/ to gitignore for test screenshots
The CSS Grid approach allows tabs to overlay without absolute positioning,
preventing content cutoff while maintaining smooth transitions without
container resizing jumps.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Implement real-time vector sync status updates in the /app UI without
requiring page refreshes. The status (indexed documents, pending
documents, sync state) now updates automatically every 3 seconds.
Changes:
- Add vector_sync_status_fragment() endpoint that returns HTML fragment
with current vector sync status
- Modify user_info_html() to use htmx loading for vector sync section
with hx-trigger="load" on initial render
- Status fragment includes hx-trigger="every 3s" for continuous polling
- Add /app/vector-sync/status route to browser_routes
The implementation uses htmx (already loaded on page) to poll the status
endpoint, providing near real-time updates with minimal overhead. The
endpoint queries Qdrant for indexed count and reads from memory streams
for pending count, returning only the status HTML fragment.
Pattern follows existing webhook management UI which also uses htmx
for dynamic loading.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Simplified the webapp routing structure by consolidating the admin UI
to a single clean endpoint.
Changes:
- Moved webapp from /user/page to /app (root of mount)
- Removed /user JSON endpoint (no longer needed)
- Updated mount point from /user to /app in app.py
- Updated all route path checks (3 locations)
- Updated OAuth redirects to point to /app
- Updated all HTMX endpoint references
- Updated documentation (ADR-007, CHANGELOG)
- Added redirect from /app to /app/ for trailing slash handling
New Route Structure:
- /app - Main webapp (HTML UI with tabs)
- /app/revoke - Revoke background access
- /app/webhooks - Webhook management UI
- /app/webhooks/enable/{preset_id} - Enable webhook preset
- /app/webhooks/disable/{preset_id} - Disable webhook preset
Breaking Change: Existing bookmarks to /user or /user/page will no longer work.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Refactored the storage system to use a unified SQLite database for both
webhook tracking and OAuth token storage, available in both BasicAuth
and OAuth modes.
Changes:
- Renamed refresh_token_storage.py → storage.py
- Made TOKEN_ENCRYPTION_KEY optional (only required for OAuth token ops)
- Added registered_webhooks table with schema versioning
- Added webhook storage methods (store, get, delete, list, clear)
- Initialize storage in both BasicAuth and OAuth modes
- Updated webhook routes to persist registrations in database
- Database-first pattern for webhook status checks (performance)
- Updated all imports across codebase
Storage Behavior:
- Database created automatically at startup if needed
- Existing databases detected and reused
- Server fails fast if database initialization fails
- No migrations needed (OAuth feature is experimental)
Testing:
- Added 13 comprehensive unit tests for webhook storage
- All 118 unit tests pass
- All 5 smoke tests pass
- Verified fail-fast behavior on initialization errors
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Manual testing of Nextcloud webhook_listeners app to validate webhook
payloads against ADR-010 expected schemas and document implementation
requirements for webhook-based vector synchronization.
## Changes
- Add test webhook endpoint at /webhooks/nextcloud in app.py
- Captures and logs webhook payloads for analysis
- Returns 200 OK immediately for webhook delivery confirmation
- Create webhook-testing-findings.md with comprehensive test results
- Captured payloads for 5/6 webhook event types
- Critical findings: missing node.id in deletions, type mismatches
- Implementation recommendations with code examples
- Update ADR-010 with Appendix A: Manual Webhook Testing Results
- Document actual vs expected webhook behavior
- Update event mapping table with tested webhook status
- Add 6 specific implementation recommendations
- Include testing implications for future development
## Testing Results
✅ NodeCreatedEvent - fires correctly, includes node.id (integer)
✅ NodeWrittenEvent - fires correctly, includes node.id (integer)
✅ NodeDeletedEvent - fires but missing node.id field (path only)
✅ CalendarObjectCreatedEvent - fires correctly with full iCal
✅ CalendarObjectUpdatedEvent - fires correctly with full iCal
❌ CalendarObjectDeletedEvent - does not fire (potential NC bug)
## Key Findings
1. NodeDeletedEvent missing node.id field - requires path-based fallback
2. node.id returns integer not string - needs casting for consistency
3. Multiple webhooks fire per operation - needs deduplication logic
4. Calendar deletion webhooks don't fire - reported as issue #53497
5. Calendar webhooks include full iCal content - enables rich parsing
## GitHub Issues
- Created issue #56371: NodeDeletedEvent missing node.id field
- Commented on issue #53497: CalendarObjectDeletedEvent not firing
Closes#283
---
_This commit was generated with the help of AI, and reviewed by a Human_
Simplifies the OpenTelemetry tracing setup by removing the redundant
OTEL_ENABLED flag and using the presence of OTEL_EXPORTER_OTLP_ENDPOINT
to determine if tracing should be enabled. This follows the standard
OpenTelemetry environment variable conventions more closely.
Changes:
- Remove OTEL_ENABLED/tracing_enabled flag in favor of checking if
OTEL_EXPORTER_OTLP_ENDPOINT is set
- Add OTEL_EXPORTER_VERIFY_SSL configuration option for OTLP endpoints
with self-signed certificates (defaults to false for development)
- Move HTTPXClientInstrumentor initialization to module level to ensure
httpx calls are traced across all Nextcloud API requests
- Add tracing spans to vector sync operations (scan_user_documents)
- Fix authorization header logging to only warn about missing headers
in OAuth mode (BasicAuth mode doesn't use Authorization headers)
- Update observability documentation to reflect simplified configuration
- Refactor Dockerfile to use --no-editable flag for uv sync
Breaking changes:
- OTEL_ENABLED environment variable is removed
- Tracing is now automatically enabled when OTEL_EXPORTER_OTLP_ENDPOINT
is set
Migration guide:
- Remove OTEL_ENABLED=true from environment configuration
- Tracing will be enabled automatically if OTEL_EXPORTER_OTLP_ENDPOINT
is configured
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
The test_attachments_category_change_handling test was failing in CI with
HTTP 412 Precondition Failed errors. This is caused by the background vector
scanner (runs every 10 seconds) modifying notes between when the test fetches
the ETag and when it attempts to update the category.
Solution: Added retry logic (up to 3 attempts) that refetches the latest ETag
and retries the update operation when encountering 412 errors. This handles
the race condition gracefully while still catching genuine errors.
Health check and metrics endpoints are frequently polled and don't
provide meaningful trace data. This change skips OpenTelemetry span
creation for:
- /health/* (liveness, readiness checks)
- /metrics (Prometheus metrics)
These endpoints still record Prometheus metrics (request count, latency,
in-flight requests) but no longer create trace spans, reducing tracing
noise and storage costs.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
The Nextcloud Notes API intentionally returns all note IDs (with only 'id'
field) in the last chunk to enable deletion detection. Without using the
pruneBefore parameter, this causes duplicates - all notes appear with full
data in chunks, then again with minimal data in the last chunk.
This commit implements proper pruneBefore support:
- NotesClient.get_all_notes() now accepts prune_before timestamp parameter
- Scanner calculates max(indexed_at) from Qdrant to use as prune threshold
- Only notes modified after this timestamp are sent with full data
- Deduplication logic handles the API's deletion detection pattern
- Significantly reduces data transfer for incremental syncs
The behavior is documented in Notes API v1 spec - this is not an API bug,
but a feature we weren't utilizing correctly.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Add architecture decision record for integrating Nextcloud webhooks
into the vector database synchronization system.
Key features:
- Webhook endpoint at /webhooks/nextcloud receives push notifications
- Complements existing polling (ADR-007) without replacing it
- Optional authentication via WEBHOOK_SECRET
- Simple architecture: webhooks are just another DocumentTask producer
- Administrators can reduce polling frequency when webhooks are configured
Benefits:
- Reduced latency: seconds to minutes instead of up to 1 hour
- Lower API load: ~95% reduction when polling frequency is increased
- Better scalability: only process changed documents
- No changes required to scanner or processor components
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Add support for configurable document chunking parameters to Helm chart
to match docker-compose and application capabilities.
Changes:
1. values.yaml:
- Add documentChunking section with chunkSize (512) and chunkOverlap (50)
- Include comprehensive comments explaining chunking strategies
- Positioned between vectorSync and qdrant sections
2. templates/deployment.yaml:
- Add DOCUMENT_CHUNK_SIZE and DOCUMENT_CHUNK_OVERLAP env vars
- Always set (not conditional), used by vector sync processor
- Environment variables follow same pattern as config.py defaults
3. README.md:
- Add documentChunking parameter table in Vector Search section
- Document chunking strategies (small/medium/large chunks)
- Explain overlap recommendations (10-20% of chunk size)
Validation:
- helm lint: Passes
- helm template: Environment variables correctly generated
- Custom values: Work as expected (tested with chunkSize=1024)
- Always present: Not conditional on vectorSync.enabled
This maintains feature parity between Helm and docker-compose deployments,
allowing users to tune chunking for their embedding models and use cases.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Changes to make tests work without external qdrant/ollama dependencies:
1. docker-compose.yml (mcp service):
- Switch from QDRANT_URL (network mode) to QDRANT_LOCATION=":memory:"
- Comment out QDRANT_URL and QDRANT_API_KEY (not needed for in-memory)
- Keep OLLAMA_BASE_URL commented out (use SimpleEmbeddingProvider fallback)
2. nextcloud_mcp_server/vector/qdrant_client.py:
- Fix collection creation bug in in-memory mode
- Previously: All ValueError exceptions were re-raised
- Now: Only dimension mismatch ValueError is re-raised
- Allows "Collection not found" ValueError to trigger auto-creation
3. tests/integration/test_sampling.py:
- Update test to handle all sampling unsupported cases
- Check for multiple fallback search_method values
- Skip test gracefully when sampling unavailable
This configuration enables:
- CI testing without external services (qdrant, ollama)
- In-memory vector database (ephemeral but sufficient for tests)
- SimpleEmbeddingProvider for embeddings (feature hashing, 384 dims)
- Automatic collection creation on first use
Test result: test_semantic_search_answer_successful_sampling now passes
(skipped with appropriate message when sampling unsupported)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This PR enables safe switching between embedding models and multi-server
deployments by implementing auto-generated Qdrant collection names based on
deployment ID and model name.
## Problem
Previously, all deployments used a single hardcoded collection name
"nextcloud_content", which caused two critical issues:
1. **Dimension mismatches when switching models**: Changing
OLLAMA_EMBEDDING_MODEL (e.g., nomic-embed-text at 768D → all-minilm at
384D) would cause runtime errors as vectors couldn't be inserted into a
collection with incompatible dimensions.
2. **Collection collisions in multi-server setups**: Multiple MCP servers
sharing a single Qdrant instance would overwrite each other's data,
making horizontal scaling impossible.
## Solution
### Auto-Generated Collection Naming
Collections are now automatically named using the pattern:
\`{deployment-id}-{model-name}\`
**Deployment ID**: Uses \`OTEL_SERVICE_NAME\` if configured (and not default
value), otherwise falls back to \`hostname\` for simple Docker deployments.
**Model Name**: From \`OLLAMA_EMBEDDING_MODEL\` with path separators sanitized.
**Examples**:
- \`my-mcp-server-nomic-embed-text\` (with OTEL_SERVICE_NAME=my-mcp-server)
- \`mcp-container-all-minilm\` (simple Docker, hostname=mcp-container)
**Override**: Users can still set \`QDRANT_COLLECTION\` explicitly to bypass
auto-generation for backward compatibility.
### Dimension Validation
Added startup validation that checks collection dimensions match the
embedding service. If a mismatch is detected, the server fails fast with a
clear error message explaining:
- Expected vs actual dimensions
- Likely cause (model change)
- Solutions (delete collection, use different name, or revert model)
### Improved Sampling Error Handling
Enhanced MCP sampling rejection handling to treat user rejections as normal
behavior rather than errors:
- **User rejections** ("rejected", "denied") → INFO log, no traceback
- **Unsupported clients** → INFO log, no traceback
- **Other MCP errors** → WARNING log, no traceback
- **Unexpected errors** → ERROR log WITH traceback
This aligns with the MCP specification where clients SHOULD prompt users for
approval/denial of sampling requests.
## Changes
### Core Implementation
- **nextcloud_mcp_server/config.py**: Added \`get_collection_name()\` method
with deployment ID detection and model name sanitization
- **nextcloud_mcp_server/vector/qdrant_client.py**: Dimension validation on
collection open with helpful error messages
- **nextcloud_mcp_server/vector/{scanner,processor}.py**: Updated to use
\`get_collection_name()\`
- **nextcloud_mcp_server/auth/userinfo_routes.py**: Vector sync status uses
\`get_collection_name()\`
- **nextcloud_mcp_server/server/semantic.py**:
- Updated semantic search tools to use \`get_collection_name()\`
- Improved sampling rejection error handling (McpError vs Exception)
### Documentation
- **docs/semantic-search-architecture.md**: New comprehensive architecture
document (557 lines) covering background sync, semantic search flow, RAG
implementation, and deployment modes
- **docs/configuration.md**: Added detailed "Qdrant Collection Naming"
section with examples and multi-server deployment guidance
- **docker-compose.yml**: Added comments explaining collection naming behavior
- **README.md**: Updated semantic search descriptions to clarify
experimental status, Notes-only support, and infrastructure requirements
## Migration Guide
**For existing single-server deployments:**
Option 1 (Recommended): Use explicit collection name for continuity
\`\`\`bash
QDRANT_COLLECTION=nextcloud_content # Keep existing collection
\`\`\`
Option 2: Allow auto-generation and re-embed
\`\`\`bash
# Remove QDRANT_COLLECTION override
# New collection will be created based on deployment ID + model
# Requires re-embedding all documents (may take time)
\`\`\`
**For new multi-server deployments:**
Set unique OTEL service names per server:
\`\`\`bash
# Server 1
OTEL_SERVICE_NAME=mcp-prod
OLLAMA_EMBEDDING_MODEL=nomic-embed-text
# → Collection: "mcp-prod-nomic-embed-text"
# Server 2
OTEL_SERVICE_NAME=mcp-staging
OLLAMA_EMBEDDING_MODEL=nomic-embed-text
# → Collection: "mcp-staging-nomic-embed-text"
\`\`\`
## Benefits
✅ **Safe model switching**: Each model gets its own collection, preventing
dimension mismatch errors
✅ **Multi-server support**: Multiple MCP servers can share one Qdrant
instance without conflicts
✅ **Clear ownership**: Collection names show which deployment and model owns
the data
✅ **Better error messages**: Dimension validation provides actionable
guidance
✅ **Backward compatible**: Existing deployments can continue using
\`QDRANT_COLLECTION\` override
## Testing
Validated with:
- Single-server deployments (default hostname-based naming)
- Multi-server deployments (OTEL service name-based naming)
- Model switching scenarios (dimension validation)
- Collection override scenarios (backward compatibility)
Next steps: Testing various Ollama embedding models to investigate optimal
chunk sizes and performance characteristics.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Security fix: Move Prometheus metrics endpoint from main HTTP port to
dedicated port 9090 to prevent external exposure of metrics data.
Changes:
- Use prometheus_client.start_http_server() for dedicated metrics server
- Remove /metrics route from main application routes
- Metrics now only accessible on port 9090 (configurable via METRICS_PORT)
- Main application port no longer serves /metrics endpoint
This follows security best practice of isolating monitoring endpoints
from application traffic.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
The readiness probe incorrectly tried to connect to an external Qdrant service
even when using memory or persistent mode (embedded Qdrant). This caused pods
to never become ready in Kubernetes deployments using the default configuration.
Root cause:
- In memory/persistent modes, QDRANT_URL env var is NOT set
- Readiness check used default 'http://qdrant:6333' anyway
- Tried to connect to non-existent service
- Connection failed -> 503 -> pod stuck in not-ready state
Fix:
- Only check external Qdrant health if QDRANT_URL is explicitly set (network mode)
- For embedded modes (memory/persistent), report status as 'embedded' without blocking
- Background scanner tasks don't block readiness (already non-blocking via anyio.start_soon)
This allows pods to become ready immediately when using embedded Qdrant,
while still validating external Qdrant connectivity in network mode.
Fixes: Kubernetes pods failing readiness check with default Qdrant configuration
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
The vector scanner crashed when encountering notes without a 'modified' field,
causing KeyError and preventing initial sync from completing.
Changes:
- Use dict.get() with fallback value (0) instead of direct key access
- Log warnings for notes missing 'modified' field
- Apply fix to both initial sync and incremental sync code paths
This ensures the scanner continues processing all notes even if some have
missing metadata fields, preventing scanner crashes that could affect
deployment readiness.
Fixes: Notes without 'modified' field causing scanner crash and readiness check failure
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add Prometheus metrics for HTTP, MCP tools, Nextcloud API, OAuth, vector sync, and DB operations
- Add OpenTelemetry distributed tracing with OTLP export
- Add structured JSON logging with trace context correlation
- Add ObservabilityMiddleware for automatic HTTP instrumentation
- Add app_name attribute to all client classes for per-app metrics
- Add configuration for metrics, tracing, and logging via environment variables
- Add documentation in docs/observability.md
- Fix graceful degradation when tracing is disabled (default state)
- Fix uvicorn logging configuration to use observability formatters
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
The Qdrant subchart was being included by default even in memory/persistent
modes. Changed the dependency condition from `qdrant.enabled` to
`qdrant.networkMode.deploySubchart` to align with the three-mode structure.
Now the Qdrant subchart is ONLY deployed when:
- qdrant.mode: "network"
- qdrant.networkMode.deploySubchart: true
Verified all three modes:
- Memory mode (:memory:): No subchart, QDRANT_LOCATION=:memory:
- Persistent mode (path): No subchart, QDRANT_LOCATION=/app/data/qdrant, PVC created
- Network mode (subchart): Qdrant subchart deployed, QDRANT_URL=http://...:6333
- Network mode (external): No subchart, QDRANT_URL=<external-url>
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
The chart-releaser was failing because it couldn't resolve the
dependencies (Qdrant and Ollama subcharts) when packaging.
Changes:
- Add azure/setup-helm action to install Helm v3.16.0
- Add step to add Qdrant and Ollama Helm repositories
- Run helm dependency update before chart-releaser runs
This fixes the error:
"Error: no repository definition for https://qdrant.github.io/qdrant-helm, https://otwld.github.io/ollama-helm"
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Add support for three Qdrant deployment modes in Helm chart:
1. In-memory mode (:memory:) - Default, zero-config, ephemeral storage
2. Persistent local mode (path-based) - File-based storage with PVC
3. Network mode (URL-based) - Dedicated Qdrant service or external instance
Changes:
- Restructured qdrant configuration in values.yaml with mode selector
- Added conditional environment variable logic in deployment.yaml
- Created PVC template for persistent local mode with optional existingClaim
- Added qdrantPvcName helper template in _helpers.tpl
- Updated README.md with Helm registry URL (https://cbcoutinho.github.io/nextcloud-mcp-server)
Breaking change: Default changed from requiring qdrant.enabled to using
in-memory mode (:memory:) when no Qdrant configuration is provided.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Adds flexible Qdrant deployment modes to reduce infrastructure requirements
for local development and smaller deployments:
**Configuration Changes:**
- Add QDRANT_LOCATION environment variable (mutually exclusive with QDRANT_URL)
- Three modes: network (URL), in-memory (:memory:, default), persistent (file path)
- Settings dataclass validation via __post_init__ ensures mutual exclusivity
- API key warning when set in local mode (ignored, only for network mode)
**Client Initialization:**
- Auto-detect mode: network (url + api_key) vs local (:memory: or path=)
- In-memory: AsyncQdrantClient(":memory:") - zero config default
- Persistent: AsyncQdrantClient(path="/app/data/qdrant") - file storage
- Network: AsyncQdrantClient(url, api_key) - production mode
**Docker Compose Updates:**
- Qdrant service moved to optional profile (--profile qdrant)
- MCP service uses QDRANT_LOCATION=:memory: by default
- Added mcp-data volume for persistent storage (/app/data)
- No hard dependency on qdrant service
**Documentation:**
- Comprehensive configuration guide in docs/configuration.md
- All three modes documented with pros/cons
- Docker Compose examples for each mode
- Environment variable reference table
**Tests:**
- 13 new config validation tests (mutual exclusivity, defaults, warnings)
- Persistent mode integration test (create, close, reopen, verify persistence)
- All 82 unit tests + 5 smoke tests pass
**Breaking Change:**
- Default changed from QDRANT_URL=http://qdrant:6333 to QDRANT_LOCATION=:memory:
- Simplifies local development (no external service needed)
- Production deployments: explicitly set QDRANT_URL or QDRANT_LOCATION
Related: ADR-007 background vector sync implementation
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Replace asyncio.Queue with anyio.create_memory_object_stream() throughout
the vector sync system for better library consistency and improved shutdown
semantics.
## Changes Made
**scanner.py**:
- Changed parameter type from `asyncio.Queue` to `MemoryObjectSendStream[DocumentTask]`
- Replaced all `await document_queue.put()` calls with `await send_stream.send()`
- Wrapped scanner loop in `async with send_stream:` context manager for automatic cleanup
- Updated log messages: "Queued" → "Sent"
- Removed `import asyncio` (no longer needed)
**processor.py**:
- Changed parameter type from `asyncio.Queue` to `MemoryObjectReceiveStream[DocumentTask]`
- Replaced `asyncio.wait_for(document_queue.get(), timeout=1.0)` with `anyio.fail_after(1.0)` + `await receive_stream.receive()`
- Removed all `document_queue.task_done()` calls (not needed with streams)
- Added `anyio.EndOfStream` exception handling for graceful shutdown when scanner closes
- Removed `import asyncio` (no longer needed)
**app.py**:
- Removed `import asyncio` from top-level imports
- Added `from anyio.streams.memory import MemoryObjectReceiveStream, MemoryObjectSendStream`
- Updated AppContext dataclass:
- Replaced `document_queue: Optional[asyncio.Queue]` with:
- `document_send_stream: Optional[MemoryObjectSendStream]`
- `document_receive_stream: Optional[MemoryObjectReceiveStream]`
- Updated `app_lifespan_basic()`:
- Replaced `asyncio.Queue(maxsize=...)` with `anyio.create_memory_object_stream(max_buffer_size=...)`
- Pass `send_stream` to scanner_task
- Pass `receive_stream.clone()` to each processor_task (enables multiple consumers)
- Updated AppContext yield to include both streams
- Updated `starlette_lifespan()`:
- Same changes as app_lifespan_basic for streamable-http transport
- Removed `import asyncio as asyncio_module` (no longer needed)
- Updated app.state storage to use send_stream and receive_stream
**semantic.py**:
- Updated `nc_get_vector_sync_status()` tool:
- Access `document_receive_stream` instead of `document_queue` from lifespan context
- Use `stream_stats.current_buffer_used` instead of `queue.qsize()` for pending count
- More reliable metrics (qsize() was not guaranteed accurate)
## Benefits
1. **Library Consistency**: Pure anyio throughout codebase (was mixing asyncio.Queue with anyio.Event and anyio.create_task_group)
2. **Graceful Shutdown**: `async with send_stream:` automatically closes stream on exit, signaling EndOfStream to all processors
3. **Better Timeout Handling**: `anyio.fail_after()` is more idiomatic than `asyncio.wait_for()`
4. **Stream Cloning**: Easy to add multiple consumers via `receive_stream.clone()`
5. **Better Statistics**: `.statistics()` provides accurate buffer metrics (qsize() was unreliable)
6. **Type Safety**: Separate send/receive types prevent accidental misuse
7. **No task_done() tracking**: Streams handle completion automatically
## Testing
- ✅ All 69 unit tests passing
- ✅ All 5 smoke tests passing
- ✅ No regressions in functionality
- ✅ Graceful shutdown behavior improved
## References
- https://anyio.readthedocs.io/en/stable/why.html#queue-fix
- https://anyio.readthedocs.io/en/stable/streams.html#memory-object-streams🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This implements ADR-009, which documents the decision to use a generic
`semantic:read` OAuth scope instead of requiring all app-specific scopes
for semantic search functionality.
Changes:
- Created new `nextcloud_mcp_server/models/semantic.py` with semantic search models
- SemanticSearchResult (with new doc_type field for multi-app support)
- SemanticSearchResponse
- SamplingSearchResponse
- VectorSyncStatusResponse
- Created new `nextcloud_mcp_server/server/semantic.py` with semantic search tools
- nc_semantic_search (renamed from nc_notes_semantic_search)
- nc_semantic_search_answer (renamed from nc_notes_semantic_search_answer)
- nc_get_vector_sync_status (renamed from nc_notes_get_vector_sync_status)
- All tools now use @require_scopes("semantic:read") instead of "notes:read"
- Updated `nextcloud_mcp_server/server/notes.py`
- Removed semantic search tools (moved to semantic.py)
- Removed semantic search model imports
- Removed unused MCP imports (ModelHint, ModelPreferences, etc.)
- Updated `nextcloud_mcp_server/models/notes.py`
- Removed semantic search models (moved to semantic.py)
- Updated `nextcloud_mcp_server/app.py`
- Import configure_semantic_tools
- Register semantic tools when VECTOR_SYNC_ENABLED=true
- Updated `nextcloud_mcp_server/server/__init__.py`
- Export configure_semantic_tools
- Updated tests
- tests/integration/test_sampling.py: Use new tool names
- tests/unit/test_response_models.py: Import from semantic.py, add doc_type field
Architecture:
- Semantic search is now a cross-app feature, not tied to Notes
- Uses dual-phase authorization: semantic:read scope + per-document verification
- Supports future multi-app indexing (notes, calendar, deck, files, contacts)
Test results:
- All 69 unit tests passing
- All 5 smoke tests passing
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Replace static VECTOR_SYNC_ENABLED_APPS environment variable with per-user
database storage for which apps to index. This allows each user to control
their own indexing preferences (e.g., enable notes and calendar but not
deck or files).
Rationale:
- Nextcloud doesn't support granular OAuth scopes at the app level
- Per-user settings provide flexibility for multi-user deployments
- Users control app enablement via nc_enable_vector_sync MCP tool
- Aligns with OAuth architecture where users manage their own settings
Changes:
- ADR-007: Remove VECTOR_SYNC_ENABLED_APPS from configuration section
- ADR-007: Update scanner implementation to read from database
- ADR-007: Add explanation of per-user app enablement mechanism
- ADR-007: Clarify that nc_enable_vector_sync tool manages this setting
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Update ADRs to reflect that vector database and semantic search support
multiple Nextcloud apps (notes, calendar, deck, files, contacts) rather
than being notes-specific. Introduce semantic:read/write OAuth scopes
to replace app-specific scope requirements for cross-app search.
Changes:
- ADR-007: Add plugin architecture (DocumentScanner, DocumentProcessor,
DocumentVerifier) for multi-app vector sync
- ADR-008: Rename tools from nc_notes_semantic_* to nc_semantic_*, update
scope from notes:read to semantic:read
- ADR-009: NEW - Document decision to use generic semantic:read scope
with dual-phase authorization instead of requiring all app scopes
- oauth-architecture.md: Add semantic:read/write scope documentation
- README.md: Move semantic search to dedicated section separate from Notes
This is a breaking change that correctly positions semantic search as a
cross-app capability before broader adoption. Existing deployments will
need to re-authenticate with the new semantic:read scope.
Relates to user request to decouple vector database from notes-only model
and establish proper OAuth scope boundaries for multi-app semantic search.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit addresses issues with vector database synchronization that
were causing test failures:
1. **Deletion Grace Period** (scanner.py)
- Fixed premature deletion of documents due to pagination cursor
inconsistencies in Notes API
- Implemented 2-scan verification with 1.5x scan interval grace period
(15 seconds default)
- Documents must be missing for 2 consecutive scans before deletion
- Documents that reappear are removed from deletion tracking
- Prevents false deletions during concurrent note creation/indexing
2. **Vector Sync Status Tool** (server/notes.py, models/notes.py)
- Added nc_notes_get_vector_sync_status MCP tool
- Returns indexed_count, pending_count, status, and enabled fields
- Enables tests and clients to wait for vector sync completion
- Uses lifespan context to access document queue and Qdrant client
3. **Test Improvements** (test_sampling.py, conftest.py)
- Added temporary_note_factory fixture for creating multiple test notes
- Updated all sampling tests to wait for vector sync completion
- Adjusted score_threshold to 0.0 for SimpleEmbeddingProvider
(feature hashing produces low-quality embeddings)
- Fixed CallToolResult extraction (removed ["result"] key access)
- Removed invalid @pytest.mark.asyncio markers (anyio mode)
All integration tests now pass successfully.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Add nc_notes_semantic_search_answer tool that combines semantic search
with MCP sampling to generate natural language answers from retrieved
Nextcloud Notes. This enables Retrieval-Augmented Generation (RAG)
patterns without requiring a server-side LLM.
Key features:
- Client-side LLM generation via ctx.session.create_message()
- Graceful fallback when sampling unavailable
- Proper source citations in generated answers
- No results optimization (skips sampling when no docs found)
- Comprehensive unit and integration tests
Implementation details:
- SamplingSearchResponse model with generated_answer and sources
- Fixed prompt template with document context and citation instructions
- Model preferences hint Claude Sonnet for balanced performance
- Falls back to returning documents without answer on sampling failure
Updates:
- Add ADR-008 documenting sampling architecture decision
- Add MCP sampling pattern guidance to CLAUDE.md
- Update README.md and docs/notes.md (7 → 9 tools)
- Add 4 unit tests and 6 integration tests
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Add support for deploying Qdrant vector database and Ollama embedding
service as optional helm chart dependencies. Enables semantic search
capabilities for Nextcloud content with flexible deployment options.
Chart Dependencies:
- Add Qdrant v0.9.0 from qdrant/qdrant-helm (conditional)
- Add Ollama v1.33.0 from otwld/ollama-helm (conditional)
- Both dependencies only deploy when enabled
Configuration (values.yaml):
- vectorSync: Background sync settings (interval, workers, queue size)
- qdrant: Subchart config with persistence, resources, clustering
- ollama: Subchart config with model pull, persistence, resources
- Support for external Ollama via ollama.url (no subchart deployment)
- openai: Alternative embedding provider (OpenAI or compatible API)
Environment Variables (deployment.yaml):
- VECTOR_SYNC_* variables when vectorSync.enabled
- QDRANT_URL, QDRANT_COLLECTION when qdrant.enabled
- OLLAMA_BASE_URL, OLLAMA_EMBEDDING_MODEL when ollama enabled or URL set
- OPENAI_API_KEY when openai.enabled
Documentation:
- README: New "Vector Search & Semantic Capabilities" section
- README: Example 5 showing three deployment patterns
- NOTES.txt: Conditional guidance when vector features enabled
- Secret template for OpenAI API key management
All features disabled by default for backward compatibility.
Tested with helm template and helm lint.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Add real-time processing status display to the browser UI at /user/page
showing indexed document count, pending queue size, and sync status.
Implements the status display described in ADR-007 lines 280-298.
Changes:
- Store document_queue and related state in app.state for route access
- Add _get_processing_status() helper to query Qdrant and check queue
- Display status section in user_info_html() with indexed/pending counts
- Show color-coded status badge (green "Idle" or orange "Syncing")
- Only displays when VECTOR_SYNC_ENABLED=true
Status appears in both BasicAuth and OAuth modes, positioned after
session info but before logout buttons. Numbers are formatted with
commas for readability.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Replace deprecated qdrant_client.search() with query_points() API
- Update semantic search implementation in notes.py
- Update all integration tests to use query_points()
- Fix Keycloak login in test_keycloak_dcr.py to use form.submit() instead of button click
- Remove unnecessary popup handler code
- Simplify consent screen logging
The urllib3<2.0 constraint was added unnecessarily during troubleshooting.
urllib3 2.x works perfectly fine with qdrant-client. The import path for
urllib3.util.Url and parse_url remains the same across 1.x and 2.x versions.
Changes:
- Remove urllib3<2.0 constraint from pyproject.toml
- Upgrade to urllib3 2.5.0 (latest)
- All integration tests pass with urllib3 2.x
Verified:
- from urllib3.util import Url, parse_url works in 2.5.0
- All 6 semantic search integration tests pass
- qdrant-client 1.15.1 works correctly with urllib3 2.5.0
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Adds comprehensive integration tests for vector database semantic search that
work without external dependencies (Ollama), making them suitable for CI/CD.
Changes:
- Add SimpleEmbeddingProvider: in-process TF-IDF-like embeddings using feature hashing
- Make Ollama optional: embedding service now falls back to SimpleEmbeddingProvider
- Add 6 integration tests covering semantic search, filtering, and batch operations
- Downgrade urllib3 to 1.26.x for qdrant-client compatibility
- Update docker-compose.yml to comment out Ollama configuration (optional)
The SimpleEmbeddingProvider generates deterministic, normalized embeddings
suitable for testing semantic similarity without requiring external services.
Tests validate that similar texts have higher cosine similarity and that
semantic search correctly ranks results by relevance.
Test coverage:
- Deterministic embedding generation
- Semantic similarity between texts
- Full search flow with Qdrant (in-memory)
- Category filtering
- Empty result handling
- Batch embedding generation
All tests pass and can run in GitHub CI without Ollama infrastructure.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Completes the ADR-007 implementation by adding user-facing semantic search
functionality. Previous phases implemented scanner and processor for background
indexing; this adds the query interface.
Changes:
- Add nc_notes_semantic_search MCP tool for natural language queries
- Fix Qdrant point IDs to use UUIDs instead of strings (was causing 400 errors)
- Reduce scan interval default from 1 hour to 5 minutes for faster updates
- Add SemanticSearchResult and SemanticSearchNotesResponse models
- Implement dual-phase authorization (Qdrant filter + Nextcloud API verification)
The semantic search enables finding notes by meaning rather than exact keywords,
using vector embeddings to understand query intent. Point ID fix resolves
critical bug where all document indexing failed with "invalid point ID" errors.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Fixes background task startup for streamable-http transport by integrating
vector sync initialization into the Starlette lifespan context manager.
Starlette Lifespan Integration:
- Moved background task startup from FastMCP lifespan to Starlette lifespan
- FastMCP lifespan only triggers on MCP session establishment
- Starlette lifespan runs on server startup (correct timing)
- Fixed module scoping issues with local imports (anyio_module, asyncio_module)
- Added conditional startup based on oauth_enabled flag
Scanner Fixes:
- Fixed NotesClient method: list_notes() → get_all_notes()
- Properly handle AsyncIterator with list comprehension
- Collects all notes before processing
Verified Working:
- Background tasks start successfully on server startup
- Scanner fetches notes from Nextcloud API
- Processor pool (3 workers) ready for document processing
- Health endpoint reports Qdrant status
- No startup errors
Phase 3 Complete:
- BasicAuth mode with vector sync fully functional
- Background tasks integrate cleanly with streamable-http transport
- Graceful shutdown with coordinated task cancellation
Related: ADR-007 Background Vector Database Synchronization
🤖 Generated with Claude Code (https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Implements background vector database synchronization using anyio
TaskGroups for BasicAuth mode with single-user credentials.
Scanner Implementation:
- Periodic document discovery (hourly, configurable)
- Timestamp-based change detection (Nextcloud vs Qdrant)
- Wake event for immediate scanning on-demand
- Supports both initial sync (all docs) and incremental sync (changes only)
- Detects deleted documents and queues for removal
Processor Implementation:
- Concurrent document processing pool (3 workers default)
- I/O-bound embedding generation via Ollama API
- Retry logic with exponential backoff (3 retries)
- Document chunking (512 words, 50-word overlap)
- Handles both index and delete operations
- Upserts vectors to Qdrant with rich metadata
App Lifespan Integration:
- Extended AppContext with background task state
- Modified app_lifespan_basic() to start tasks via anyio TaskGroups
- Graceful shutdown with coordinated task cancellation
- Only activates when VECTOR_SYNC_ENABLED=true
Embedding Service:
- OllamaEmbeddingProvider with TLS support
- Singleton pattern for shared client instances
- Batch embedding support for efficiency
- Auto-detects embedding dimension (768 for nomic-embed-text)
Qdrant Client:
- Async client wrapper with singleton pattern
- Auto-creates collection on first use
- COSINE distance metric for semantic similarity
- Integrates with embedding service for dimension detection
Health Check Enhancement:
- Added Qdrant status check to /health/ready endpoint
- Only checks when VECTOR_SYNC_ENABLED=true
- 2-second timeout for health probe
- Reports connection errors with details
Configuration:
- VECTOR_SYNC_ENABLED: Enable background sync
- VECTOR_SYNC_SCAN_INTERVAL: Scanner frequency (3600s default)
- VECTOR_SYNC_PROCESSOR_WORKERS: Concurrent processors (3 default)
- QDRANT_URL, QDRANT_API_KEY, QDRANT_COLLECTION: Vector DB config
- OLLAMA_BASE_URL, OLLAMA_EMBEDDING_MODEL: Embedding service config
Dependencies Added:
- qdrant-client>=1.7.0: Vector database client
Docker Compose:
- Added Qdrant service with health check
- Exposed ports 6333 (REST) and 6334 (gRPC)
- Configured MCP service with vector sync environment
- Added qdrant-data volume for persistence
Known Issue:
- FastMCP lifespan not triggering for streamable-http transport
- Background tasks will start once lifespan integration is complete
- Lifespan triggers on MCP session establishment, not server startup
Related: ADR-007 Background Vector Database Synchronization
🤖 Generated with Claude Code (https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Add comprehensive ADR-007 documenting background vector database
synchronization architecture using anyio TaskGroups for in-process
concurrency. This supersedes ADR-003's conceptual background worker.
Key decisions:
- In-process architecture using anyio TaskGroups (not Celery)
- Scanner task runs hourly, detects changes via timestamp comparison
- In-memory asyncio.Queue for pending documents
- Pool of 3 concurrent processor tasks for I/O-bound embedding workloads
- Qdrant metadata as single source of truth for indexing state
- Simple user controls: enable/disable with status visibility
Benefits:
- Single container deployment (was 3: mcp, celery-worker, celery-beat)
- No distributed task queue infrastructure
- Shared process state (no volume coordination)
- Sufficient throughput for I/O-bound embedding APIs
- Simpler debugging and deployment
Update ADR-003 status to "Superseded by ADR-007" with reference link.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Improve debugging capabilities for OAuth flow in elicitation callback:
- Add detailed logging for consent screen handling
- Capture screenshots when consent screen not detected or fails
- Replace networkidle wait with explicit callback URL polling
- Add 2-second grace period for server-side callback processing
- Log page title and current URL for debugging
This helps diagnose issues like expired OAuth clients or authorization failures during real elicitation testing.
Test results: All 4 elicitation integration tests passing
- test_check_logged_in_with_real_elicitation_callback ✓
- test_elicitation_callback_url_extraction ✓
- test_elicitation_stores_refresh_token ✓
- test_second_check_logged_in_does_not_elicit ✓
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit adds proper integration testing of the login elicitation flow
(ADR-006) using python-sdk's MCP client with actual elicitation callback
support, and fixes user_id extraction to support both JWT and opaque tokens.
## Changes
### 1. Enhanced create_mcp_client_session helper (tests/conftest.py)
- Added `elicitation_callback` parameter to function signature
- Pass callback to ClientSession constructor
- Added necessary imports: RequestContext, ElicitRequestParams,
ElicitResult, ErrorData from mcp package
- Allows fixtures to provide custom elicitation handlers
### 2. New fixture: nc_mcp_oauth_client_with_elicitation (tests/conftest.py)
- Creates MCP client with Playwright-based elicitation callback
- Callback implementation:
- Extracts OAuth URL from elicitation message using regex
- Uses Playwright browser to complete OAuth flow automatically
- Handles Nextcloud login form (username/password)
- Handles consent screen if present
- Waits for OAuth callback completion
- Returns ElicitResult(action="accept") on success
- Function-scoped to allow independent test state
- Tracks elicitation invocations via session.elicitation_triggered
### 3. Fixed user_id extraction for opaque tokens (oauth_tools.py)
- Created extract_user_id_from_token() helper to handle both JWT and
opaque tokens by calling userinfo endpoint when needed
- Fixed check_logged_in to use helper instead of broken ctx.authorization
- Fixed revoke_nextcloud_access to use helper instead of ctx.context.get()
- Both tools now properly extract user_id from access tokens
### 4. Enhanced integration tests (test_elicitation_integration.py)
- Updated tests to revoke refresh tokens via MCP tool
- All 4 tests now pass:
- test_check_logged_in_with_real_elicitation_callback: Complete flow
- test_elicitation_callback_url_extraction: URL extraction validation
- test_elicitation_stores_refresh_token: Token persistence verification
- test_second_check_logged_in_does_not_elicit: No redundant elicitations
### 5. Added diagnostic logging (oauth_routes.py)
- Track user_id extraction from ID tokens during OAuth callbacks
- Log refresh token storage with user_id and flow_type
## Test Results
✅ 4/4 tests pass
The test suite successfully validates:
- Elicitation callback is triggered when no refresh token exists
- Playwright automation completes OAuth flow
- Refresh token is stored after OAuth with correct user_id
- Tool returns "yes" after successful login
- Already-logged-in users don't get redundant elicitations
## Why This Matters
Previous tests (test_login_elicitation.py) only validated response
formats and acknowledged they couldn't test real elicitation protocol.
This test exercises the REAL MCP elicitation protocol end-to-end:
1. MCP server calls ctx.elicit()
2. python-sdk ClientSession invokes custom callback
3. Callback completes OAuth via Playwright
4. Client returns acceptance to server
5. Tool proceeds with authenticated state
This proves the python-sdk MCP client can handle elicitation in
production environments with both JWT and opaque tokens.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit addresses the "Login not detected" issue after completing
OAuth login via elicitation by unifying the session architecture and
adding comprehensive visibility into background session status.
## Changes
### 1. Enhanced check_logged_in with comprehensive logging (oauth_tools.py)
- Added detailed logging at each step of token lookup
- Implemented fallback strategy: first search by provisioning_client_id,
then fall back to user_id lookup
- This allows detection of refresh tokens created via any flow
(elicitation or browser login)
- Log messages include flow_type, provisioned_at, and provisioning_client_id
for debugging
### 2. Unified session architecture (browser_oauth_routes.py)
- Browser login now stores provisioning_client_id=state when saving
refresh token
- This makes browser and elicitation flows consistent - both can be
found by the same state parameter
- Treats Flow 2 (elicitation) and browser login as the same "background
session"
### 3. Enhanced /user/page with session status (userinfo_routes.py)
- Added comprehensive background access section showing:
- Background Access: Granted/Not Granted (with visual indicators)
- Flow Type: browser/flow2/hybrid
- Provisioned At: timestamp
- Token Audience: nextcloud/mcp
- Scopes: detailed scope list
- Status displayed regardless of which flow created the session
(browser login or elicitation)
### 4. Added revoke functionality (userinfo_routes.py, app.py)
- New POST endpoint: /user/revoke
- Allows users to revoke background access (delete refresh token)
- Browser session cookie remains valid for UI access
- Confirmation dialog before revocation
- Success page with auto-redirect back to /user/page
- Registered route in app.py browser_routes
## Testing
All tests pass:
- 6/6 login elicitation tests pass
- 21/21 core OAuth tests pass
- Comprehensive logging helps debug future issues
## Fixes
Resolves: "Login not detected. Please ensure you completed the login
at the provided URL before clicking OK."
The issue occurred because elicitation and browser login created
separate sessions. Now they are unified under the same architecture.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Enhanced test suite to validate:
1. Elicitation URL format and Flow 2 endpoint routing
2. Server-side refresh token validation via check_provisioning_status API
3. Proper separation of concerns - tests use MCP server API, not direct storage access
The refresh token validation test validates server responses:
- is_provisioned=true: Server has valid refresh token
- is_provisioned=false: No token or invalid token
- Error response: Token validation failed
This ensures the MCP server properly validates refresh tokens internally
and reports status correctly through its public API.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This PR fixes multiple OAuth-related issues:
## Unified OAuth Callback
- Consolidated `/oauth/callback-nextcloud` and `/oauth/login-callback` into single `/oauth/callback` endpoint
- Flow type determined by session lookup via state parameter (no query params in redirect_uri)
- Fixes redirect_uri validation issues with IdPs requiring exact match
- Legacy endpoints kept as aliases for backwards compatibility
## PKCE Implementation
- Implemented PKCE (RFC 7636) for Flow 2 (resource provisioning)
- Generate code_verifier and code_challenge
- Store code_verifier in session storage
- Retrieve and use in token exchange
- Fixed PKCE for browser login (integrated mode)
- Previously only worked for external IdP (Keycloak)
- Now works for both Nextcloud OIDC and external IdP
## Login Elicitation Fixes (ADR-006)
- Fixed elicitation URL to route through MCP server endpoint
- Changed from direct Nextcloud URL to `/oauth/authorize-nextcloud`
- Ensures PKCE is properly handled by server
- Fixed login detection after OAuth flow completes
- Look up refresh token by state parameter instead of user_id
- Works even when Flow 1 token not present
- Added `get_refresh_token_by_provisioning_client_id()` method
## Session Authentication
- Fixed `/user/page` redirect loop
- Shared oauth_context with mounted browser_app
- SessionAuthBackend can now validate sessions correctly
## Tests
- Added comprehensive login elicitation test suite
- Updated scope authorization test expectations
- All 43 OAuth tests passing
## Files Changed
- `app.py`: Shared oauth_context, unified callback route
- `oauth_routes.py`: Unified callback, PKCE for Flow 2
- `browser_oauth_routes.py`: PKCE for integrated mode
- `oauth_tools.py`: Fixed elicitation URL generation
- `refresh_token_storage.py`: Added lookup by provisioning_client_id
- `test_login_elicitation.py`: New test suite
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit completes the OAuth audience validation implementation per RFC 7519,
RFC 8707 (Resource Indicators), and RFC 9728 (Protected Resource Metadata).
## Key Changes
### OAuth Resource Parameters (RFC 8707)
- Add `resource` parameter to Flow 1 (MCP client auth) with MCP server audience
- Add `resource` parameter to Flow 2 (Nextcloud access) with Nextcloud audience
- Add `nextcloud_resource_uri` to oauth_context configuration
- Fix undefined variable error in starlette_lifespan
### PRM-Based Resource Discovery (RFC 9728)
- Update tests to fetch resource identifier from PRM endpoint
- Add fallback to hardcoded value if PRM fetch fails
- Demonstrate correct OAuth client implementation pattern
### ADR-005 Documentation Updates
- Update to reflect simplified RFC 7519 compliant implementation
- Document that MCP validates only its own audience (not Nextcloud's)
- Add section on OAuth resource parameters and PRM discovery
- Update implementation checklist to show completed items
- Mark status as "Implemented" with update date
## Implementation Details
The solution follows RFC 7519 Section 4.1.3: resource servers validate only
their own presence in the audience claim. This simplifies the logic while
maintaining security:
- MCP server validates MCP audience only
- Nextcloud independently validates its own audience
- No dual validation required at MCP layer
- Token reuse is allowed per RFC 8707 for multi-audience tokens
## Test Results
✅ test_mcp_oauth_server_connection - PASSED
✅ test_deck_board_view_permissions - PASSED
✅ test_prm_endpoint - PASSED
All OAuth flows now properly specify target resources, resulting in correct
audience validation throughout the system.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Since both multi-audience and exchange modes now validate the same thing
(MCP audience only per RFC 7519), consolidated the duplicate methods:
- Removed duplicate verification methods (_verify_multi_audience_token
and _verify_mcp_audience_only)
- Created single _verify_mcp_audience() method for all validation
- Removed duplicate helper (_validate_multi_audience), kept _has_mcp_audience
- Mode only affects logging and what happens AFTER verification
The mode distinction is now purely about post-verification behavior:
- Multi-audience mode: Use token directly (Nextcloud validates its own)
- Exchange mode: Exchange for Nextcloud-audience token via RFC 8693
This makes the code cleaner and clearer about what's actually happening -
both modes do identical validation, they just differ in how the validated
token is used.
All tests pass: unit (65), OAuth integration confirmed working.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Per RFC 7519 Section 4.1.3, resource servers should only validate their
own presence in the audience claim, not check for other resource servers.
Changes:
- UnifiedTokenVerifier now validates only MCP audience (not Nextcloud's)
- Nextcloud independently validates its own audience when receiving API calls
- This is NOT token passthrough (we validate tokens before use)
- This IS token reuse which is explicitly allowed by RFC 8707
Updates:
- Simplified _validate_multi_audience() to follow OAuth spec
- Updated docstrings and comments to clarify RFC 7519 compliance
- Fixed unit tests that expected dual-audience validation
- Updated ADR-005 to document the correct OAuth interpretation
- All tests pass: unit (65), smoke (5), OAuth integration
This makes the implementation simpler, more maintainable, and properly
aligned with OAuth 2.0 specifications while maintaining security.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Fix external IdP token exchange by using the correct audience identifier
for Keycloak.
Keycloak uses client IDs as audience identifiers, not URLs. The token
exchange was failing with "Audience not found" because it was requesting
audience "http://localhost:8080" but Keycloak only knows about the
"nextcloud" client ID.
Changes:
- Update mcp-keycloak service NEXTCLOUD_RESOURCE_URI from
"http://localhost:8080" to "nextcloud"
- Matches Keycloak's client ID convention for resource identifiers
- Token exchange now requests audience "nextcloud" which matches the
Keycloak resource server client configuration
Note: mcp-oauth service keeps URL-based resource URI because Nextcloud's
integrated OIDC app expects URLs, not client IDs. Different IdPs have
different conventions for audience/resource identifiers.
Test result: test_external_idp_token_validation now passes
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Fix two issues preventing OAuth tests from passing:
1. Set oidc_client_id and oidc_client_secret on Settings object
- These were being read from environment but not propagated to the
UnifiedTokenVerifier settings instance
2. Use client_issuer instead of issuer for JWT validation
- client_issuer accounts for NEXTCLOUD_PUBLIC_ISSUER_URL override
- Fixes "Invalid issuer" errors when public URL differs from internal
3. Accept resource URL with /mcp path in audience validation
- During DCR, resource_url is registered as "{mcp_server_url}/mcp"
- Tokens correctly include this full path as audience
- Verifier now accepts both "http://localhost:8001" and
"http://localhost:8001/mcp" as valid MCP audiences
These changes restore OAuth functionality while maintaining ADR-005
security requirements for proper audience validation.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Replace two non-compliant token verifiers (NextcloudTokenVerifier and
ProgressiveConsentTokenVerifier) with a single UnifiedTokenVerifier that properly
validates token audiences per MCP Security Best Practices specification.
The previous implementation had a critical security vulnerability where tokens
intended for the MCP server were passed directly to Nextcloud APIs without
proper audience validation (token passthrough anti-pattern). This violates
OAuth 2.0 security principles and the MCP specification.
Changes:
- Add UnifiedTokenVerifier supporting two compliant modes:
* Multi-audience mode (default): Validates tokens contain BOTH MCP and
Nextcloud audiences, enabling direct use without exchange
* Token exchange mode (opt-in): Validates MCP audience only, exchanges
for Nextcloud tokens via RFC 8693 with caching to minimize latency
- Remove token passthrough vulnerability from context.py and context_helper.py
- Implement token exchange caching (5-minute TTL default) to reduce network calls
- Add required environment variables for audience validation:
* NEXTCLOUD_MCP_SERVER_URL - MCP server URL (used as audience)
* NEXTCLOUD_RESOURCE_URI - Nextcloud resource identifier
* TOKEN_EXCHANGE_CACHE_TTL - Cache TTL for exchanged tokens
- Update docker-compose.yml with resource URI configuration for both OAuth modes
- Add comprehensive test suite (29 tests) covering both authentication modes
- Remove legacy NextcloudTokenVerifier and ProgressiveConsentTokenVerifier
Security improvements:
- Eliminates token passthrough anti-pattern
- Enforces proper audience separation between MCP and Nextcloud
- Complies with MCP Security Best Practices and RFC 8707/8693
- Maintains performance with token exchange caching
Test results: 65/65 unit tests passed, 5/5 smoke tests passed
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This ADR addresses the critical token passthrough vulnerability identified
in Issue #261 by proposing a unified token verifier that eliminates the
security issue while maintaining flexibility.
Key changes:
- Consolidates two non-compliant verifiers into single UnifiedTokenVerifier
- Implements two-layer architecture (verification + exchange)
- Supports multi-audience mode (default) and token exchange mode (opt-in)
- Removes all token passthrough paths to comply with MCP security spec
- Works within python-sdk constraints using proper separation of concerns
The solution provides:
- Single source of truth for token validation
- MCP specification compliance
- Minimal performance impact (1-2% of LLM request time)
- Clear migration path for existing deployments
BREAKING CHANGE: All OAuth deployments must be reconfigured to specify
resource URIs (NEXTCLOUD_MCP_SERVER_URL and NEXTCLOUD_RESOURCE_URI) and
choose between multi-audience or token exchange mode.
Related: #261
Supersedes: Token passthrough mode in ADR-004
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Fix nc_get_capabilities resource handler that was missing await when
calling get_nextcloud_client(ctx), causing error:
'coroutine' object has no attribute 'capabilities'
Root cause:
- get_nextcloud_client() is an async function (context.py:9)
- Returns a coroutine that must be awaited
- app.py:737 called it without await
Solution:
- Add await: client = await get_nextcloud_client(ctx)
- The handler is already async, so can await the call
Test fixed:
- test_mcp_resources_access now passes
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Fix three tests in test_token_exchange.py that were using invalid
Fernet encryption keys (b"test-key-" + b"0" * 32), causing ValueError
due to invalid base64 encoding.
Root cause:
- Tests manually created invalid Fernet keys
- token_storage and token_broker fixtures generated different keys
- Encryption/decryption operations failed due to key mismatch
Solution:
- Expose valid encryption key from token_storage fixture via _test_encryption_key
- Update token_broker fixture to use same encryption key from token_storage
- Update all tests to use token_storage._test_encryption_key
Tests fixed:
- test_get_background_token
- test_session_background_separation
- test_background_token_different_scopes
All 13 tests in test_token_exchange.py now pass.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Chris Coutinho