- Cap all_results to limit*2 after sort in the per-doc_types branch of
nc_semantic_search to bound over-verification (was unbounded N-types).
- Switch BatchVerifier from (client, doc_ids, user_id) to (client, results,
semaphore). Verifiers now read file paths and deck board/stack ids from
SearchResult.metadata instead of doing fresh Qdrant scrolls — eliminates
one duplicate round-trip per file/deck-card verification.
- Bound per-id verification concurrency with a shared anyio.Semaphore
(default 20, matching server/semantic.py context-expansion convention).
Prevents httpx pool exhaustion / rate limiting on large search pages.
- Propagate stack_id from Qdrant payload to SearchResult.metadata in both
bm25_hybrid.py and semantic.py (board_id was already propagated).
- Drop now-unused _resolve_file_path / _resolve_deck_metadata helpers.
- Drop redundant int(d) in requested predicate from _verify_news_items.
- Rewrite eviction comment to be honest about inline (not background)
execution and the resulting latency coupling.
- ADR-019 status: Proposed -> Accepted.
- Add news property to NextcloudClientProtocol.
- Widen SearchResult.id and SemanticSearchResult.id to int | str to match
BatchVerifier signature and document support for future string-id types.
- Flip openWorldHint to True on nc_semantic_search_answer (it calls into
Nextcloud via nc_semantic_search).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes the seven outstanding items from the @claude review on PR #741:
1. Add empty `tests/client/talk/__init__.py` for pytest discovery parity
with `tests/client/{collectives,news}/`.
2. Standardise boolean query params to integers — `includeStatus` was the
string `"true"` in `list_conversations`/`list_participants` while every
other flag (`noStatusUpdate`, `lookIntoFuture`, `setReadMarker`,
`includeLastKnown`) used `1`/`0`.
3. Replace the `app:install || app:enable` chain in the spreed install hook
with `app:install --keep-disabled --force || true; app:enable spreed`,
so unrelated install failures surface as a clear "app not found" from
`app:enable` rather than being silently masked.
4. Add `_validate_token()` (alphanumeric whitelist) and call it from all
six TalkClient methods that interpolate the token into a URL path —
defence-in-depth against pathological tokens reaching httpx.
5. Rename `TalkConversation.type` to `room_type` with `Field(alias="type")`
and `populate_by_name=True`, so the field no longer shadows Python's
builtin while preserving spreed's wire format on input. MCP responses
now serialize `room_type` (field name) instead of `type`.
6. `mark_as_read` now passes `json=body or None` so the bodyless
"mark everything as read" call doesn't send a spurious `{}` body and
`Content-Type: application/json` header.
7. `_validate_message_text` rejects whitespace-only messages, not just
empty strings.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Comment-only follow-up to surface non-obvious behavior at the call
sites flagged in review:
- server/talk.py: note the `uuid.uuid4().hex` 32-char no-dashes format
(spreed accepts either form).
- models/talk.py: warn that spreed returns `lastReadMessage: 0` rather
than `null` for unread rooms, so consumers should compare to ``None``
rather than rely on truthiness.
- 10-install-spreed-app.sh: document that the `app:install || app:enable`
fallback also masks unrelated install failures, and limit its use to
dev fixtures.
No runtime behavior changes; tests unchanged (still 13 unit + 7 integ).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Four targeted fixes from the AI code review:
1. TalkConversation.description: drop the misleading `str | None`
union (spreed always sends `""`, never null) — type is now `str`
with default `""`.
2. get_messages: guard the X-Chat-Last-Given int parse with
try/except so a misbehaving proxy can't crash the read flow;
logs a warning and falls back to None.
3. get_messages: clamp `limit` to [1, 200] in the client (spreed
caps server-side at 200 and silently truncates) so the returned
`count` always matches what was actually requested. Both client
and server-tool docstrings updated to state the valid range.
4. Add an integration test covering the 32000-char message ceiling
in talk_send_message — the empty-message case was already tested,
the over-length case was not.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds 6 MCP tools so an LLM can read a user's Talk conversations and
post messages on their behalf, addressing the "read my chats and reply"
use case from issue #720:
- talk_list_conversations
- talk_get_conversation
- talk_get_messages
- talk_list_participants
- talk_send_message (auto-attaches a referenceId for retry dedup)
- talk_mark_as_read
Edit/delete messages, reactions, threads, and call/session ops are
intentionally out of scope for this first PR.
The TalkClient also exposes create_conversation/delete_conversation
for the integration test fixture; these are not registered as MCP
tools. A post-installation hook enables spreed in the docker dev env
so the integration suite has a real Talk backend to talk to.
Closes#720
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Wrap raw DeckComment returns in CardCommentResponse(BaseResponse) for
create/update so the success/timestamp envelope matches other deck tools
(#737 review issue 2).
- Rename ListCardCommentsResponse.total → count and clarify in the
description that it's the page size, not a server-side total — the Deck
list endpoint does not expose one (#737 review issue 3).
- Validate the documented 1000-character limit on create/update with an
inline length check + ValueError, matching the pattern in
api/management.py (#737 review issue 4).
- Use modern int | None union syntax for the new parent_id parameter
(#737 review issue 1); rest of the file is left in the existing
Optional[...] style.
Also add an MCP-level test that the >1000 char message is rejected, and
update the existing comment tests to unwrap the new comment field.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Expose four new MCP tools backed by existing DeckClient comment methods:
- deck_get_card_comments — list with limit/offset pagination
- deck_create_card_comment — top-level or threaded (via parent_id)
- deck_update_card_comment — author-only on the server
- deck_delete_card_comment — author-only, destructive, idempotent
Adds ListCardCommentsResponse and CardCommentOperationResponse models, and
extends the client unit tests to cover replies, deletion, pagination, and
the request shape for updates.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two upstream Pydantic ValidationErrors that took down whole list responses.
#704: Contact.birthday is declared str, but vobject parses BDAY as a
datetime.date — any contact with a populated BDAY broke nc_contacts_list_contacts
entirely. Add a field_validator(mode="before") that coerces date / datetime
to ISO strings. Strings and None pass through unchanged. Defense in depth:
existing call sites already coerce, but the model is now correct on its own
so any future code path that constructs Contact from raw vobject output
stays safe.
#728: Tables app v2.0.1 stopped emitting owner_display_name on the top-level
table payload (still present inside views via get_schema), so list_tables
failed for every user with a Pydantic ValidationError. Make the field
Optional[str] = None — captures the value when present, won't blow up when
missing.
Six new direct-construction unit tests in tests/unit/test_response_models.py
pin both fixes (date / datetime / str / None for birthday; with / without
owner_display_name for Table) so the regressions can't recur silently.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Many identity providers (AWS Cognito, Okta, Azure AD) reject or mishandle
colons in OAuth scope names. This migrates all custom scopes from
`resource:action` to `resource.action` format (e.g., `notes:read` →
`notes.read`), which is universally accepted and aligns with industry
conventions (Microsoft, Google).
Includes Alembic migration 004 for stored scope strings and ADR-024
documenting the rationale and RFC references.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Fix emoji clearing bug: use _UNSET sentinel in update_collective so
emoji=None sends {"emoji": null} instead of raising ValueError
- Move collectives_get_trashed_collectives to Read Tools section
- Remove redundant is_trash field from ListTrashedPagesResponse
- Add page lifecycle note to collectives_trash_page docstring
- Add unit test for clearing collective emoji via update_collective
- Add integration test for clearing collective emoji via MCP tool
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Fix inconsistent error code in set_collective_emoji (400 → -32603)
- Allow clearing emoji via set_collective_emoji(emoji=None)
- Remove destructiveHint from trash operations (soft deletes are recoverable)
- Change delete_collective to idempotentHint=False (requires trash precondition)
- Add restore_collective and get_trashed_collectives tools
- Add unit tests for ValueError guard, clear-emoji path, and new tools
- Add integration test for full trash/restore/delete lifecycle
- Verify move_page returns new title in response message
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Fix assign_tag sending Content-Type header with no body
- Mark collectives_update_collective as idempotent (no ETag involved)
- Raise OCSError when 'data' key missing instead of silent fallback
- Tighten color validator to 3 or 6 hex chars only
- Add comment explaining null emoji semantics in set_page_emoji
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Bugs:
- assign_tag/remove_tag now call _unwrap_ocs to surface OCS-level errors
- trash_page changed to idempotentHint=False (trashing twice errors)
- WebDAV path parts stripped of slashes to prevent double-slash paths
Robustness:
- _unwrap_ocs uses ocs.get("data", {}) instead of ocs["data"]
- Unit test added for missing data key in OCS envelope
Minor:
- MCP error codes use -1 (project convention) instead of HTTP status codes
- update_collective docstring notes that emoji is required
- CollectiveTag.color validated as hex format via field_validator
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Bug fixes:
- Catch OCSError/HTTPStatusError in all server tools, convert to McpError
- Guard update_collective against empty body (raise ValueError)
- Use restore_page response data in status message
ADR-017 annotation fix:
- Distinguish "remove" (reversible association) from "delete" (permanent):
remove_tag and deck_remove_label_from_card no longer set destructiveHint
- Update annotation test to exclude "remove" from destructive keywords
Data model improvements:
- Add trashTimestamp field to PageInfo
- Create ListTrashedPagesResponse with is_trash context flag
- Add collective_id to ListTagsResponse
Test robustness:
- Read NC credentials from environment variables (not hardcoded)
- Filter landing page by parentId == 0 instead of assuming pages[0]
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Implement MCP tools for the Collectives wiki/documentation app, enabling
agentic workflows for team knowledge base management.
16 tools covering collectives, pages, tags, search, and trash:
- Read: list collectives, list/get pages (with WebDAV content), search,
list tags, list trashed pages
- Write: create/update collective, create/move/trash/restore pages,
set emoji, create/assign/remove tags
Includes Docker hook for app installation, OCS API client with envelope
unwrapping, Pydantic models, unit tests (16), and integration tests (10).
Closes#621
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add 429 retry with exponential backoff to register_client() (fixes CI
oauth matrix failures from parallel DCR requests)
- Make client_id, redirect_uri, and PKCE mandatory at token endpoint
- Add null-checks for discovery_url and OAuth credentials in proxy flows
- Add OIDC discovery document caching with 5-min TTL
- Add per-IP rate limiting on /oauth/register DCR proxy
- Discover DCR endpoint from OIDC discovery instead of hardcoding
- Extract extract_user_id_from_token to auth/token_utils.py (breaks
circular imports between server/ and auth/ layers)
- Add TTL scope cache in scope_authorization.py (avoids DB hit per tool)
- Add defense-in-depth scope validation in storage layer
- Broaden elicitation exception handling with graceful fallback
- Add idempotentHint to nc_auth_check_status, return "pending" status
after accepted elicitation, add polling interval to description
- Change ALL_SUPPORTED_SCOPES from tuple to frozenset for O(1) lookups
- Replace Optional[str] with str | None throughout config.py
- Use default_factory for ProxyCodeEntry/ASProxySession dataclasses
- Add proxy code/session cleanup to background loop
- Fix OIDC verification CI step to only run for oauth/login-flow modes
- Add unit tests for access.py REST endpoints (10 tests)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Consolidate MCP session + login flow cleanup into _mcp_session_with_login_flow() helper,
replacing 4 duplicated AsyncExitStack sites in app.py
- Fix get_shared_storage() race condition by using module-level anyio.Lock() init
(reverts regression from ba59763)
- Collapse cosmetic if/else branching in scope_authorization.py
- Consolidate dual password storage paths into single store_app_password_with_scopes() call
- Mark unused request param as _ in list_supported_scopes
- Make ALL_SUPPORTED_SCOPES an immutable tuple; use list() instead of .copy()
- Add hasattr(ctx, "elicit") guard in elicitation.py, narrow except to NotImplementedError
- Add YAML comment explaining --oauth flag for mcp-login-flow service
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix anyio.Lock() created at module import time; use lazy init in
get_shared_storage() to avoid instantiation before event loop exists
- Stop get_login_flow_session from silently swallowing DB exceptions;
re-raise and handle in caller with proper error response
- Update ProvisionAccessResponse and UpdateScopesResponse status field
docs to include all actual values (declined, cancelled, unchanged)
- Narrow except clause in present_login_url to (AttributeError,
NotImplementedError) instead of bare Exception
- Add KeyError handling in LoginFlowV2Client.initiate() and poll() for
clear errors on malformed Nextcloud responses
- Simplify redundant env-var bypass branches in scope_authorization.py
- Extract _maybe_login_flow_cleanup() context manager to replace 4
inline cleanup loop registrations in app.py; move sleep to end of
loop body so cleanup runs once at startup
- Replace fragile string replacement in _rewrite_login_flow_url with
proper urllib.parse URL handling
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Use lowercase generics (list[...]) in new deck response models
- Add clarifying comment on AddressBook.uri slug semantics
- Fall back calendar_display_name to calendar_name when absent
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Restore contact email/birthday/nickname data and per-event calendar
source that were silently dropped during response model wrapping.
Remove dead elif branches in OAuth deck tests, add regression tests.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
MCP tools returning raw lists caused FastMCP's _convert_to_content() to create
one TextContent block per element. Most MCP clients only read content[0], so
they saw a single result instead of the full list.
Wrapped 9 tool functions in proper response objects:
- deck: deck_get_boards, deck_get_stacks, deck_get_cards, deck_get_labels
- calendar: nc_calendar_list_events, nc_calendar_get_upcoming_events
- contacts: nc_contacts_list_addressbooks, nc_contacts_list_contacts
- tables: nc_tables_list_tables
Closes#568
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Changes:
- Add file_path to metadata in semantic and BM25 hybrid search algorithms
for PDF viewer integration (search/semantic.py:161-163, search/bm25_hybrid.py:230-232)
- Include chunk_start_offset, chunk_end_offset, page_number, and page_count
in search results for rich chunk display (api/management.py:981-1004)
- Add point_id field to SearchResult for batch retrieval (models/semantic.py)
- Fix type narrowing for chunk context API parameters (api/management.py:1102-1111)
- Fix None-safety in doc_types discovery (search/algorithms.py:114)
This enables the Astroglobe UI to display PDF pages at the correct
location for matched chunks.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Add full integration for the Nextcloud News (RSS/Atom reader) app:
- Add NewsClient with complete CRUD operations for folders, feeds, and items
- Add 8 read-only MCP tools for listing/getting folders, feeds, items
- Add Pydantic models for News entities with camelCase alias support
- Add vector sync support for starred + unread items
- Add HTML to Markdown converter using markdownify for better embeddings
- Add Docker post-install hook to enable News app
- Add 25 unit tests for NewsClient API methods
Vector sync indexes starred and unread items, providing a balanced approach
that captures important (starred) and current (unread) content without
indexing the entire article history.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- algorithms.py: Revert SearchResult.id to int (all docs use int IDs now)
- semantic.py: Revert SemanticSearchResult.id to int, remove Union import
- viz_routes.py: Remove str() conversion when querying doc_id from Qdrant
- viz_routes.py: Convert doc_id from query param to int in chunk context
Fixes vector visualization which was collapsing all chunks to a single
point because Qdrant queries were failing to match doc_id (string vs int).
Major improvements to vector visualization page:
- Refactor PCA to display individual chunks instead of averaged documents
- Add context expansion module for fetching surrounding text from notes and PDFs
- Update deduplication to use (doc_id, doc_type, chunk_start, chunk_end) keys
- Fix Alpine.js rendering with chunk-specific keys including offsets
- Refactor authentication helper to return NextcloudClient for better reuse
- Add async context manager support to NextcloudClient
Technical details:
- viz_routes.py: Fetch specific chunk vectors instead of averaging per document
- context.py: New module supporting both notes and PDF text extraction via PyMuPDF
- search algorithms: Extract page_number, chunk_index, total_chunks from Qdrant
- vector-viz.js/html: Use chunk positions in expansion tracking keys
This enables users to see which specific chunks match their query
and view them with surrounding context in the PCA visualization.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Fix false-positive validation error where DBSF (Distribution-Based Score
Fusion) correctly produces scores > 1.0 but SearchResult validation
incorrectly rejected them.
**Root Cause**: SearchResult.__post_init__() enforced scores in [0.0, 1.0]
range, but DBSF sums normalized scores from multiple retrieval systems
(dense semantic + sparse BM25), resulting in scores like 1.55 when both
systems strongly agree a document is relevant.
**Changes**:
- Relaxed validation to allow any score ≥ 0.0 (algorithms.py:147-157)
- Updated SearchResult and SemanticSearchResult documentation to explain
score ranges for RRF ([0.0, 1.0]) vs DBSF (unbounded)
- Added comprehensive test coverage for both fusion methods
- Added DBSF fusion option to vector visualization UI
- Updated viz routes and vizApp() to support fusion parameter selection
**Testing**: All 157 unit tests pass, type checking passes, ruff passes
Fixes error: "Configuration error: Score must be between 0.0 and 1.0, got 1.1528953"
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This implements ADR-009, which documents the decision to use a generic
`semantic:read` OAuth scope instead of requiring all app-specific scopes
for semantic search functionality.
Changes:
- Created new `nextcloud_mcp_server/models/semantic.py` with semantic search models
- SemanticSearchResult (with new doc_type field for multi-app support)
- SemanticSearchResponse
- SamplingSearchResponse
- VectorSyncStatusResponse
- Created new `nextcloud_mcp_server/server/semantic.py` with semantic search tools
- nc_semantic_search (renamed from nc_notes_semantic_search)
- nc_semantic_search_answer (renamed from nc_notes_semantic_search_answer)
- nc_get_vector_sync_status (renamed from nc_notes_get_vector_sync_status)
- All tools now use @require_scopes("semantic:read") instead of "notes:read"
- Updated `nextcloud_mcp_server/server/notes.py`
- Removed semantic search tools (moved to semantic.py)
- Removed semantic search model imports
- Removed unused MCP imports (ModelHint, ModelPreferences, etc.)
- Updated `nextcloud_mcp_server/models/notes.py`
- Removed semantic search models (moved to semantic.py)
- Updated `nextcloud_mcp_server/app.py`
- Import configure_semantic_tools
- Register semantic tools when VECTOR_SYNC_ENABLED=true
- Updated `nextcloud_mcp_server/server/__init__.py`
- Export configure_semantic_tools
- Updated tests
- tests/integration/test_sampling.py: Use new tool names
- tests/unit/test_response_models.py: Import from semantic.py, add doc_type field
Architecture:
- Semantic search is now a cross-app feature, not tied to Notes
- Uses dual-phase authorization: semantic:read scope + per-document verification
- Supports future multi-app indexing (notes, calendar, deck, files, contacts)
Test results:
- All 69 unit tests passing
- All 5 smoke tests passing
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit addresses issues with vector database synchronization that
were causing test failures:
1. **Deletion Grace Period** (scanner.py)
- Fixed premature deletion of documents due to pagination cursor
inconsistencies in Notes API
- Implemented 2-scan verification with 1.5x scan interval grace period
(15 seconds default)
- Documents must be missing for 2 consecutive scans before deletion
- Documents that reappear are removed from deletion tracking
- Prevents false deletions during concurrent note creation/indexing
2. **Vector Sync Status Tool** (server/notes.py, models/notes.py)
- Added nc_notes_get_vector_sync_status MCP tool
- Returns indexed_count, pending_count, status, and enabled fields
- Enables tests and clients to wait for vector sync completion
- Uses lifespan context to access document queue and Qdrant client
3. **Test Improvements** (test_sampling.py, conftest.py)
- Added temporary_note_factory fixture for creating multiple test notes
- Updated all sampling tests to wait for vector sync completion
- Adjusted score_threshold to 0.0 for SimpleEmbeddingProvider
(feature hashing produces low-quality embeddings)
- Fixed CallToolResult extraction (removed ["result"] key access)
- Removed invalid @pytest.mark.asyncio markers (anyio mode)
All integration tests now pass successfully.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Add nc_notes_semantic_search_answer tool that combines semantic search
with MCP sampling to generate natural language answers from retrieved
Nextcloud Notes. This enables Retrieval-Augmented Generation (RAG)
patterns without requiring a server-side LLM.
Key features:
- Client-side LLM generation via ctx.session.create_message()
- Graceful fallback when sampling unavailable
- Proper source citations in generated answers
- No results optimization (skips sampling when no docs found)
- Comprehensive unit and integration tests
Implementation details:
- SamplingSearchResponse model with generated_answer and sources
- Fixed prompt template with document context and citation instructions
- Model preferences hint Claude Sonnet for balanced performance
- Falls back to returning documents without answer on sampling failure
Updates:
- Add ADR-008 documenting sampling architecture decision
- Add MCP sampling pattern guidance to CLAUDE.md
- Update README.md and docs/notes.md (7 → 9 tools)
- Add 4 unit tests and 6 integration tests
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Completes the ADR-007 implementation by adding user-facing semantic search
functionality. Previous phases implemented scanner and processor for background
indexing; this adds the query interface.
Changes:
- Add nc_notes_semantic_search MCP tool for natural language queries
- Fix Qdrant point IDs to use UUIDs instead of strings (was causing 400 errors)
- Reduce scan interval default from 1 hour to 5 minutes for faster updates
- Add SemanticSearchResult and SemanticSearchNotesResponse models
- Implement dual-phase authorization (Qdrant filter + Nextcloud API verification)
The semantic search enables finding notes by meaning rather than exact keywords,
using vector embeddings to understand query intent. Point ID fix resolves
critical bug where all document indexing failed with "invalid point ID" errors.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Import recipes from URLs using schema.org metadata
- Full CRUD operations for recipes
- Search, categorize, and organize recipes
- Manage keywords/tags and categories
- Configure app settings and trigger reindexing
