- Tighten verify_search_results signature: client: Any → NextcloudClientProtocol
- Collapse 3 copy-pasted lock-justification comments to a single-line pointer
- Add logger.debug timing around the verify_search_results call site
- Add logger.debug timing around the unbounded news.get_items fetch
- Rename SemanticSearchResponse.dropped_count → dropped_document_count to make
the chunks-vs-documents unit asymmetry explicit at the API boundary
- Drop unreachable duplicate 409 branch in WebDAVClient.move_resource
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Rename `verified_count` → `verified_chunk_count` to make the count
granularity explicit at the field name (chunks vs unique docs).
- News verifier now fails open *per-item* on non-numeric stored doc_ids
(matches notes/files/deck shape); a single bad id no longer rescues
definitively-missing siblings from eviction.
- Update note-verifier integration test to use string doc_ids end-to-end
to match production storage (scanner.py:241 stringifies note ids).
- Add regression test for the closed-task-group race guard in
`verify_search_results` so the RuntimeError swallow is locked in.
- Convert remaining f-string logger calls in `server/semantic.py` to
lazy %-style formatting (per repo convention).
- Document `evict_on_missing` as a developer/test flag (no env var) and
flag the `get_file_info` 404→raise contract change in its docstring.
- Add a TODO(ADR-019) breadcrumb for the hardcoded 2× over-fetch so
future tuning has a clear hook.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Round 7 raised 5 issues; this round addresses all of them and fixes
the underlying causes (not just the comments) where applicable so
they don't get re-flagged in future passes.
Critical:
- verified_count description in SemanticSearchResponse said "unique
documents" but the value is len(verified_results), a chunk count.
Description rewritten to accurately document chunk-level granularity
AND explicitly call out the asymmetry with dropped_count (which
counts unique (doc_id, doc_type) pairs).
- _verify_files false-eviction risk: the round-6 doc-only fix was
re-flagged. Address at the source — widen WebDAVClient.get_file_info
to raise HTTPStatusError on 404 (matching the rest of the client
convention) and reserve None for the genuinely ambiguous
malformed-PROPFIND case. _verify_files now keeps the result on None
(cannot tell whether the file exists) and evicts only on a
definitive HTTPStatusError 404. Tests updated; new test added for
the malformed-XML keep-result path.
Non-critical:
- News verifier semaphore lifetime now explicitly documented: one
slot held for one deduplicated fetch per search is the correct
backpressure behaviour.
- Cross-reference comments in _verify_notes / _verify_deck_cards no
longer claim "Mirrors X" pointing at functions defined later in
the file; now use direction-neutral "parallel implementation in".
- accessible_by_type is mutated by concurrent run_verifier tasks; a
comment explains why this is race-free under anyio's cooperative
multitasking (distinct keys per task, no await between read and
write) so a future reader doesn't add a redundant lock.
- Knock-on: tests/integration/test_rag.py wraps get_file_info in a
try/except for the new contract.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Addresses round-3 review feedback on PR #747:
- webhook_receiver: wrap send_stream.send() in anyio.fail_after(1.0)
and return 503 with reason="queue full" if the queue is saturated.
Avoids pinning the handler until NC's outbound timeout fires; the
503 retry contract is the same as the existing "sync not running"
branch.
- webhook_receiver: revise the compare_digest comment to match what
the function actually guarantees — it avoids the per-character
short-circuit of `==` but is not fully constant-time across length
differences.
- _get_webhook_uri: read WEBHOOK_INTERNAL_URL and
NEXTCLOUD_MCP_SERVER_URL via dynaconf so operators using
settings.toml (rather than env vars) aren't silently routed into
the docker/localhost fallback. Adds webhook_internal_url to
Settings/_DEFAULTS/_field_map; nextcloud_mcp_server_url already
existed. Docker-detection markers stay on os.getenv since they're
container-runtime signals, not user-facing config.
- webhook_routes: sweep remaining f-string logger calls to lazy %s
formatting per CLAUDE.md.
- client/webhooks: modernise full file's type hints to
dict / list / | None per CLAUDE.md.
Tests:
- New test_returns_503_when_queue_is_full exercises the timeout
branch with a saturated buffer and a shortened deadline.
- test_webhook_uri tests now patch get_settings (matching the
auth-pair tests in the same file) instead of monkeypatching env
vars directly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- webhook_receiver: always run hmac.compare_digest (drop the
`not provided or` short-circuit) so the constant-time path is
taken regardless of whether the Authorization header is present.
- client/webhooks: modernise the new `auth_data` type hint to
`dict[str, str] | None` per CLAUDE.md.
- tests/client: rename `test_create_webhook_with_auth_headers` →
`test_create_webhook_with_static_headers` and use
`auth_method="header"` (NC's webhook_listeners only supports
"none" and "header"; the previous "bearer" value was invalid).
- auth/webhook_routes: extract `_register_preset_webhooks` from
`enable_webhook_preset` so the auth-threading behaviour is
testable without standing up a Starlette app + auth middleware.
- tests/unit: new test_webhook_routes_register covering the helper
with secret set / unset, and verifying ids round-trip in order.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds optional shared-secret authentication for /webhooks/nextcloud,
addressing the security follow-up flagged in #747.
Behavior:
- WEBHOOK_SECRET set: registrations pass authMethod="header" with
authData={"Authorization": "Bearer <secret>"} (encrypted at-rest in
Nextcloud's DB and forwarded on every delivery). The receiver
validates the same header with hmac.compare_digest before parsing
any payload; missing/invalid → 401.
- WEBHOOK_SECRET unset: registrations stay on authMethod="none" and
the receiver accepts unauthenticated POSTs (logging a one-time
startup warning). Backward compatible — operators can roll out at
their own pace.
Implementation notes:
- WebhooksClient.create_webhook gains an `auth_data` parameter mapped
to NC's `authData` body field; this is distinct from the existing
`headers` parameter (`headers` is plaintext static request headers,
`authData` is encrypted at-rest in NC and only emitted when
authMethod="header"). The previous `auth_method="bearer"` mention in
the docstring was incorrect — NC supports only "none" and "header".
- A small `webhook_auth_pair()` helper in auth/webhook_routes.py
centralises the secret→(auth_method, auth_data) resolution so the
preset flow and the Astrolabe-facing /api/v1/webhooks endpoint stay
in sync.
Also addresses the smaller review points from #747:
- f-string → lazy %s formatting in webhook_receiver.py and
webhook_routes.py.
- Move `int(time)` inside webhook_parser's try/except so a malformed
`time` field returns None instead of raising ValueError.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes the seven outstanding items from the @claude review on PR #741:
1. Add empty `tests/client/talk/__init__.py` for pytest discovery parity
with `tests/client/{collectives,news}/`.
2. Standardise boolean query params to integers — `includeStatus` was the
string `"true"` in `list_conversations`/`list_participants` while every
other flag (`noStatusUpdate`, `lookIntoFuture`, `setReadMarker`,
`includeLastKnown`) used `1`/`0`.
3. Replace the `app:install || app:enable` chain in the spreed install hook
with `app:install --keep-disabled --force || true; app:enable spreed`,
so unrelated install failures surface as a clear "app not found" from
`app:enable` rather than being silently masked.
4. Add `_validate_token()` (alphanumeric whitelist) and call it from all
six TalkClient methods that interpolate the token into a URL path —
defence-in-depth against pathological tokens reaching httpx.
5. Rename `TalkConversation.type` to `room_type` with `Field(alias="type")`
and `populate_by_name=True`, so the field no longer shadows Python's
builtin while preserving spreed's wire format on input. MCP responses
now serialize `room_type` (field name) instead of `type`.
6. `mark_as_read` now passes `json=body or None` so the bodyless
"mark everything as read" call doesn't send a spurious `{}` body and
`Content-Type: application/json` header.
7. `_validate_message_text` rejects whitespace-only messages, not just
empty strings.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Addresses the missing-test and Content-Type points from the latest
PR #741 review:
- client/talk.py _talk_headers(): drop the manual `Content-Type:
application/json`. httpx sets it automatically on requests that pass
`json=`, and we no longer leak it onto bodyless GETs and DELETEs.
- tests/client/talk/test_talk_api.py:
- new `test_talk_list_participants_with_include_status` asserting
`includeStatus=true` is forwarded.
- new `test_talk_get_messages_invalid_last_given_header` covering
the defensive try/except around the `X-Chat-Last-Given` parse —
asserts the fallback `last_given=None` and that a warning is
logged.
- existing `test_talk_list_participants` extended to assert that
`includeStatus` is *absent* by default.
Unit tests: 13 → 15. Integration tests still 7/7.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Four targeted fixes from the AI code review:
1. TalkConversation.description: drop the misleading `str | None`
union (spreed always sends `""`, never null) — type is now `str`
with default `""`.
2. get_messages: guard the X-Chat-Last-Given int parse with
try/except so a misbehaving proxy can't crash the read flow;
logs a warning and falls back to None.
3. get_messages: clamp `limit` to [1, 200] in the client (spreed
caps server-side at 200 and silently truncates) so the returned
`count` always matches what was actually requested. Both client
and server-tool docstrings updated to state the valid range.
4. Add an integration test covering the 32000-char message ceiling
in talk_send_message — the empty-message case was already tested,
the over-length case was not.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds 6 MCP tools so an LLM can read a user's Talk conversations and
post messages on their behalf, addressing the "read my chats and reply"
use case from issue #720:
- talk_list_conversations
- talk_get_conversation
- talk_get_messages
- talk_list_participants
- talk_send_message (auto-attaches a referenceId for retry dedup)
- talk_mark_as_read
Edit/delete messages, reactions, threads, and call/session ops are
intentionally out of scope for this first PR.
The TalkClient also exposes create_conversation/delete_conversation
for the integration test fixture; these are not registered as MCP
tools. A post-installation hook enables spreed in the docker dev env
so the integration suite has a real Talk backend to talk to.
Closes#720
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Notes app v5.0.0 has scenarios where the API returns a JSON list where the
MCP server expects a single note object — notably the notes_api#fail
catch-all returning [] for unmatched routes. Without a guard, callers hit
a cryptic Pydantic "argument after ** must be a mapping, not list" from
Note(**payload).
Add a small _expect_note_object helper at the client layer:
- dict → pass through (the healthy case)
- single-element list → unwrap and warn (Notes v5.0.0 quirk)
- empty list, multi-element list, non-dict → raise a diagnostic ValueError
that names the operation and points at the likely root cause (URL prefix,
unmatched route, wrong API version)
Wire it into get_note / create_note / update so any list-shaped response
fails clearly instead of cryptically.
Six unit tests pin every branch of the helper.
Note: The 405s the issue reports for update_note / append_content match
Notes v5.0.0's documented routes (PUT /api/v1/notes/{id}) per upstream
appinfo/routes.php. They are most likely a downstream effect of #732
(missing /index.php URL prefix on installs without Pretty URLs) — the fix
in PR #733 should resolve those once it lands.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
caldav 3.x lists niquests as a mandatory dependency and prefers it over
httpx. Passing httpx.BasicAuth via the auth= argument breaks under the
niquests backend with "Unexpected non-callable authentication" — see #731.
Switch CalendarClient.__init__ from auth=Auth|None to keyword-only
password/token, and forward them to AsyncDAVClient as password= plus an
explicit auth_type ("basic" or "bearer"). caldav then builds whichever
auth object its active backend needs (niquests.auth.HTTPBasicAuth or
httpx.BasicAuth), so we stay backend-agnostic.
Threaded raw credentials through NextcloudClient — added keyword-only
password/token to its __init__, and updated from_env, from_token, and
the four call sites that build NextcloudClient (context.py basic-auth
and Login Flow paths, auth/userinfo_routes.py, vector/oauth_sync.py).
Four new unit tests pin the construction wiring so the niquests
regression can't recur silently — basic, bearer, no-creds, and
password-precedence cases.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Bare /apps/<app>/... URLs return 404 on Nextcloud installs without Pretty
URLs (URL rewriting), which is opt-in and not the default — see #732. The
/index.php/apps/... form is the universal entry point and works regardless
of web-server config, matching how /remote.php/dav and /ocs/v2.php already
have dedicated entry points.
Add a small _resolve_url helper on BaseNextcloudClient that rewrites
/apps/... → /index.php/apps/... at the top of _make_request, so every
current call site (notes, deck, cookbook, news) and any future ones are
covered transparently with no per-client churn.
Other path prefixes (/remote.php, /ocs, absolute URLs, already-prefixed
/index.php/apps) pass through unchanged. New unit tests in
tests/unit/client/test_base.py pin all six cases.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
pythonvCard4 parses vCard BDAY fields into datetime.date objects, but
the Contact model expects Optional[str]. This caused a validation error
that crashed the entire contact list. Convert at the client layer
(consistent with the calendar client pattern) with a defensive check
at the server mapping layer.
Closes#672
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Fix emoji clearing bug: use _UNSET sentinel in update_collective so
emoji=None sends {"emoji": null} instead of raising ValueError
- Move collectives_get_trashed_collectives to Read Tools section
- Remove redundant is_trash field from ListTrashedPagesResponse
- Add page lifecycle note to collectives_trash_page docstring
- Add unit test for clearing collective emoji via update_collective
- Add integration test for clearing collective emoji via MCP tool
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Fix inconsistent error code in set_collective_emoji (400 → -32603)
- Allow clearing emoji via set_collective_emoji(emoji=None)
- Remove destructiveHint from trash operations (soft deletes are recoverable)
- Change delete_collective to idempotentHint=False (requires trash precondition)
- Add restore_collective and get_trashed_collectives tools
- Add unit tests for ValueError guard, clear-emoji path, and new tools
- Add integration test for full trash/restore/delete lifecycle
- Verify move_page returns new title in response message
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Fix assign_tag sending Content-Type header with no body
- Mark collectives_update_collective as idempotent (no ETag involved)
- Raise OCSError when 'data' key missing instead of silent fallback
- Tighten color validator to 3 or 6 hex chars only
- Add comment explaining null emoji semantics in set_page_emoji
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Validate OCS envelope in trash_collective, delete_collective, trash_page
- Guard _unwrap_ocs against non-OCS responses with informative OCSError
- Remove _get_ocs_headers() indirection, use class constants directly
- Split headers: _OCS_HEADERS (GET) vs _OCS_HEADERS_JSON (with body)
- Fix docstring claiming emoji param is required when it is optional
- Rename misleading test, add test for non-OCS envelope handling
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add collectives_trash_collective and collectives_delete_collective MCP
tools with proper destructiveHint annotations. Refactor integration test
fixture to use MCP tools for cleanup instead of direct httpx/OCS calls.
Optimize _get_ocs_headers() to class-level constant.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Bugs:
- assign_tag/remove_tag now call _unwrap_ocs to surface OCS-level errors
- trash_page changed to idempotentHint=False (trashing twice errors)
- WebDAV path parts stripped of slashes to prevent double-slash paths
Robustness:
- _unwrap_ocs uses ocs.get("data", {}) instead of ocs["data"]
- Unit test added for missing data key in OCS envelope
Minor:
- MCP error codes use -1 (project convention) instead of HTTP status codes
- update_collective docstring notes that emoji is required
- CollectiveTag.color validated as hex format via field_validator
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Bug fixes:
- Catch OCSError/HTTPStatusError in all server tools, convert to McpError
- Guard update_collective against empty body (raise ValueError)
- Use restore_page response data in status message
ADR-017 annotation fix:
- Distinguish "remove" (reversible association) from "delete" (permanent):
remove_tag and deck_remove_label_from_card no longer set destructiveHint
- Update annotation test to exclude "remove" from destructive keywords
Data model improvements:
- Add trashTimestamp field to PageInfo
- Create ListTrashedPagesResponse with is_trash context flag
- Add collective_id to ListTagsResponse
Test robustness:
- Read NC credentials from environment variables (not hardcoded)
- Filter landing page by parentId == 0 instead of assuming pages[0]
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Validate OCS envelope status before unwrapping data (raise OCSError on
statuscode >= 400)
- Fix test data: filePath should be "" for root-level pages, not filename
- Catch specific exceptions (HTTPStatusError, OSError) instead of bare
Exception in WebDAV content fetch, include error in log message
- Return updated resource data from update_collective, move_page, and
set_page_emoji instead of discarding API responses
- Fix create_page docstring to mention collectivePath/filePath/fileName
- Remove unused additional_headers parameter from _get_ocs_headers
- Add unit test for OCS error status validation
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Implement MCP tools for the Collectives wiki/documentation app, enabling
agentic workflows for team knowledge base management.
16 tools covering collectives, pages, tags, search, and trash:
- Read: list collectives, list/get pages (with WebDAV content), search,
list tags, list trashed pages
- Write: create/update collective, create/move/trash/restore pages,
set emoji, create/assign/remove tags
Includes Docker hook for app installation, OCS API client with envelope
unwrapping, Pydantic models, unit tests (16), and integration tests (10).
Closes#621
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Update stale httpx reference to niquests in calendar.py type comment
- Replace inline inspect.isawaitable with _maybe_await helper in tests
- Fix incorrect port number in docker-compose unstructured comment
- Remove commented-out smithery service block (dead code)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When Nextcloud stores CalDAV objects, the server-side filename may differ
from the VTODO/VEVENT UID. The caldav fork constructed object URLs from
the UID instead of the actual <d:href> from REPORT responses, causing
list_todos to return wrong hrefs, delete_todo to silently no-op, and
update_todo to fail.
Upstream caldav v3.0.1 fixes this in _async_request_report_build_resultlist
by passing url=self.url.join(url) when constructing result objects.
Key changes:
- Replace caldav fork with upstream caldav>=3.0.1,<4.0
- Update imports to caldav.aio module
- Add _maybe_await() helper for v3's dual-mode methods that return
either objects or coroutines depending on async context
- Add _async_object_by_uid() to work around upstream's get_object_by_uid
not being async-aware (it iterates a coroutine synchronously)
- Adapt save_event/save_todo (no longer return tuples)
- Pass url=calendar.url.join(href) in _search_events_by_date
- Pass include_completed=True in list_todos to match previous behavior
- Add integration test for filename != UID scenario
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Fix three related contacts bugs:
- Parse dict-format vCard fields ({value, type}) that pythonvCard4 returns,
which previously crashed Pydantic validation expecting plain strings
- Include tel field in client output so phone numbers reach MCP tools
- Clarify addressbook parameter expects URI slug, not displayname
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Enable ruff PLC0415 rule for all source files (tests excluded via
per-file-ignores). Move 136 inline imports to top-level across 33 files.
8 imports suppressed with noqa for legitimate reasons: circular
dependencies (client/__init__.py, context.py), optional dependency
guards (app.py document processors, auth/userinfo_routes.py), and
post-env-setup imports (smithery_main.py).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
caldav types declare ssl_verify_cert as Union[bool, str] but the value
is passed through to httpx which accepts ssl.SSLContext. Add type: ignore
annotation to satisfy the ty type checker in CI.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add NEXTCLOUD_VERIFY_SSL and NEXTCLOUD_CA_BUNDLE env vars to configure
TLS certificate verification for all outbound Nextcloud connections.
Centralizes SSL config via a new HTTP client factory (http.py) used by
all 27 Nextcloud-bound call sites, including API clients, OIDC endpoints,
OAuth flows, and health checks.
Closes#560
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Trim whitespace from comma-separated category values in all three
methods: _create_ical_event, _merge_ical_properties, and
_merge_ical_todo_properties. Prevents leading/trailing spaces in
category names from inputs like "work, meeting".
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
_merge_ical_properties() only handled a subset of event fields, silently
dropping categories, recurrence_rule, attendees, and reminder_minutes
during updates. These fields were fully supported by _create_ical_event()
and accepted by the MCP tool, but never applied.
Closes#544
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
PR #539 fixed date-range filtering so events outside the queried range
are excluded. However, recurring events still returned the master event
with its original DTSTART instead of expanded occurrences.
Add <C:expand> element to CalDAV REPORT requests (RFC 4791 §9.6.5) when
both date bounds are provided, so the server returns one VEVENT per
occurrence with the correct DTSTART. Refactor VEVENT parsing into a
shared helper and add _parse_all_ical_events() to handle multi-VEVENT
responses from expanded results.
Closes#538
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
get_calendar_events() accepted start/end datetime parameters but called
calendar.events() which fetches all events, silently discarding the
date filters. This caused nc_calendar_list_events and
nc_calendar_get_upcoming_events to return the entire calendar history.
Add _search_events_by_date() helper that builds a CalDAV REPORT query
with a <time-range> filter (RFC 4791 §9.9) for server-side filtering.
Falls back to calendar.events() when no dates are given.
Closes#538
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The reorder_card method was using the API route
/api/v1.0/boards/{boardId}/stacks/{stackId}/cards/{cardId}/reorder
which has a parameter conflict: the URL's {stackId} (current stack)
overrides the body's stackId (target stack) in Nextcloud's routing.
This caused cards to stay in their original stack even when the API
reported success.
Switched to the non-API route /cards/{cardId}/reorder which correctly
reads stackId from the request body, matching the behavior of the
working curl command reported in the issue.
Also added the required OCS-APIRequest headers that were missing.
Fixes#469
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The Deck PUT API is a full replacement, not a partial update.
Previously, title and description were conditionally sent, causing:
- 400 errors when title not provided (it's required)
- Description being cleared when not explicitly set
Now all required fields (title, type, owner) and description are
always included in the payload using current card values when not
explicitly provided. This matches the existing pattern for type/owner.
Also simplified owner extraction since DeckCard.validate_owner
already ensures it's always a string.
Fixes#452🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add type casts for Starlette app state access
- Add assertions for cipher, card, board, stack after initialization
- Add None checks for XML element text attributes
- Handle __package__ being None in tracing setup
- Fix TokenBrokerService initialization to use storage credentials
Resolves 42 type warnings from ty-check, enabling CI linting to pass.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Reverts the "perf(news): use direct API endpoint for get_item()" change
from commit 92c4bf3 which incorrectly assumed GET /items/{itemId} exists.
The News API (v1-2, v1-3, v2) does not provide a direct endpoint to
retrieve individual items. The only /items/{itemId} routes are POST
operations for marking items read/unread/starred.
Changes:
- Restore original get_item() implementation that fetches all items
and filters in Python
- Update exception from HTTPStatusError to ValueError
- Restore documentation explaining API limitation
- Update unit tests to mock get_items() instead of _make_request()
- Add test for ValueError when item not found
Fixes vector processor 405 errors when indexing news items.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Fixed 8 type checker errors across the codebase:
- vector/scanner.py: Handle None scroll results with null-safe iteration
- search/{bm25_hybrid,semantic}.py: Add None checks for result.payload
- auth/{unified_verifier,webhook_routes}.py: Assert non-None auth credentials
- client/webdav.py: Add None checks before int() conversions
- providers/openai.py: Assert embedding_model is not None
- search/algorithms.py: Explicitly type doc_types set and cast values
- observability/logging_config.py: Match parent class signature (log_data)
Also fixed test_create_tag_creates_system_tag to match WebDAV implementation
(was testing OCS API endpoint, now tests correct WebDAV endpoint with
Content-Location header).
Type checker: 0 errors (down from 8), 20 warnings (ignored)
Tests: All 192 unit tests passing
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Replace O(n) fetch-all-and-filter approach with O(1) direct API call.
The News API v1-3 supports GET /items/{id} for single-item retrieval.
- Update get_item() to use direct endpoint
- Add unit test for get_item() method
- Fixes critical performance issue identified in code review
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Add full integration for the Nextcloud News (RSS/Atom reader) app:
- Add NewsClient with complete CRUD operations for folders, feeds, and items
- Add 8 read-only MCP tools for listing/getting folders, feeds, items
- Add Pydantic models for News entities with camelCase alias support
- Add vector sync support for starred + unread items
- Add HTML to Markdown converter using markdownify for better embeddings
- Add Docker post-install hook to enable News app
- Add 25 unit tests for NewsClient API methods
Vector sync indexes starred and unread items, providing a balanced approach
that captures important (starred) and current (unread) content without
indexing the entire article history.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Change create_tag() to use WebDAV POST instead of OCS API which
returned 404 in some Nextcloud versions
- Add llm_judge() helper that evaluates system output against ground
truth with simple TRUE/FALSE prompt
- Replace keyword-based assertions in RAG tests with LLM judge for
more flexible semantic evaluation
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add get_file_info() to get file info including file ID via PROPFIND
- Add create_tag() to create system tags via OCS API
- Add get_or_create_tag() for idempotent tag creation
- Add assign_tag_to_file() to assign tags to files via WebDAV
- Add remove_tag_from_file() to remove tags from files
Also refactors RAG evaluation:
- Add indexed_manual_pdf fixture using existing nc_client/nc_mcp_client
- Remove manual tag creation steps from workflow (now handled by fixture)
- Add comprehensive unit tests for new WebDAV methods
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Major improvements to vector visualization page:
- Refactor PCA to display individual chunks instead of averaged documents
- Add context expansion module for fetching surrounding text from notes and PDFs
- Update deduplication to use (doc_id, doc_type, chunk_start, chunk_end) keys
- Fix Alpine.js rendering with chunk-specific keys including offsets
- Refactor authentication helper to return NextcloudClient for better reuse
- Add async context manager support to NextcloudClient
Technical details:
- viz_routes.py: Fetch specific chunk vectors instead of averaging per document
- context.py: New module supporting both notes and PDF text extraction via PyMuPDF
- search algorithms: Extract page_number, chunk_index, total_chunks from Qdrant
- vector-viz.js/html: Use chunk positions in expansion tracking keys
This enables users to see which specific chunks match their query
and view them with surrounding context in the PCA visualization.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit addresses multiple issues with async operations, PDF metadata
extraction, and type safety in document processing and search.
## Async/Await Fixes
- processor.py:259 - Added await for chunker.chunk_text(content)
- processor.py:270 - Added await for bm25_service.encode_batch(chunk_texts)
- tests/unit/test_document_chunker.py - Converted all 12 test methods to async
## PDF Metadata Enhancement
- pymupdf.py:143 - Added file_size metadata extraction
- pymupdf.py:145-206 - Refactored to extract text page-by-page
- Manually loop through pages instead of using page_chunks=True
- Generate page_boundaries metadata for precise page tracking
- Works around pymupdf.layout.activate() breaking page_chunks=True
- processor.py:32-66 - Added assign_page_numbers() helper function
- Assigns page numbers to chunks based on overlap with page boundaries
- Handles chunks spanning multiple pages
- processor.py:298-300 - Call assign_page_numbers() for PDF files
## Type Safety Fixes
- bm25_hybrid.py:184 - Removed int() conversion of doc_id
- semantic.py:131 - Removed int() conversion of doc_id
- viz_routes.py:275 - Removed int() conversion of doc_id
- Added comments documenting that doc_id can be int (notes) or str (file paths)
## Testing
- All 18 tests passing (12 unit + 6 integration)
- No type errors in modified files
- Container logs show successful processing
- Vector viz searches working correctly
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit addresses critical performance issues with vector visualization
search (reducing time from 40s to ~2s) and improves result visualization
through better visual encoding.
## Performance Fixes
### 1. Fix blocking sleep in retry decorator (base.py:51)
- Changed `time.sleep(5)` to `await anyio.sleep(5)` in @retry_on_429
- Prevents entire event loop from freezing during rate limit retries
- Impact: Reduced search time from 22s to 16s initially
### 2. Add concurrency limiting for verification (verification.py:77-93)
- Added `anyio.Semaphore(20)` to limit concurrent HTTP requests
- Prevents connection pool exhaustion (RequestError) from 90+ simultaneous requests
- Fixes false filtering (was filtering 77/90 results incorrectly)
- Note: Semaphore still in code but verification removed from viz endpoint
### 3. Remove unnecessary verification from viz endpoint (viz_routes.py:483-486)
- Visualization only needs Qdrant metadata (title, excerpt), not full content
- Verification only required for sampling (LLM needs full note content)
- Impact: Reduced search time from 43.7s to ~2s (final fix)
### 4. Restore streaming scanner pattern (scanner.py)
- Process notes one-at-a-time using async generator
- Avoids loading all notes into memory
## Visualization Improvements
### 5. Result-relative score normalization (viz_routes.py:489-504)
- Normalize scores within result set: best=1.0, worst=0.0
- Removes arbitrary RRF normalization (theoretical max didn't make sense)
- Makes visual encoding meaningful regardless of algorithm scores
### 6. Power scaling for marker sizes (userinfo_routes.py:743)
- Changed from linear `8 + (score * 12)` to power `6 + (score² * 14)`
- Creates dramatic visual contrast: 0.0→6px, 0.5→9.5px, 1.0→20px
- Combined with opacity (0.2-1.0) for clear visual hierarchy
### 7. Multi-channel visual encoding (userinfo_routes.py:740-745)
- Size: Exponentially scaled with score²
- Opacity: Linear 0.2-1.0 (keeps all points visible)
- Color: Viridis gradient (blue→yellow)
- Effect: Top results are large/bright/opaque, context results small/dim/transparent
## Result
- Search time: 40s → ~2s (20x faster)
- Visual contrast: Subtle → dramatic (clear result hierarchy)
- No arbitrary cutoffs: All results visible, best naturally highlighted
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
