Latest reviewer comment flagged five items on top of the original PR. This
commit addresses every one:
🟡 1. Skip the offset-based Qdrant fallback for `doc_type=file` when
`chunk_index` is supplied. Qdrant Cloud's strict mode rejects unindexed
filter fields with HTTP 400, which `_get_chunk_from_qdrant` catches and
logs at `logger.error` — masking real Qdrant problems in monitoring.
Notes/cards keep the offset fallback (cheap, useful for legacy data).
🟡 2. Add a `logger.warning` and clarifying inline comment in the doc-text
fallback path when `chunk_index` is None — surfaces the pre-existing
"0/N misreport" so callers can detect it. Type-nullability propagation
is deferred to a follow-up (out of scope for this hotfix).
🟢 3. Simplify `if chunk_text and doc_id_int is not None:` →
`if chunk_text:` with an inner `assert doc_id_int is not None` for
`ty` narrowing. The outer second clause was dead.
🟢 4. Add `doc_type` `FieldCondition` to the offset-based image lookup in
both `visualization.py` and `viz_routes.py` for parity with the
`chunk_index` branches.
🟢 5. Inline the `chunk_filter` local in `visualization.py` directly into
the `must=[]` list (matches `viz_routes.py` style).
Adds `tests/unit/test_chunk_context_offset_gate.py` with three regression
tests covering the gate matrix: (file, with-index → skip offset),
(note, with-index → still tries offset), (file, no-index → still tries
offset). Lives at top-level rather than `tests/unit/search/` to side-step
a pre-existing circular-init issue in `nextcloud_mcp_server.search`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per-chunk PDF page renders (~150–700 KB base64 PNG each) were the dominant
disk consumer in production, repeatedly tripping `No space left on device:
WAL buffer size exceeds available disk space` on welcomed-malamute Qdrant.
Replace the inline highlighted_page_image / highlighted_page_number /
highlight_count fields with a small `chunk_bbox` field:
list[(x0, y0, x1, y1)] of normalized [0, 1] floats, ~32 bytes per chunk.
Astrolabe (the only known consumer) renders the highlight client-side as
a percentage-positioned overlay on top of the existing /api/v1/pdf-preview
render-on-demand path (cbcoutinho/astrolabe#76).
- pdf_highlighter: new compute_chunk_bboxes_batch() that reuses the
existing _find_chunk_bbox text-search path, skipping all pixmap/PIL/PNG
work.
- processor: store chunk_bbox + chunk_bbox_page in the Qdrant payload,
drop highlighted_page_image + friends, drop the base64 import.
- visualization /api/v1/chunk-context and auth/viz_routes: read
chunk_bbox instead of highlighted_page_image.
- vector/__init__: stop eagerly re-exporting `processor`/`scanner` —
fixes a pre-existing circular import (search.algorithms ->
vector.placeholder -> vector/__init__ -> processor -> scanner ->
server.semantic -> search.bm25_hybrid -> search.algorithms partial).
Test suite that was broken on master (test_bm25_hybrid.py et al.) now
collects and passes.
- scripts/purge_page_images.py: ad-hoc, idempotent migration that
delete_payload's the legacy keys from existing points. No reindex
required; legacy chunks render the page with no overlay.
Pairs with cbcoutinho/astrolabe#76. Frontend handles missing chunk_bbox
gracefully, so this can land in either order.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Add `doc_type` FieldCondition to the chunk_index-path highlighted-image
Qdrant filter in both `api/visualization.py` and `auth/viz_routes.py`,
matching the shape of `_get_chunk_by_index_from_qdrant`. Safe today (the
block is guarded by `doc_type == "file"` and Nextcloud file IDs are
globally unique) but prevents a latent bug if other doc types start
storing highlighted images.
- Demote `viz_routes.py` `ValueError` log from `error` to `warning` (lazy
%-style) — `_parse_int_param` raises on user-supplied bad input, which
is a 400 not a server error and shouldn't pollute error logs.
- Hoist `effective_chunk_index` to compute once at the top of
`get_chunk_with_context`, removing two duplicate assignments.
- Add `test_file_doc_type_qdrant_miss_yields_fast_404` to the management
endpoint tests, locking in the proxy-timeout fix contract.
- Add `tests/unit/test_viz_routes_chunk_context.py` mirroring management
coverage for the OAuth-session route: param forwarding (chunk_index /
total_chunks), `doc_type=file` fast 404, and 400 on invalid int params.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Production was logging two cascading classes of Qdrant errors against the
welcomed-malamute deployment:
1. HTTP 400 — "Bad request: Index required but not found for \"doc_id\" of
one of the following types: [keyword]". The collection was created via
create_collection() with no payload indexes, so any FieldCondition
filter on doc_id failed at the Qdrant layer (placeholder writes/reads,
eviction, search context lookups).
2. Compounding the missing index, producers wrote a mix of int and str
doc_ids: webhook_parser stringified node_id, scanner stringified note
IDs, news IDs, and deck card IDs — but the file scanner passed the
numeric file_id through unchanged. A keyword index would not have
covered both kinds even if it had existed.
This change:
- Normalizes doc_id to str at every producer site (scanner.py:459,
DocumentTask.doc_id, indexed_*_ids reads from Qdrant).
- Tightens str|int annotations to str across placeholder.py,
eviction.py, search/verification.py, search/context.py,
SearchResult.id, and the auth/api visualization endpoints.
- Defensive str() coercion on doc_id reads in semantic.py /
bm25_hybrid.py / vector/visualization.py for the transition window
before the backfill runs.
- Adds an idempotent startup migration in get_qdrant_client():
- _ensure_keyword_payload_indexes creates KEYWORD indexes for
doc_id, user_id, and doc_type (tolerates "already exists" 400s).
- _backfill_doc_id_to_string scrolls the collection once and rewrites
int doc_ids to str. Skipped after a quick sample shows no legacy
int payloads.
- Public API preserved: SemanticSearchResult.id stays int via explicit
int(r.id) narrowing in server/semantic.py — surfaces a TypeError with
actionable context if a future doc_type ships non-numeric ids.
- Documents the startup migration in docs/configuration.md.
Tests: 11 new unit tests in tests/unit/vector/test_qdrant_client.py
covering happy path / already-exists / unrelated-400 for the index
helpers, and sample-skip / mixed-batch rewrite / payload=None edge cases
for the backfill. 889 unit tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Replace bare int() casts for start/end/context_chars in chunk_context_endpoint
with _parse_int_param, matching visualization.py bounds (0–10M for offsets,
0–10K for context_chars), and add the missing end > start guard.
- Initialize page_number from chunk_context.page_number so non-file doc_types
surface it; include page_number, chunk_index, and total_chunks unconditionally
in the response. Only highlighted_page_image stays gated on its own truthiness.
- Add a chunk_index forwarding regression test that asserts the new kwargs reach
get_chunk_with_context and appear in the response payload.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PR #767 review noted that the OAuth viz route used bare int() parsing for
chunk_index and total_chunks while the bearer-token visualization route
validates them via _parse_int_param. Mirror the same bounds check so
total_chunks=0 and negative chunk_index return 400 instead of silently
suppressing adjacent-chunk context.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two related bugs surfaced in production while viewing chunks from the
Astrolabe frontend on the AWS-hosted MCP server:
1. PyMuPDF document closed: in _fetch_document_text the fallback path
referenced pdf_doc.page_count after pdf_doc.close(), raising
"document closed" and returning None. The slow PDF re-parse already
completed but its result was discarded. Capture page_count into a
local before close().
2. Slow/fragile chunk lookup: get_chunk_with_context filtered Qdrant by
(chunk_start_offset, chunk_end_offset). Those fields are not part of
the always-indexed payload schema, and with strict_mode enabled they
yield 400 errors. Even with manually-added indexes the filter is
fragile if a doc is re-chunked. Switch to chunk_index (always
indexed) as the primary lookup key, falling back to offset-based
lookup when callers don't supply it.
Plumb chunk_index/total_chunks through both the management API
(api/visualization.py) and the OAuth viz route (auth/viz_routes.py).
Apply the same change to the highlighted-image lookup so all four
chunk-context Qdrant queries prefer the indexed field.
Skip the slow PDF re-parse fallback entirely for files: when both the
chunk_index and offset Qdrant lookups miss, re-downloading and
re-parsing the source PDF won't find the chunk either, and routinely
exceeds 30s on large documents - which is the proxy timeout in
Astrolabe. Notes/cards keep the document-fetch fallback (cheap).
Removes dead code (_get_file_path_from_qdrant) that was only used by
the now-unreachable file fallback path.
Companion change in the Astrolabe app passes chunk_index from search
results through to the new endpoint params.
---
_This PR was generated with the help of AI, and reviewed by a Human_
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Gate browser session creation on a successful refresh token. When the
IdP returns no refresh token, SessionAuthBackend would silently reject
every subsequent request and bounce the user back to /oauth/login in a
loop. The callback now bails with a 400 + correlation ID + actionable
hint about offline_access *before* writing browser_sessions or setting
the cookie. Pinned by a new end-to-end unit test.
- Evict orphaned browser_sessions rows in SessionAuthBackend when the
associated refresh token is gone, instead of letting them accumulate
until TTL cleanup. Best-effort; deletion errors stay non-fatal.
- Demote identity-bearing logs in the Flow 2 OAuth callback (user_id,
scopes, audience, expires_at) from INFO to DEBUG so they don't leak
into multi-tenant log aggregation on every provision.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Coerce refresh_expires_in to int before arithmetic in both callback
paths so IdPs that serialize the field as a JSON string (e.g. AWS
Cognito) don't trigger an unhandled TypeError 500.
- Drop the orphaned oauth_session row written by _check_logged_in. The
canonical Flow 2 row is created by generate_oauth_url_for_flow2 keyed
by `state`, which is what the unified callback looks up; the
flow2_<hex> session_id was never matched and just churned the table
for 10 minutes per call.
- Match delete_cookie attributes (httponly, secure, samesite) to the
set_cookie call on logout so browsers reliably evict the cookie even
on implementations that consider security flags during deletion.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Five findings from the latest review on #758 (2 medium, 3 nit):
Medium:
- browser_oauth_routes.oauth_login_callback + oauth_routes.oauth_callback_nextcloud:
fail closed with 400 when the oauth_session row is unknown/expired. Previously
both callbacks fell through with code_verifier="" and expected_nonce=None,
silently bypassing the PKCE + nonce protections introduced in earlier rounds.
Symmetric unit tests pin both contracts.
- token_utils.verify_id_token: use secrets.compare_digest for the nonce check
instead of short-circuit !=. Mirrors the sibling PKCE verifier comparison;
closes the last secret-equality timing-side-channel surface in the auth path.
Nit:
- Tighten the comment at all 4 mcp_authorization_code/code_verifier store +
retrieve sites so a future refactor sees the field reuse immediately
(renaming the column requires a schema migration).
- _should_use_secure_cookies: explicit string normalisation instead of
bool(settings.cookie_secure). Dynaconf normally coerces but tests / direct
settings.set calls can leave the raw string in place — bool("false") is True.
New parametrized unit tests cover the coercion matrix + http/https fallback.
- oauth_routes.py:591 f-string log converted to lazy %s formatting (folded into
the Flow 2 callback rewrite).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three findings from the latest review on #758 (1 medium, 2 low):
Medium:
- browser_oauth_routes.oauth_logout: move delete_browser_session into a
finally block so an error from delete_refresh_token can no longer leave
an orphan browser_sessions row. The orphan was not exploitable
(SessionAuthBackend rejects sessions without a live refresh token), but
it lingered until the hourly cleanup cron — a correctness gap. New
regression test pins the fix.
Low:
- oauth_callback_nextcloud: drop redundant ``or None`` from
``expected_nonce=nonce``. ``nonce`` is already ``str | None`` and
``secrets.token_urlsafe`` never produces an empty string, so the
coercion was a no-op that could mislead future readers into thinking
empty-string was a valid skip-the-check path.
- storage.RefreshTokenStorage.initialize: fail fast at startup when
SQLite < 3.35, since ``DELETE ... RETURNING`` (used in
``delete_browser_session``) needs that minimum. Ubuntu 20.04 ships
3.31 and would otherwise hit OperationalError on every logout.
Prerequisite also documented in docs/installation.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Seven findings from the latest review on #758 (3 medium, 4 low/nit):
Medium:
- storage.py: replace 5 ``assert self.cipher is not None`` sites with
explicit ``RuntimeError`` so missing TOKEN_ENCRYPTION_KEY can't silently
become an AttributeError under ``python -O``
- session_backend.py: document the silent-invalidation invariant —
refresh-token TTL expiry without explicit logout deliberately makes
the browser session unusable; future readers must not relax it
- server/oauth_tools.py: drop user_id from the Flow 2 session_id
identifier — use ``flow2_{secrets.token_hex(16)}`` so audit logs and
DB rows don't carry user_id in the session_id field
Low / nit:
- token_utils.py: drop _fetch_locks dict entry in finally so a probed
deployment can't grow the lock dict without bound; coalescing test
now pins the invariant with len(_fetch_locks) == 0
- browser_oauth_routes.py: strip trailing slash from settings.nextcloud_host
before constructing the well-known URL so a host configured as
``https://cloud.example.com/`` doesn't produce a double-slash
- browser_oauth_routes.py: add comment explaining the three-layer CSRF
policy on the mcp_session cookie set (SameSite=Lax + POST-only logout
+ Origin/Referer check)
- oauth_routes.py: convert all 23 f-string log calls to lazy %-style
per the CLAUDE.md / memory feedback_lazy_logging convention
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Seven findings from the latest review on #758, plus a regression test
catching the substance of the cache-stampede fix:
- verify_id_token: widen id_token annotation to str | None to match
callers passing nc_token_response.get("id_token")
- extract_user_id_from_token: use JSON-RPC reserved error code -32001
instead of -1
- _get_cached: per-URL anyio.Lock dict + meta-lock coalesces concurrent
cache misses into a single IdP fetch (mirrors token_broker.py idiom)
- delete_browser_session: collapse SELECT+DELETE into atomic
DELETE ... RETURNING user_id (SQLite >= 3.35)
- new test_origin_normalise.py: parametrized port/scheme/host equivalence
cases for the CSRF Origin guard
- browser_oauth_routes: correct misleading "PR #758 finding 5" cross-
references (finding 5 was Fernet-key hardening, not CSRF)
- ASProxySession.nonce: make required, drop spurious "legacy session"
default; reword the in-flight `or None` comment to reflect that
ASProxySession is purely in-memory
- new test_get_cached_coalesces_concurrent_misses: pins the
cache-stampede protection — fires 10 concurrent _get_cached calls and
asserts exactly one HTTP fetch
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Flow 2 (oauth_authorize_nextcloud) now generates a nonce, stores it on
the oauth_session row, forwards it to the IdP, and verifies it via
expected_nonce in oauth_callback_nextcloud — closes the last replay-
protection gap (round-3 finding 1).
- _origin_matches_self fails closed when mcp_server_url is missing
instead of allowing the logout, and the diagnostic log is promoted
from warning to error so the misconfiguration is monitorable
(round-3 finding 2). New regression test pins the new behaviour.
- The five user_id-accepting helpers in oauth_tools.py (get_provisioning_status,
provision_nextcloud_access, revoke_nextcloud_access, check_provisioning_status,
check_logged_in) are renamed with leading underscores to make the
trust boundary structural rather than documentary
(round-3 finding 3).
- create_browser_session and delete_browser_session now emit audit_log
rows so session establishment / teardown match the pattern used by
the rest of the security-relevant storage operations
(round-3 nit 5). delete_browser_session selects user_id before delete
so the audit row is attributable.
- oauth_login_callback no longer reflects raw IdP-error text or
exception strings into the HTML failure page; users see a generic
"internal error occurred" message + a correlation ID, with the
detail logged server-side keyed by the same ID (round-3 nit 6).
The XSS regression test is updated to pin the stricter contract.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- oauth_login_callback's integrated-mode token-exchange branch now reuses
the shared discovery cache via get_oidc_discovery (round-2 finding 1).
- AS proxy flow now generates an OIDC nonce in oauth_authorize, stores it
on ASProxySession, forwards it to the IdP, and passes it as
expected_nonce to verify_id_token in _oauth_callback_as_proxy
(round-2 finding 2).
- Consolidate the two parallel discovery caches: oauth_routes' local
_discovery_cache and _get_cached_discovery are removed; all callers
now go through token_utils.get_oidc_discovery, which acquires the
follow_redirects=True knob it needs for Nextcloud installs without
pretty URLs (round-2 finding 3).
- Demote per-user INFO logs in oauth_tools.py (check_logged_in,
get_provisioning_status) to DEBUG; the elicitation auth URL is no
longer logged because it contains a sensitive state token
(round-2 finding 4).
Also pin nonce binding behaviour with a new unit test that asserts
_oauth_callback_as_proxy forwards session.nonce to verify_id_token, and
update test mocks to track the cache consolidation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Blocking:
- AS proxy callback now calls verify_id_token before caching the proxy
code so a tampered IdP response can't smuggle identity claims.
Important:
- Browser OAuth flow generates and verifies an OIDC nonce; new alembic
migration 006 adds the nonce column to oauth_sessions.
- _origin_matches_self logs a warning when CSRF check is bypassed.
- oauth_tools.py uses get_shared_storage instead of fresh handles.
Nits:
- New token_utils.get_oidc_discovery shares the 5-minute cache with
verify_id_token; oauth_login (integrated) and _revoke_refresh_token_at_idp
now use it instead of issuing fresh discovery fetches.
- Drop typing.Optional from oauth_tools.py in favour of X | None.
CI:
- test.yml generates an ephemeral Fernet TOKEN_ENCRYPTION_KEY per run
with openssl, removing the dependency on a missing repo secret.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Addresses the two remaining 🟡 findings from the PR #758 follow-up review:
1. extract_user_id_from_token previously fell back to "default_user" when
the verified access token had no sub claim. In a multi-tenant deployment
a malformed IdP token could have bucketed every request under a single
sentinel user, risking cross-tenant data exposure. The function now
raises McpError on that branch; the BasicAuth no-token sentinel path is
preserved.
2. oauth_callback_nextcloud (Flow 2) read the PKCE code_verifier from
oauth_sessions but never deleted the row, leaving the verifier valid for
the full 10-minute TTL. The row is now deleted eagerly inside the same
branch, mirroring oauth_login_callback in browser_oauth_routes.
Also wires TOKEN_ENCRYPTION_KEY through the docker-compose step in the CI
test workflow so the integration matrix can boot — every job had been
failing fast on the ${TOKEN_ENCRYPTION_KEY:?...} interpolation guard added
in PR #758 finding 5.
Tests pin both fixes (test_token_utils_user_id.py,
test_oauth_callback_session_cleanup.py).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Six findings from the latest claude-bot review on PR #758:
- JWKS cache had no kid-miss refresh path (Medium): on IdP key
rotation every login failed for up to _OIDC_CACHE_TTL. Evict
and refetch once before raising, per OIDC core §10.1.1.
- _should_use_secure_cookies fell back to nextcloud_host scheme,
but the cookie is issued by the MCP server. Switch to
settings.nextcloud_mcp_server_url so split-scheme deployments
get the right Secure flag.
- _origin_matches_self compared raw netloc strings, which include
the port. Browsers omit default ports per RFC 6454 §6.2; an
mcp_server_url like :443 falsely 403'd every legitimate logout.
Normalise (scheme, host, port) tuples with default ports stripped.
- delete_oauth_session exists in storage.py — drop the stale
"we don't have this method" comment and call it eagerly so
replays can't be processed and the table doesn't accumulate
completed-but-not-yet-expired browser-login rows.
- extract_user_id_from_token's unused ctx param renamed to _ctx
to signal "intentionally unused" at the signature level.
- provisioning_decorator instantiated RefreshTokenStorage per
call. Switch to get_shared_storage() for the lock-protected
process-wide singleton.
Plus pre-push self-review catch: lazy-logging on the unchanged
except arm in session_backend.py.
Adds 5 regression tests:
- JWKS rotation: success on refetch
- JWKS rotation: still-missing-kid surfaces original error
- JWKS rotation: network error during refresh wrapped as
IdTokenVerificationError
- default-port CSRF: explicit :443 in config + portless Origin
- default-port CSRF: portless config + explicit :443 in Origin
- scheme-mismatch CSRF: same host, different scheme rejected
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
After merging master, _should_use_secure_cookies was refactored to read
from Settings instead of os.getenv, which dropped `import os` from
browser_oauth_routes.py — leaving _revoke_refresh_token_at_idp's four
remaining os.getenv() calls undefined (CI ruff F821).
Migrate the helper to the same Settings-based pattern:
- oidc_discovery_url → settings.oidc_discovery_url
- OIDC_CLIENT_ID → settings.oidc_client_id
- OIDC_CLIENT_SECRET → settings.oidc_client_secret
- NEXTCLOUD_HOST → settings.nextcloud_host
Drive-by: the previous fallback read OIDC_CLIENT_ID, but the canonical
env var per env.sample / docker-compose is NEXTCLOUD_OIDC_CLIENT_ID.
The Settings layer handles this mapping via dynaconf, so the corrected
name is now used automatically.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Addresses all 9 findings from the review on PR #758:
Blocking:
- _revoke_refresh_token_at_idp now reads config from oauth_ctx["config"]
(the production-shaped nested dict). Previously read flat keys, causing
IdP revocation to silently no-op in production. Test fixtures rebuilt
to the realistic nested shape so the bug can't regress unnoticed.
- HTML error responses in oauth_login_callback now wrap IdP-controlled
error_body, str(e), and the attacker-controlled error/error_description
query params in html_escape. New test_browser_oauth_xss.py pins this.
Important:
- New _safe_next_url helper validates the ?next= query param at write
time (oauth_login), in oauth_logout, and on read from the session row
in oauth_login_callback. Blocks https://, // (protocol-relative), and
CRLF/whitespace injection.
- verify_id_token now caches discovery + JWKS (5-min TTL) using the
same pattern as oauth_routes._get_cached_discovery. New caching
regression test pins to one fetch per URL across multiple calls.
- /oauth/logout is now POST-only at the route layer (defeats passive
CSRF via <img src>). oauth_logout also validates Origin/Referer
against the configured mcp_server_url. Logout UI in user_info.html
converted from <a href> to <form method="post">.
- New storage.cleanup_expired_browser_sessions() called from the hourly
cleanup loop in app.py — previously these rows accumulated for users
who never explicitly logged out.
Nits:
- Demoted INFO logs that leaked oauth_config.keys() / client_id /
token-storage state to DEBUG. Operator-relevant outcome lines
(login successful, refresh token stored, logged out) stay INFO.
- verify_id_token algorithms widened to RS256, PS256, ES256 — covers
Azure AD (PS256) and Cognito/some Keycloak realms (ES256). Symmetric
and "none" remain off the allowlist.
- Migrated all Optional[X] usages in auth/storage.py to X | None per
CLAUDE.md.
Breaking change: GET /oauth/logout now returns 405. The in-tree logout
UI was migrated to a POST form; any external bookmark or curl-based
caller that relied on GET will need to switch.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three review items from the third-round review on PR #757:
- scope_authorization: split the combined logger.warning(error_msg) in
the require_scopes decorator's missing-app-password branch into two
lazy %-style logger calls (one per branch), keeping the f-string
error_msg for the exception only. The else branch also logs the
elicit_result for diagnostics. Bypassing lazy %-interpolation in
security-sensitive code formatted the message regardless of log level
and matched the repo-wide lazy-logging preference; the new code now
conforms.
- config + browser_oauth_routes: wire COOKIE_SECURE through Settings
(cookie_secure: bool | None = None) so _should_use_secure_cookies()
reads it via get_settings() rather than os.getenv. Completes the
consolidation pass that touched this file in commit 7464340 and
removes the last raw os.getenv from browser_oauth_routes.py
(import os dropped). Dynaconf auto-coerces "true"/"false" → bool;
"1"/"0" arrive as int and are normalised by an explicit bool() at
the consumer.
- elicitation: clarify the _astrolabe_settings_url docstring to call
out that the empty-string case is also a None-return path (matches
the existing `if not base:` guard).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pre-launch hardening for the hosted Astrolabe Cloud offering. Addresses
all five findings raised in #626 (Tim Kaufmann, code review of v0.65.0).
Re-verified against master before fixing.
Finding 3 (LLM-controllable user_id) — drop user_id from the public
signatures of provision_nextcloud_access, revoke_nextcloud_access,
check_provisioning_status, check_logged_in. Tool wrappers now always
derive identity from the verified AccessToken; user_id is no longer
accepted as MCP input. Adds parameterized CI-guard test that locks the
schema.
Finding 2 (predictable session cookie) — replace mcp_session=<user_id>
cookie with a cryptographically random session_id mapped server-side
(new browser_sessions table, alembic 005). Cookie value is opaque,
expires, revocable. SessionAuthBackend looks up user_id via the new
mapping and additionally requires a refresh token to fail closed.
Finding 4 (logout doesn't revoke refresh token) — oauth_logout now
calls the IdP revocation_endpoint (RFC 7009) when advertised, deletes
the stored refresh token regardless, and clears the browser_sessions
row. Cleanup is best-effort: logout always 302s.
Finding 1 (unverified ID token decodes) — verify_id_token helper does
JWKS signature + issuer + audience + exp + nonce checks per OIDC core
3.1.3.7. Used by both OAuth callback handlers (browser + MCP). Removes
the four "verify_signature: False" decodes that previously trusted IdP
claims unconditionally. Drops dead-code _validate_token_audience in
token_broker. Refactors token_utils + provisioning_decorator to read
user_id from the verified AccessToken instead of re-decoding the JWT.
Finding 5 (hardcoded Fernet keys in docker-compose.yml) — replace the
three inline TOKEN_ENCRYPTION_KEY values with required env var
interpolation; document in env.sample.
Test coverage: 4 new unit test modules (signature pinning, browser
sessions, ID-token verification, logout + revoke + session backend).
693 unit tests pass; ruff/format/ty clean.
Migration note: existing browser admin-UI sessions become invalid on
rollout (cookies are looked up against the new browser_sessions table,
which starts empty). Users re-login. MCP API access is unaffected.
Tracked on Astrolabe Cloud POC board card #37.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The elicitation flow points users to the Astrolabe web route or the
BasicAuth REST endpoint to provision their app password. Both paths
stored the password without clearing the in-process scope cache, so a
user who provisioned through them would keep hitting
ProvisioningRequiredError for up to _SCOPE_CACHE_TTL (5 min) afterwards.
Add invalidate_scope_cache(user_id) to both write-paths (matching the
existing pattern in nc_auth_check_status), correct the now-misleading
comment in scope_authorization.py to name all three invalidation paths,
and add a one-line hint above the first elicitation patch in the test
file so future authors don't "fix" the patch target to the wrong module.
Addresses PR #757 round-3 review feedback.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Four review items from the second-round review on PR #757:
- scope_authorization: broaden the post-elicit retry message to acknowledge
the 5-minute scope-cache TTL — if the LFv2 poller is still in-flight at
acknowledge-time, the immediate retry can still hit a stale cache.
- elicitation: extract a shared `_run_elicit(ctx, message, schema, *,
log_label)` helper so `present_login_url` and
`present_provisioning_required` no longer duplicate the
hasattr-guard / try-NotImplementedError / try-Exception fallback block.
The data-acknowledged warning specific to login-flow stays in
`present_login_url` so behaviour is preserved exactly.
- elicitation: detect missing http:// / https:// scheme in
`_astrolabe_settings_url`, log a warning, and return None — caller
renders the safe tool-only fallback instead of producing a broken link.
New unit test locks this in.
- browser_oauth_routes: replace the stray
`os.getenv(\"NEXTCLOUD_HOST\")` in `_should_use_secure_cookies` with
`get_settings().nextcloud_host` for consistency with the rest of the
file (PR #757 review nit).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Lift NEXTCLOUD_PUBLIC_ISSUER_URL out of raw os.getenv reads into
Settings.nextcloud_public_issuer_url across all 8 production call sites
(app.py x2, oauth_routes.py x2, browser_oauth_routes.py,
provision_routes.py, userinfo_routes.py, elicitation.py). cli.py
remains the env-write source so the existing config-by-flag pipeline
still works.
Also addresses remaining PR #757 review nits:
- elicitation.py: align URL-present/absent wording on "open in your
browser" so users don't try clicking in the terminal
- test_scope_authorization_stored.py: lock in the deliberately-shared
fall-through branch with explicit declined/cancelled decorator tests
- test_elicitation.py: switch from monkeypatch.setenv to
patch(get_settings) since Settings is now the canonical surface
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Branch the ProvisioningRequiredError message on the elicit result so a
user who acknowledged the prompt isn't told to call
nc_auth_provision_access (which would loop an LLM that just confirmed
via elicitation). Other paths keep the existing instruction.
- Convert present_login_url's f-string logger.warning to lazy %s, matching
present_provisioning_required and the repo's lazy-logging preference.
- Add a test for NEXTCLOUD_PUBLIC_ISSUER_URL trailing-slash normalization.
- Strengthen the decorator-elicits test: split into the "accepted" and
"message_only" branches so the error-message change is regression-tested.
Refs: cbcoutinho/nextcloud-mcp-server#757#issuecomment-4363552487
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When a tool requiring Nextcloud access is called without a stored app
password (Login Flow v2 mode), the @require_scopes decorator now invokes
MCP elicitation with a clickable Astrolabe settings URL — reconstructed
from NEXTCLOUD_PUBLIC_ISSUER_URL / NEXTCLOUD_HOST — before raising
ProvisioningRequiredError. Clients without elicitation support fall back
to the existing text error.
Surfaced by cbcoutinho/nextcloud-mcp-server#752, where users hit a 401
after OAuth and had no clickable URL to start Login Flow v2 from.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Address round-5 reviewer feedback on PR #747:
- Escape `webhook_uri` in the admin pane HTML template so an operator-
controlled env value (`WEBHOOK_INTERNAL_URL`, `NEXTCLOUD_MCP_SERVER_URL`)
can't inject markup. The sibling `preset_id` and exception messages were
already escaped — this one was the odd one out.
- Convert the eight remaining f-string `logger.warning`/`logger.error`
calls in `api/webhooks.py` to lazy `%s` formatting, matching the style
already adopted by `webhook_receiver.py` and `webhook_routes.py`.
- Document why the 401 from `handle_nextcloud_webhook` deliberately omits
`WWW-Authenticate`: NC's webhook delivery worker has no auth-flow state
machine to negotiate against, the bearer is a static shared secret
configured out-of-band via `WEBHOOK_SECRET`, and a challenge response
wouldn't change client behaviour. The existing warning log already
records the rejection.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Address the two Security findings from PR review:
- webhook_receiver: encode Authorization header and expected bearer to
utf-8 bytes before hmac.compare_digest. Conventional form; doesn't
rely on Python's implicit ASCII encoding.
- webhook_routes: html.escape user-influenced and exception-derived
strings before interpolating into HTMLResponse content. Covers the
preset_id path param echoed in the "Unknown preset" branch and the
str(e) text rendered on handler exceptions.
Adds regression tests verifying compare_digest is invoked on bytes and
that <script> payloads (in preset_id and exception messages) are
emitted as escaped entities, not active markup.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Addresses round-3 review feedback on PR #747:
- webhook_receiver: wrap send_stream.send() in anyio.fail_after(1.0)
and return 503 with reason="queue full" if the queue is saturated.
Avoids pinning the handler until NC's outbound timeout fires; the
503 retry contract is the same as the existing "sync not running"
branch.
- webhook_receiver: revise the compare_digest comment to match what
the function actually guarantees — it avoids the per-character
short-circuit of `==` but is not fully constant-time across length
differences.
- _get_webhook_uri: read WEBHOOK_INTERNAL_URL and
NEXTCLOUD_MCP_SERVER_URL via dynaconf so operators using
settings.toml (rather than env vars) aren't silently routed into
the docker/localhost fallback. Adds webhook_internal_url to
Settings/_DEFAULTS/_field_map; nextcloud_mcp_server_url already
existed. Docker-detection markers stay on os.getenv since they're
container-runtime signals, not user-facing config.
- webhook_routes: sweep remaining f-string logger calls to lazy %s
formatting per CLAUDE.md.
- client/webhooks: modernise full file's type hints to
dict / list / | None per CLAUDE.md.
Tests:
- New test_returns_503_when_queue_is_full exercises the timeout
branch with a saturated buffer and a shortened deadline.
- test_webhook_uri tests now patch get_settings (matching the
auth-pair tests in the same file) instead of monkeypatching env
vars directly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- webhook_receiver: always run hmac.compare_digest (drop the
`not provided or` short-circuit) so the constant-time path is
taken regardless of whether the Authorization header is present.
- client/webhooks: modernise the new `auth_data` type hint to
`dict[str, str] | None` per CLAUDE.md.
- tests/client: rename `test_create_webhook_with_auth_headers` →
`test_create_webhook_with_static_headers` and use
`auth_method="header"` (NC's webhook_listeners only supports
"none" and "header"; the previous "bearer" value was invalid).
- auth/webhook_routes: extract `_register_preset_webhooks` from
`enable_webhook_preset` so the auth-threading behaviour is
testable without standing up a Starlette app + auth middleware.
- tests/unit: new test_webhook_routes_register covering the helper
with secret set / unset, and verifying ids round-trip in order.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds optional shared-secret authentication for /webhooks/nextcloud,
addressing the security follow-up flagged in #747.
Behavior:
- WEBHOOK_SECRET set: registrations pass authMethod="header" with
authData={"Authorization": "Bearer <secret>"} (encrypted at-rest in
Nextcloud's DB and forwarded on every delivery). The receiver
validates the same header with hmac.compare_digest before parsing
any payload; missing/invalid → 401.
- WEBHOOK_SECRET unset: registrations stay on authMethod="none" and
the receiver accepts unauthenticated POSTs (logging a one-time
startup warning). Backward compatible — operators can roll out at
their own pace.
Implementation notes:
- WebhooksClient.create_webhook gains an `auth_data` parameter mapped
to NC's `authData` body field; this is distinct from the existing
`headers` parameter (`headers` is plaintext static request headers,
`authData` is encrypted at-rest in NC and only emitted when
authMethod="header"). The previous `auth_method="bearer"` mention in
the docstring was incorrect — NC supports only "none" and "header".
- A small `webhook_auth_pair()` helper in auth/webhook_routes.py
centralises the secret→(auth_method, auth_data) resolution so the
preset flow and the Astrolabe-facing /api/v1/webhooks endpoint stay
in sync.
Also addresses the smaller review points from #747:
- f-string → lazy %s formatting in webhook_receiver.py and
webhook_routes.py.
- Move `int(time)` inside webhook_parser's try/except so a malformed
`time` field returns None instead of raising ValueError.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The /webhooks/nextcloud endpoint was a no-op stub that logged the
payload and returned 200 OK; webhook deletions never reached Qdrant.
Compounding that, _get_webhook_uri() registered the docker-compose
internal hostname (http://mcp:8000) with Nextcloud whenever
/.dockerenv existed — including ECS Fargate — so cloud deployments
were registering a URL NC could not resolve.
- New vector/webhook_parser.py extracts a DocumentTask from
NodeCreatedEvent / NodeWrittenEvent / BeforeNodeDeletedEvent
payloads scoped to */files/Notes/*.md (matching the registered
preset filters).
- New vector/webhook_receiver.py pushes that task onto the same
send-stream the scanner uses (app.state.document_send_stream),
with 503 when sync is not running so NC retries delivery.
- _get_webhook_uri() now prefers NEXTCLOUD_MCP_SERVER_URL over the
/.dockerenv branch, so the explicit public URL set on cloud tasks
wins; docker-compose dev still falls back to the internal name when
no public URL is configured.
Calendar / Tables event parsing is intentionally out of scope here.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Both auth surfaces now fail-closed by default:
- ALLOWED_MCP_CLIENTS: removed the silent `claude-desktop` and
`test-mcp-client` fallbacks. Empty/unset env var leaves the registry
empty so /oauth/authorize rejects every client_id.
- ALLOWED_MGMT_CLIENT (new): comma-separated list of OIDC client_ids
whose tokens are accepted by /api/management/*. Enforced in
verify_token_for_management_api on both the cache-hit and cache-miss
paths against the token's client_id claim. Unset/empty rejects all.
Compose: set ALLOWED_MGMT_CLIENT=nextcloudMcpServerUIPublicClient on
mcp-multi-user-basic so the existing Astrolabe integration test
(test_astrolabe_chunk_context.py) still passes.
env.sample documents both vars and notes they may be consolidated later.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Nextcloud installs without pretty URLs return a 301 from
`/.well-known/openid-configuration` to
`/index.php/.well-known/openid-configuration` (e.g. Hetzner StorageShare).
`_get_cached_discovery` did not enable follow_redirects, so httpx raised
HTTPStatusError on the 301 and the AS-proxy authorize handler returned
500, breaking client connections (e.g. claude.ai).
Pass `follow_redirects=True` to the httpx client used for the discovery
fetch only — downstream OIDC endpoints (token, userinfo, etc.) are
absolute URLs read from the discovery doc and are unaffected.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds 6 MCP tools so an LLM can read a user's Talk conversations and
post messages on their behalf, addressing the "read my chats and reply"
use case from issue #720:
- talk_list_conversations
- talk_get_conversation
- talk_get_messages
- talk_list_participants
- talk_send_message (auto-attaches a referenceId for retry dedup)
- talk_mark_as_read
Edit/delete messages, reactions, threads, and call/session ops are
intentionally out of scope for this first PR.
The TalkClient also exposes create_conversation/delete_conversation
for the integration test fixture; these are not registered as MCP
tools. A post-installation hook enables spreed in the docker dev env
so the integration suite has a real Talk backend to talk to.
Closes#720
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
caldav 3.x lists niquests as a mandatory dependency and prefers it over
httpx. Passing httpx.BasicAuth via the auth= argument breaks under the
niquests backend with "Unexpected non-callable authentication" — see #731.
Switch CalendarClient.__init__ from auth=Auth|None to keyword-only
password/token, and forward them to AsyncDAVClient as password= plus an
explicit auth_type ("basic" or "bearer"). caldav then builds whichever
auth object its active backend needs (niquests.auth.HTTPBasicAuth or
httpx.BasicAuth), so we stay backend-agnostic.
Threaded raw credentials through NextcloudClient — added keyword-only
password/token to its __init__, and updated from_env, from_token, and
the four call sites that build NextcloudClient (context.py basic-auth
and Login Flow paths, auth/userinfo_routes.py, vector/oauth_sync.py).
Four new unit tests pin the construction wiring so the niquests
regression can't recur silently — basic, bearer, no-creds, and
password-precedence cases.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The /api/v1/chunk-context and /api/v1/pdf-preview handlers in
api/visualization.py forwarded the incoming OAuth bearer directly to
Nextcloud via NextcloudClient.from_token. In multi-user BasicAuth mode
Nextcloud has no validator for those bearers on Notes/WebDAV, so it
treats the request as anonymous and returns 401 — surfaced to the user
as a 500 from /apps/astrolabe/api/chunk-context. Search worked because
it only hits Qdrant.
Architecturally, OAuth is only for Astrolabe→MCP server; MCP server→
Nextcloud always uses the per-user app password stored during provision
(background sync already does this via vector.oauth_sync).
- Resolve the Nextcloud client through get_user_client_basic_auth in
both get_chunk_context and get_pdf_preview, surfacing
NotProvisionedError as a clean 401 instead of opaque 500.
- Apply the same fix to the session-cookie variant in
auth/viz_routes.chunk_context_endpoint for the internal viz UI.
Tests:
- New unit file test_management_chunk_context_endpoint.py, including a
regression guard that asserts get_user_client_basic_auth is awaited
(so reverting to from_token fails without needing a live Nextcloud).
- Updated test_management_pdf_preview_endpoint.py to mock the new auth
path (drops extract_bearer_token / NextcloudClient.from_token patches).
- New integration test test_astrolabe_chunk_context.py drives the full
chain (browser → Astrolabe → MCP → Nextcloud) in multi-user BasicAuth
mode, plus bare-bones 401 checks on the MCP endpoint.
Full unit suite: 546 passed.
Companion PR on astrolabe (cbcoutinho/astrolabe#66) sends the Nextcloud
UID as loginName in the app-password POST body so the stored record is
complete. Submodule bump to that branch will follow once CI reproduces
the failure on the old submodule.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two bugs made `uvx --from . nextcloud-mcp-server run` (and any pip install)
unusable outside Docker:
1. Dynaconf was configured with ignore_unknown_envvars=True and relied on
settings.toml to declare the key schema. With no settings.toml in a wheel
install, every env var (NEXTCLOUD_HOST, MCP_DEPLOYMENT_MODE, ...) was
silently dropped. Moved the schema into a Python _DEFAULTS dict passed
directly to Dynaconf, kept settings.toml as an optional external override
(renamed to settings.toml.example, gitignored), and pointed docker-compose
at the example file.
2. Token SQLite DB defaulted to /app/data/tokens.db in multiple places
(auth/storage.py, migrations.py, alembic/env.py, cli.py db subcommands),
which blew up at uvicorn startup with FileNotFoundError on non-Docker
hosts. Replaced with a new config.get_token_db_path() helper that
resolves TOKEN_STORAGE_DB if explicitly set, otherwise allocates a
per-process tempfile cleaned up at interpreter exit via atexit — mirroring
the "ephemeral by default" pattern used for QDRANT_LOCATION=:memory:.
Containers are unaffected: docker-compose services now explicitly set
TOKEN_STORAGE_DB=/app/data/tokens.db (the fourth service that was missing
this pin has been brought in line with the other three).
Verified end-to-end in an isolated /tmp venv: env-var-only startup, Alembic
migrations run against the tempfile, Application startup complete, /health/live
returns 200, tempfile deleted on SIGTERM. Unit tests (464) + ruff + ty pass.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
External IdPs like AWS Cognito return scopes prefixed with the resource
server identifier (e.g. https://mcp.example.com/notes.read). MCP tools
use bare scope names (notes.read) in @require_scopes decorators. Without
stripping the prefix, scope matching fails and only identity-only tools
(openid/profile/email) are visible — resulting in 4/125 tools shown.
Strip the OIDC_RESOURCE_SERVER_ID prefix in both get_access_token_scopes()
(used by list_tools filtering) and the require_scopes decorator (used at
tool execution time).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
AWS Cognito provides refresh tokens automatically with the authorization
code flow but does not list offline_access as a supported scope. Check
the IdP's scopes_supported discovery field before including it in
requests, and always accept refresh tokens from responses regardless.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Flow 2 hardcoded offline_access in the scope string, but providers
like AWS Cognito don't support this scope (they handle refresh tokens
via client config). This caused invalid_scope errors on the Astrolabe
semantic search enablement flow.
Only include offline_access when enable_offline_access is explicitly
set, matching the behavior of DCR scope registration.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
AWS Cognito access tokens do not include an `aud` claim per RFC 7519 —
they use `client_id` instead. This causes `_has_mcp_audience` to reject
all Cognito-issued tokens with "Missing MCP audience. Got []".
When `aud` is empty, fall back to the `client_id` JWT claim for audience
validation. The MCP server's own client_id will be present there since
the AS proxy exchanges the authorization code using its credentials.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Use dynaconf (get_settings()) instead of os.getenv for OIDC_RESOURCE_SERVER_ID
- Re-add Settings field, _field_map entry, and settings.toml default
- Add trailing-slash guard (.rstrip("/")) to prevent double-slash in scopes
- Add double-prefixing guard: skip scopes already carrying the prefix
- Add @pytest.mark.unit to test module
- Add test for already-prefixed scopes
- Document OIDC_RESOURCE_SERVER_ID in docs/configuration.md
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add offline_access to OIDC standard scopes exclusion list to prevent it
from being incorrectly prefixed, which would break Cognito refresh token
flows. Extract scope transformation into testable _transform_scopes_for_idp()
helper, add debug logging for prefixed scopes, remove unused Settings field
(oauth_routes.py consistently uses os.getenv), and add unit tests.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When OIDC_RESOURCE_SERVER_ID is set, prefix resource scopes with the
identifier when forwarding to the IdP (e.g., calendar.read becomes
https://example.com/calendar.read). Required for IdPs like AWS Cognito
that mandate {resource_server_id}/{scope} format for custom scopes.
OIDC standard scopes (openid, profile, email) are forwarded as-is.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
