After restoring the background-sync UI ids, the two indexing-dependent
multi-user-basic tests (chunk_context_uses_app_password, plotly_with_basic_auth)
surfaced a second, previously-masked failure: they provision a user, create a
note, then wait ~90s for it to be indexed.
The mcp-multi-user-basic service ran with the production cadence — scan
interval 60s and the default user-poll interval 60s. A freshly-provisioned
user isn't even *discovered* by the background-sync user manager for up to
60s, leaving too little of the 90s budget for the scan + single-worker
indexing to finish (observed: pending docs still "syncing" at timeout, or the
scanner not yet started → "idle" with 0 indexed).
Match the single-user service's short cadence (5s) and add a matching 5s
user-poll interval so discovery + scan + index complete well within the test
budget. Test-only config; production deployments set their own intervals.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Stop mounting the vendored astrolabe submodule into the Nextcloud `app`
container by default (comment out the /opt/apps/astrolabe bind mount). With
the mount absent, the post-installation hook (20-install-astrolabe-app.sh)
falls through to `occ app:install astrolabe` + `app:enable`, so the dev/CI
stack now exercises the published app-store package rather than a locally
built dev copy. This catches packaging issues (e.g. missing built assets in
the released app) that a source build would mask.
Bump the third_party/astrolabe submodule to v0.16.6, which includes the
background-indexing re-login fix (astrolabe#93).
The dev mount and the CI "Build Astrolabe app" step are retained (commented
mount can be re-enabled locally) so developers can still iterate against the
vendored source on demand.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Astrolabe was refactored to mint session-derived JWTs (TokenGenerationRequest
Event) and a one-click background-indexing opt-in, dropping the OAuth
authorize/callback/refresh surface. Bump the submodule and bring the test
suite in line:
- New test_astrolabe_session_jwt_search.py: a logged-in user searches via the
minted JWT with no provisioning (replaces the obsolete login_flow_provisioning
OAuth-authorize test; token_refresh test deleted — refresh flow is gone).
- settings_buttons: assert the new revoke endpoint + that oauth/disconnect is
gone (404).
- multi_user_background_sync / plotly / chunk_context: drop the OAuth authorize
step; provision via the one-click "Enable background indexing" button
(#mcp-enable-background-button -> #mcp-revoke-background-button) instead of
generating + pasting an app password.
- docker-compose.yml: mount the astrolabe submodule into the app container.
- third_party/astrolabe: bump to the one-click opt-in commit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a `DATABASE_URL` setting that lets `RefreshTokenStorage` run against
any SQLAlchemy async backend, primarily `postgresql+asyncpg://...` for
HA k8s deployments. Default behavior is unchanged: when `DATABASE_URL` is
unset the server falls back to the existing `TOKEN_STORAGE_DB` path /
ephemeral SQLite tempfile.
Why
---
Today every MCP pod needs its own PVC to hold the SQLite file, which
pins the Deployment to one replica and blocks horizontal scaling. With
this change, operators can point all replicas at a shared Postgres
(CNPG, RDS, etc.) and the pods become stateless. Encryption stays in
Python (Fernet); the database only sees ciphertext.
What changed
------------
- `config.get_database_url()` resolves DATABASE_URL → TOKEN_STORAGE_DB →
ephemeral tempfile in that priority order.
- `RefreshTokenStorage` builds a process-shared `AsyncEngine` in
`initialize()`. SQLite gets NullPool; Postgres gets pool_size=10,
max_overflow=20, pool_pre_ping=True. 30 aiosqlite call sites adapted
via a thin `_DBConn` / `_Cursor` / `_Row` / `_ExecuteCtx` shim so
existing method bodies need no churn beyond the connection
context-manager swap.
- 7 `INSERT OR REPLACE` statements rewritten as portable
`INSERT ... ON CONFLICT (...) DO UPDATE` (SQLite ≥ 3.24, Postgres ≥ 9.5).
- `sqlite_master` legacy-detection lookup replaced with SQLAlchemy
inspector so the path works against either backend.
- File-permission hardening + parent-dir creation gated on
`is_sqlite_url(...)` — centralized backends manage their own filesystem.
- Alembic migrations 001/002/003/005 converted from raw `op.execute(SQL)`
to portable `op.create_table()` / `op.create_index()` with SQLAlchemy
types. All timestamp columns are `sa.BigInteger` so Postgres allocates
BIGINT (unix epochs don't fit in INT4). SQLite treats BIGINT as
INTEGER, so existing deployments at revision 006 see no schema drift.
- `migrations.py` + CLI take URLs; `db {upgrade,downgrade,current,history}`
gain `--database-url / -u` alongside the legacy `--database-path / -d`.
`get_current_revision()` uses SQLAlchemy inspector instead of raw
sqlite3, so the CLI works against Postgres too.
- `docker-compose.yml` adds a `postgres-test` service under the
`postgres` profile (pinned `postgres:16-alpine` digest) for
integration testing.
- Unit storage tests parametrized over backends via shared
`tests/fixtures/storage_backend.py` — every test in
`test_app_password_storage.py` and `test_webhook_storage.py` runs
once per backend that is available. Postgres is opted in by
`TEST_DATABASE_URL`.
- New `tests/integration/test_storage_postgres.py` (5 tests, marked
`postgres` + `integration`) covers refresh-token, app-password,
OAuth-session, webhook, and audit-log paths end-to-end on Postgres.
- New `docs/ADR-026-pluggable-database-backend.md` records the decision;
`docs/configuration.md` documents `DATABASE_URL` with examples.
Out of scope
------------
- No SQLite → Postgres data migration tool (clean cutover; tokens reissue
on next login, webhooks re-register on next sync tick).
- This repo does not provision Postgres. The matching helm chart change
lives in cbcoutinho/helm-charts (database.url / existingSecret values).
Verification
------------
- `uv run pytest tests/unit/` — 1012 passed, SQLite path unchanged.
- `docker compose --profile postgres up -d postgres-test`
- `TEST_DATABASE_URL=... uv run pytest tests/integration/test_storage_postgres.py -m postgres -v`
— 5 passed.
- `TEST_DATABASE_URL=... uv run pytest tests/unit/test_app_password_storage.py
tests/unit/test_webhook_storage.py` — 50 passed (25 per backend).
- `uv run ruff check && uv run ruff format --check && uv run ty check -- nextcloud_mcp_server` — clean.
Tracked on Astrolabe Cloud POC board, card #99.
---
_This PR was generated with the help of AI, and reviewed by a Human_
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Same pattern as the ENABLE_LOGIN_FLOW removal in the previous commit:
the deployment mode (MCP_DEPLOYMENT_MODE) is the single source of truth
for selecting an auth flow. The ENABLE_MULTI_USER_BASIC_AUTH env-var
alias is redundant with `MCP_DEPLOYMENT_MODE=multi_user_basic`.
Unlike the ENABLE_LOGIN_FLOW removal — where silent removal was safe
because Login Flow v2 is the auto-detection default — silent removal
here would be a surprise: a user with only ENABLE_MULTI_USER_BASIC_AUTH=true
in their .env would auto-detect into LOGIN_FLOW after upgrade (wrong
runtime mode). Mitigation: detect_auth_mode now reads os.environ
directly for both legacy aliases and raises ValueError with a one-line
migration message if either is set. Applied retroactively to
ENABLE_LOGIN_FLOW as well — loud is better than silent.
- nextcloud_mcp_server/config.py:
- Drop the dynaconf env-var alias entry for ENABLE_MULTI_USER_BASIC_AUTH.
- Update the `enable_multi_user_basic_auth` field docstring to mark it
as derived / not user-settable.
- `_is_multi_user_mode()` (early-config helper, runs before Settings
is built) switched to checking MCP_DEPLOYMENT_MODE directly. Now
consistent with the canonical detection in detect_auth_mode.
- nextcloud_mcp_server/config_validators.py:
- Drop the auto-detection branch (`if settings.enable_multi_user_basic_auth`).
Selection of MULTI_USER_BASIC is now exclusively via the explicit
MCP_DEPLOYMENT_MODE branch.
- Add `enable_multi_user_basic_auth` to `_sync_derived_flags` alongside
`enable_login_flow` — both flags are now derived from the resolved mode.
- Drop `enable_multi_user_basic_auth` from
`MODE_REQUIREMENTS[MULTI_USER_BASIC].required` and from the
`forbidden` lists of SINGLE_USER_BASIC and LOGIN_FLOW (no longer
user input → no meaningful forbidden check).
- Add loud-deprecation `ValueError` block at the top of detect_auth_mode
that errors with a clear migration message when ENABLE_MULTI_USER_BASIC_AUTH
or ENABLE_LOGIN_FLOW is found in os.environ.
- tests/unit/test_config_validators.py:
- Switch ~10 fixtures from `enable_multi_user_basic_auth=True` to
`deployment_mode="multi_user_basic"` (mirrors `enable_login_flow`
treatment from the previous commit).
- Switch two `patch.dict(os.environ, {"ENABLE_MULTI_USER_BASIC_AUTH": "true"})`
blocks to use MCP_DEPLOYMENT_MODE.
- Rename `test_forbidden_multi_user_basic_auth` to
`test_forbidden_multi_user_basic_when_credentials_present` — the
scenario is now an explicit-mode + credentials conflict, not an
env-var-flag conflict.
- Add `test_legacy_enable_multi_user_basic_auth_env_var_errors` and
`test_legacy_enable_login_flow_env_var_errors` to exercise the new
loud-deprecation ValueError path.
- docker-compose.yml: mcp-multi-user-basic profile switched to
`MCP_DEPLOYMENT_MODE=multi_user_basic`.
- env.sample: replaced `#ENABLE_MULTI_USER_BASIC_AUTH=true` example with
`#MCP_DEPLOYMENT_MODE=multi_user_basic`.
- docs/authentication.md, configuration.md, troubleshooting.md,
auth-flows.md, webhook-management-guide.md,
configuration-migration-v2.md, ADR-025: replaced env-var examples
with the canonical MCP_DEPLOYMENT_MODE form.
- docs/ADR-020: marked partly superseded by ADR-022.
- CLAUDE.md: Multi-User BasicAuth section updated to set
MCP_DEPLOYMENT_MODE.
- nextcloud_mcp_server/vector/oauth_sync.py: module docstring updated.
BREAKING CHANGE: ENABLE_MULTI_USER_BASIC_AUTH is no longer read from
the environment, and setting it now raises a startup ValueError with
a migration message. Replace `ENABLE_MULTI_USER_BASIC_AUTH=true` with
`MCP_DEPLOYMENT_MODE=multi_user_basic`. The same loud-deprecation
check is also applied to the recently-removed ENABLE_LOGIN_FLOW —
replace with `MCP_DEPLOYMENT_MODE=login_flow` (or drop both;
`login_flow` is the auto-detect default when no other auth env vars
are set).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Once OAUTH_SINGLE_AUDIENCE was renamed to LOGIN_FLOW and the validation
gate ensured the only meaningful configuration was
`MCP_DEPLOYMENT_MODE=login_flow + ENABLE_LOGIN_FLOW=true`, the two
controls became redundant. Setting the mode is sufficient; the
ENABLE_LOGIN_FLOW env var doesn't add information.
This commit makes the deployment mode the single source of truth for
the Login Flow v2 toggle:
- `nextcloud_mcp_server/config.py`: drop the `ENABLE_LOGIN_FLOW`
dynaconf env-var alias. The `enable_login_flow` field stays as an
internal attribute so the 6 runtime call sites (app.py x4,
context.py, auth/scope_authorization.py) keep working unchanged.
Updated field docstring to flag it as derived.
- `nextcloud_mcp_server/config_validators.py`:
- Drop `enable_login_flow` from `MODE_REQUIREMENTS[LOGIN_FLOW].required`.
- Drop the validation gate that required ENABLE_LOGIN_FLOW=true for
LOGIN_FLOW mode (no longer possible to misconfigure — the flag is
derived, not user input).
- Add `_sync_derived_flags()` helper called at every return path of
`detect_auth_mode` to set `settings.enable_login_flow` from the
resolved mode.
- `tests/unit/test_config_validators.py`: drop `enable_login_flow=True`
from happy-path fixtures (no longer needed — detection sets it).
Repurpose `test_login_flow_requires_enable_login_flow_flag` into
`test_login_flow_mode_auto_derives_enable_login_flow_flag` which
asserts the new auto-derivation behaviour for both LOGIN_FLOW and a
non-LOGIN_FLOW mode.
- `docker-compose.yml`: remove `ENABLE_LOGIN_FLOW=true` from the
`mcp-login-flow` and `mcp-keycloak` profiles.
- `env.sample`: remove the ENABLE_LOGIN_FLOW reference; the comment
on `MCP_DEPLOYMENT_MODE` now notes the derived flag.
- `docs/configuration.md`, `docs/authentication.md`,
`docs/login-flow-v2.md`, `docs/auth-flows.md`,
`docs/troubleshooting.md`, `docs/ADR-025-*.md`: replace
ENABLE_LOGIN_FLOW=true examples and references with
MCP_DEPLOYMENT_MODE=login_flow.
BREAKING CHANGE: `ENABLE_LOGIN_FLOW` is no longer read from the
environment. Anyone who relied on `ENABLE_LOGIN_FLOW=true` to activate
Login Flow v2 should set `MCP_DEPLOYMENT_MODE=login_flow` instead (or
rely on it being the default when no other auth env vars are set).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Follow-up to the LOGIN_FLOW rename. The user-facing surface area —
env.sample, docker-compose.yml mcp-login-flow profile, migration
guide, ADR statuses, and the running.md boot-log examples — all need
to refer to `login_flow` rather than the deprecated
`oauth_single_audience` string.
- docker-compose.yml: add explicit MCP_DEPLOYMENT_MODE=login_flow to
the mcp-login-flow profile (no longer relying on auto-detection).
- env.sample: update the deployment-mode list and example, dropping
the removed `oauth_token_exchange` and pointing at ADR-022 for the
rename rationale.
- docs/ADR-022: flip Status to Accepted with a note that this PR
implements step 1 (rename + validation gate).
- docs/ADR-021: note that it has been partly superseded by ADR-022
(the oauth_single_audience naming is no longer accurate); cross-link.
- docs/ADR-025: drop oauth_single_audience/keycloak from the dynaconf
validator example and the [oauth_single_audience] TOML section.
- docs/configuration-migration-v2.md: bulk-replace oauth_single_audience
→ login_flow throughout (sed -i).
- docs/running.md: re-collapse the per-mode boot-log subsections (added
during the closed PR #786 workaround) back into a uniform
"<mode>"-substitution block — now correct after this PR's logging
cleanup at app.py:1172.
No code changes in this commit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The `mcp-login-flow` profile's `ALLOWED_MGMT_CLIENT` was set to the
test-fixture id `nextcloudMcpServerUIPublicClient` only, but the actual
Astrolabe app provisions its OIDC client as
`astrolabeMcpClientOAuth00000000000` (see
`app-hooks/before-starting/26-configure-astrolabe-oauth.sh:39`). All
tokens issued through the "Enable Semantic Search" flow were rejected
with HTTP 401 by `unified_verifier.py:222-227`'s allowlist check, and
the Astrolabe UI's retry loop subsequently exhausted the
`api/passwords.py` 5/hr rate limit (HTTP 429).
Switch the `mcp-login-flow` allowlist to Astrolabe's client id so
production-shaped traffic actually validates. The `mcp-multi-user-basic`
profile keeps `nextcloudMcpServerUIPublicClient` for the
`configure_astrolabe_for_mcp_server` test fixture.
Also bump `third_party/astrolabe` 0.13.12 → 0.14.0 to pull in the
chunk-context indexed-lookup fix (#75) and the PDF bbox highlight
overlay (#76) that match the master-side changes already merged on
this branch.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Astrolabe webhooks UI hits /api/v1/apps on the MCP server, which
forwarded the OAuth bearer token to /ocs/v1.php/cloud/apps?filter=enabled.
That OCS endpoint is admin-only AND @PasswordConfirmationRequired —
neither requirement is satisfiable via an OAuth bearer token, so even an
admin user's token returns a silent 401 (no entry in nextcloud.log).
Switch to /ocs/v2.php/cloud/capabilities, which has no admin or password-
confirmation gate, accepts the existing bearer token, and returns a
capabilities map keyed by app id (notes, files, tables, forms, etc.).
This is sufficient for the webhook presets UI to gate available presets
against the running Nextcloud instance's enabled apps.
Bearer is preserved on the outbound call because anonymous capabilities
omits notes/tables/forms — only authenticated capabilities exposes them.
Tests:
- New unit test covers the regression (asserts /ocs/v2.php/cloud/capabilities
is hit, NOT /cloud/apps), response parsing, sanitized error messages,
and missing-config paths.
- New integration test under tests/server/login_flow/ drives a real
OAuth flow against mcp-login-flow with a static OIDC client
(nextcloudMcpServerUIPublicClient) and asserts /api/v1/apps returns 200
with core/files in the response.
docker-compose.yml: aligns mcp-login-flow's ALLOWED_MGMT_CLIENT with
mcp-multi-user-basic so the same static-client test fixture works for both.
Follow-up to homelab-argocd #1608, which set ALLOWED_MGMT_CLIENT in
production but didn't unblock the webhooks flow.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pre-launch hardening for the hosted Astrolabe Cloud offering. Addresses
all five findings raised in #626 (Tim Kaufmann, code review of v0.65.0).
Re-verified against master before fixing.
Finding 3 (LLM-controllable user_id) — drop user_id from the public
signatures of provision_nextcloud_access, revoke_nextcloud_access,
check_provisioning_status, check_logged_in. Tool wrappers now always
derive identity from the verified AccessToken; user_id is no longer
accepted as MCP input. Adds parameterized CI-guard test that locks the
schema.
Finding 2 (predictable session cookie) — replace mcp_session=<user_id>
cookie with a cryptographically random session_id mapped server-side
(new browser_sessions table, alembic 005). Cookie value is opaque,
expires, revocable. SessionAuthBackend looks up user_id via the new
mapping and additionally requires a refresh token to fail closed.
Finding 4 (logout doesn't revoke refresh token) — oauth_logout now
calls the IdP revocation_endpoint (RFC 7009) when advertised, deletes
the stored refresh token regardless, and clears the browser_sessions
row. Cleanup is best-effort: logout always 302s.
Finding 1 (unverified ID token decodes) — verify_id_token helper does
JWKS signature + issuer + audience + exp + nonce checks per OIDC core
3.1.3.7. Used by both OAuth callback handlers (browser + MCP). Removes
the four "verify_signature: False" decodes that previously trusted IdP
claims unconditionally. Drops dead-code _validate_token_audience in
token_broker. Refactors token_utils + provisioning_decorator to read
user_id from the verified AccessToken instead of re-decoding the JWT.
Finding 5 (hardcoded Fernet keys in docker-compose.yml) — replace the
three inline TOKEN_ENCRYPTION_KEY values with required env var
interpolation; document in env.sample.
Test coverage: 4 new unit test modules (signature pinning, browser
sessions, ID-token verification, logout + revoke + session backend).
693 unit tests pass; ruff/format/ty clean.
Migration note: existing browser admin-UI sessions become invalid on
rollout (cookies are looked up against the new browser_sessions table,
which starts empty). Users re-login. MCP API access is unaffected.
Tracked on Astrolabe Cloud POC board card #37.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Both auth surfaces now fail-closed by default:
- ALLOWED_MCP_CLIENTS: removed the silent `claude-desktop` and
`test-mcp-client` fallbacks. Empty/unset env var leaves the registry
empty so /oauth/authorize rejects every client_id.
- ALLOWED_MGMT_CLIENT (new): comma-separated list of OIDC client_ids
whose tokens are accepted by /api/management/*. Enforced in
verify_token_for_management_api on both the cache-hit and cache-miss
paths against the token's client_id claim. Unset/empty rejects all.
Compose: set ALLOWED_MGMT_CLIENT=nextcloudMcpServerUIPublicClient on
mcp-multi-user-basic so the existing Astrolabe integration test
(test_astrolabe_chunk_context.py) still passes.
env.sample documents both vars and notes they may be consolidated later.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two bugs made `uvx --from . nextcloud-mcp-server run` (and any pip install)
unusable outside Docker:
1. Dynaconf was configured with ignore_unknown_envvars=True and relied on
settings.toml to declare the key schema. With no settings.toml in a wheel
install, every env var (NEXTCLOUD_HOST, MCP_DEPLOYMENT_MODE, ...) was
silently dropped. Moved the schema into a Python _DEFAULTS dict passed
directly to Dynaconf, kept settings.toml as an optional external override
(renamed to settings.toml.example, gitignored), and pointed docker-compose
at the example file.
2. Token SQLite DB defaulted to /app/data/tokens.db in multiple places
(auth/storage.py, migrations.py, alembic/env.py, cli.py db subcommands),
which blew up at uvicorn startup with FileNotFoundError on non-Docker
hosts. Replaced with a new config.get_token_db_path() helper that
resolves TOKEN_STORAGE_DB if explicitly set, otherwise allocates a
per-process tempfile cleaned up at interpreter exit via atexit — mirroring
the "ephemeral by default" pattern used for QDRANT_LOCATION=:memory:.
Containers are unaffected: docker-compose services now explicitly set
TOKEN_STORAGE_DB=/app/data/tokens.db (the fourth service that was missing
this pin has been brought in line with the other three).
Verified end-to-end in an isolated /tmp venv: env-var-only startup, Alembic
migrations run against the tempfile, Application startup complete, /health/live
returns 200, tempfile deleted on SIGTERM. Unit tests (464) + ruff + ty pass.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The root_path for dynaconf resolved to site-packages instead of the
application root when installed non-editable (Docker). This caused all
settings without env var overrides to be None, crashing on startup with
a TypeError in chunk size validation.
Fix root_path to fall back to CWD when settings.toml isn't at the
source-tree path, and refactor get_settings() to only pass values
dynaconf actually has — letting Settings dataclass defaults apply for
unconfigured keys. Mount settings.toml into all docker-compose MCP
services as a read-only volume.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Many identity providers (AWS Cognito, Okta, Azure AD) reject or mishandle
colons in OAuth scope names. This migrates all custom scopes from
`resource:action` to `resource.action` format (e.g., `notes:read` →
`notes.read`), which is universally accepted and aligns with industry
conventions (Microsoft, Google).
Includes Alembic migration 004 for stored scope strings and ADR-024
documenting the rationale and RFC references.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Merge ALLOWED_MCP_CLOUD_CLIENTS into a single ALLOWED_MCP_CLIENTS env var
that supports both simple client IDs and pipe-separated client_id|redirect_uri
entries. Enforce HTTPS for non-localhost redirect URIs, warn on malformed
entries, and use wildcard scopes for all static clients (upstream IdP enforces
actual scopes). Add deprecation warning for the old env var.
Also fixes DCR proxy error messages to reference only ALLOWED_MCP_CLIENTS and
use "Upstream" instead of "Nextcloud" for IdP-agnostic language. Enables
Login Flow v2 + DCR on the mcp-keycloak docker-compose service.
Adds 17 unit tests for ClientRegistry parsing/validation and 7 keycloak
integration tests for DCR lifecycle, AS metadata, and client authorization.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
