Files
mcp-nextcloud/nextcloud_mcp_server/cli.py
T
Chris CoutinhoandClaude Opus 4.7 292cbb3292 feat(storage): pluggable database backend via DATABASE_URL (ADR-026)
Adds a `DATABASE_URL` setting that lets `RefreshTokenStorage` run against
any SQLAlchemy async backend, primarily `postgresql+asyncpg://...` for
HA k8s deployments. Default behavior is unchanged: when `DATABASE_URL` is
unset the server falls back to the existing `TOKEN_STORAGE_DB` path /
ephemeral SQLite tempfile.

Why
---
Today every MCP pod needs its own PVC to hold the SQLite file, which
pins the Deployment to one replica and blocks horizontal scaling. With
this change, operators can point all replicas at a shared Postgres
(CNPG, RDS, etc.) and the pods become stateless. Encryption stays in
Python (Fernet); the database only sees ciphertext.

What changed
------------
- `config.get_database_url()` resolves DATABASE_URL → TOKEN_STORAGE_DB →
  ephemeral tempfile in that priority order.
- `RefreshTokenStorage` builds a process-shared `AsyncEngine` in
  `initialize()`. SQLite gets NullPool; Postgres gets pool_size=10,
  max_overflow=20, pool_pre_ping=True. 30 aiosqlite call sites adapted
  via a thin `_DBConn` / `_Cursor` / `_Row` / `_ExecuteCtx` shim so
  existing method bodies need no churn beyond the connection
  context-manager swap.
- 7 `INSERT OR REPLACE` statements rewritten as portable
  `INSERT ... ON CONFLICT (...) DO UPDATE` (SQLite ≥ 3.24, Postgres ≥ 9.5).
- `sqlite_master` legacy-detection lookup replaced with SQLAlchemy
  inspector so the path works against either backend.
- File-permission hardening + parent-dir creation gated on
  `is_sqlite_url(...)` — centralized backends manage their own filesystem.
- Alembic migrations 001/002/003/005 converted from raw `op.execute(SQL)`
  to portable `op.create_table()` / `op.create_index()` with SQLAlchemy
  types. All timestamp columns are `sa.BigInteger` so Postgres allocates
  BIGINT (unix epochs don't fit in INT4). SQLite treats BIGINT as
  INTEGER, so existing deployments at revision 006 see no schema drift.
- `migrations.py` + CLI take URLs; `db {upgrade,downgrade,current,history}`
  gain `--database-url / -u` alongside the legacy `--database-path / -d`.
  `get_current_revision()` uses SQLAlchemy inspector instead of raw
  sqlite3, so the CLI works against Postgres too.
- `docker-compose.yml` adds a `postgres-test` service under the
  `postgres` profile (pinned `postgres:16-alpine` digest) for
  integration testing.
- Unit storage tests parametrized over backends via shared
  `tests/fixtures/storage_backend.py` — every test in
  `test_app_password_storage.py` and `test_webhook_storage.py` runs
  once per backend that is available. Postgres is opted in by
  `TEST_DATABASE_URL`.
- New `tests/integration/test_storage_postgres.py` (5 tests, marked
  `postgres` + `integration`) covers refresh-token, app-password,
  OAuth-session, webhook, and audit-log paths end-to-end on Postgres.
- New `docs/ADR-026-pluggable-database-backend.md` records the decision;
  `docs/configuration.md` documents `DATABASE_URL` with examples.

Out of scope
------------
- No SQLite → Postgres data migration tool (clean cutover; tokens reissue
  on next login, webhooks re-register on next sync tick).
- This repo does not provision Postgres. The matching helm chart change
  lives in cbcoutinho/helm-charts (database.url / existingSecret values).

Verification
------------
- `uv run pytest tests/unit/` — 1012 passed, SQLite path unchanged.
- `docker compose --profile postgres up -d postgres-test`
- `TEST_DATABASE_URL=... uv run pytest tests/integration/test_storage_postgres.py -m postgres -v`
  — 5 passed.
- `TEST_DATABASE_URL=... uv run pytest tests/unit/test_app_password_storage.py
  tests/unit/test_webhook_storage.py` — 50 passed (25 per backend).
- `uv run ruff check && uv run ruff format --check && uv run ty check -- nextcloud_mcp_server` — clean.

Tracked on Astrolabe Cloud POC board, card #99.

---

_This PR was generated with the help of AI, and reviewed by a Human_

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-16 18:06:42 +02:00

491 lines
16 KiB
Python

import os
from importlib.metadata import version
import click
import uvicorn
from nextcloud_mcp_server.config import (
get_database_url,
get_settings,
is_ephemeral_token_db,
)
from nextcloud_mcp_server.migrations import (
create_migration,
downgrade_database,
get_current_revision,
show_migration_history,
upgrade_database,
)
from nextcloud_mcp_server.observability import get_uvicorn_logging_config
from nextcloud_mcp_server.server import AVAILABLE_APPS
from .app import get_app
@click.command()
@click.option(
"--host", "-h", default="127.0.0.1", show_default=True, help="Server host"
)
@click.option(
"--port", "-p", type=int, default=8000, show_default=True, help="Server port"
)
@click.option(
"--log-level",
"-l",
default="info",
show_default=True,
type=click.Choice(["critical", "error", "warning", "info", "debug", "trace"]),
help="Logging level",
)
@click.option(
"--transport",
"-t",
default="streamable-http",
show_default=True,
type=click.Choice(["streamable-http", "http", "stdio"]),
help="MCP transport protocol",
)
@click.option(
"--enable-app",
"-e",
multiple=True,
type=click.Choice(sorted(AVAILABLE_APPS.keys())),
help="Enable specific Nextcloud app APIs. Can be specified multiple times. If not specified, all apps are enabled.",
)
@click.option(
"--oauth/--no-oauth",
default=None,
help="Force OAuth mode (if enabled) or BasicAuth mode (if disabled). By default, auto-detected based on environment variables.",
)
@click.option(
"--oauth-client-id",
envvar="NEXTCLOUD_OIDC_CLIENT_ID",
help="OAuth client ID (can also use NEXTCLOUD_OIDC_CLIENT_ID env var)",
)
@click.option(
"--oauth-client-secret",
envvar="NEXTCLOUD_OIDC_CLIENT_SECRET",
help="OAuth client secret (can also use NEXTCLOUD_OIDC_CLIENT_SECRET env var)",
)
@click.option(
"--mcp-server-url",
envvar="NEXTCLOUD_MCP_SERVER_URL",
default="http://localhost:8000",
show_default=True,
help="MCP server URL for OAuth callbacks (can also use NEXTCLOUD_MCP_SERVER_URL env var)",
)
@click.option(
"--nextcloud-host",
envvar="NEXTCLOUD_HOST",
help="Nextcloud instance URL (can also use NEXTCLOUD_HOST env var)",
)
@click.option(
"--nextcloud-username",
envvar="NEXTCLOUD_USERNAME",
help="Nextcloud username for BasicAuth (can also use NEXTCLOUD_USERNAME env var)",
)
@click.option(
"--nextcloud-password",
envvar="NEXTCLOUD_PASSWORD",
help="Nextcloud password for BasicAuth (can also use NEXTCLOUD_PASSWORD env var)",
)
@click.option(
"--oauth-scopes",
envvar="NEXTCLOUD_OIDC_SCOPES",
default="openid profile email notes.read notes.write calendar.read calendar.write todo.read todo.write contacts.read contacts.write cookbook.read cookbook.write deck.read deck.write tables.read tables.write files.read files.write sharing.read sharing.write",
show_default=True,
help="OAuth scopes to request during client registration. These define the maximum allowed scopes for the client. Note: Actual supported scopes are discovered dynamically from MCP tools at runtime. (can also use NEXTCLOUD_OIDC_SCOPES env var)",
)
@click.option(
"--oauth-token-type",
envvar="NEXTCLOUD_OIDC_TOKEN_TYPE",
default="bearer",
show_default=True,
type=click.Choice(["bearer", "jwt"], case_sensitive=False),
help="OAuth token type (can also use NEXTCLOUD_OIDC_TOKEN_TYPE env var)",
)
@click.option(
"--public-issuer-url",
envvar="NEXTCLOUD_PUBLIC_ISSUER_URL",
help="Public issuer URL for OAuth (can also use NEXTCLOUD_PUBLIC_ISSUER_URL env var)",
)
def run(
host: str,
port: int,
log_level: str,
transport: str,
enable_app: tuple[str, ...],
oauth: bool | None,
oauth_client_id: str | None,
oauth_client_secret: str | None,
mcp_server_url: str,
nextcloud_host: str | None,
nextcloud_username: str | None,
nextcloud_password: str | None,
oauth_scopes: str,
oauth_token_type: str,
public_issuer_url: str | None,
):
"""
Run the Nextcloud MCP server.
\b
Authentication Modes:
- BasicAuth: Set NEXTCLOUD_USERNAME and NEXTCLOUD_PASSWORD
- OAuth: Leave USERNAME/PASSWORD unset (requires OIDC app enabled)
\b
Examples:
# BasicAuth mode with CLI options
$ nextcloud-mcp-server --nextcloud-host=https://cloud.example.com \\
--nextcloud-username=admin --nextcloud-password=secret
# BasicAuth mode with env vars (recommended for credentials)
$ export NEXTCLOUD_HOST=https://cloud.example.com
$ export NEXTCLOUD_USERNAME=admin
$ export NEXTCLOUD_PASSWORD=secret
$ nextcloud-mcp-server --host 0.0.0.0 --port 8000
# OAuth mode with auto-registration
$ nextcloud-mcp-server --nextcloud-host=https://cloud.example.com --oauth
# OAuth mode with pre-configured client
$ nextcloud-mcp-server --nextcloud-host=https://cloud.example.com --oauth \\
--oauth-client-id=xxx --oauth-client-secret=yyy
# OAuth mode with custom scopes and JWT tokens
$ nextcloud-mcp-server --nextcloud-host=https://cloud.example.com --oauth \\
--oauth-scopes="openid notes.read notes.write" --oauth-token-type=jwt
# OAuth with public issuer URL (for Docker/proxy setups)
$ nextcloud-mcp-server --nextcloud-host=http://app --oauth \\
--public-issuer-url=http://localhost:8080
# stdio transport for local use (e.g. Claude Code)
$ nextcloud-mcp-server run --transport stdio
"""
# Set env vars from CLI options if provided
if nextcloud_host:
os.environ["NEXTCLOUD_HOST"] = nextcloud_host
if nextcloud_username:
os.environ["NEXTCLOUD_USERNAME"] = nextcloud_username
if nextcloud_password:
os.environ["NEXTCLOUD_PASSWORD"] = nextcloud_password
if oauth_client_id:
os.environ["NEXTCLOUD_OIDC_CLIENT_ID"] = oauth_client_id
if oauth_client_secret:
os.environ["NEXTCLOUD_OIDC_CLIENT_SECRET"] = oauth_client_secret
if oauth_scopes:
os.environ["NEXTCLOUD_OIDC_SCOPES"] = oauth_scopes
if oauth_token_type:
os.environ["NEXTCLOUD_OIDC_TOKEN_TYPE"] = oauth_token_type
if mcp_server_url:
os.environ["NEXTCLOUD_MCP_SERVER_URL"] = mcp_server_url
if public_issuer_url:
os.environ["NEXTCLOUD_PUBLIC_ISSUER_URL"] = public_issuer_url
# Force OAuth mode if explicitly requested
if oauth is True:
# Clear username/password to force OAuth mode
if "NEXTCLOUD_USERNAME" in os.environ:
click.echo(
"Warning: --oauth flag set, ignoring NEXTCLOUD_USERNAME", err=True
)
del os.environ["NEXTCLOUD_USERNAME"]
if "NEXTCLOUD_PASSWORD" in os.environ:
click.echo(
"Warning: --oauth flag set, ignoring NEXTCLOUD_PASSWORD", err=True
)
del os.environ["NEXTCLOUD_PASSWORD"]
# Validate OAuth configuration
nextcloud_host = os.getenv("NEXTCLOUD_HOST")
if not nextcloud_host:
raise click.ClickException(
"OAuth mode requires NEXTCLOUD_HOST environment variable to be set"
)
# Check if we have client credentials OR if dynamic registration is possible
has_client_creds = os.getenv("NEXTCLOUD_OIDC_CLIENT_ID") and os.getenv(
"NEXTCLOUD_OIDC_CLIENT_SECRET"
)
if not has_client_creds:
# No client credentials - will attempt dynamic registration
# Show helpful message before server starts
click.echo("", err=True)
click.echo("OAuth Configuration:", err=True)
click.echo(" Mode: Dynamic Client Registration", err=True)
click.echo(" Host: " + nextcloud_host, err=True)
click.echo(" Storage: SQLite (TOKEN_STORAGE_DB)", err=True)
click.echo("", err=True)
click.echo(
"Note: Make sure 'Dynamic Client Registration' is enabled", err=True
)
click.echo(" in your Nextcloud OIDC app settings.", err=True)
click.echo("", err=True)
else:
click.echo("", err=True)
click.echo("OAuth Configuration:", err=True)
click.echo(" Mode: Pre-configured Client", err=True)
click.echo(" Host: " + nextcloud_host, err=True)
click.echo(
" Client ID: "
+ os.getenv("NEXTCLOUD_OIDC_CLIENT_ID", "")[:16]
+ "...",
err=True,
)
click.echo("", err=True)
elif oauth is False:
# Force BasicAuth mode - verify credentials exist
if not os.getenv("NEXTCLOUD_USERNAME") or not os.getenv("NEXTCLOUD_PASSWORD"):
raise click.ClickException(
"--no-oauth flag set but NEXTCLOUD_USERNAME or NEXTCLOUD_PASSWORD not set"
)
enabled_apps = list(enable_app) if enable_app else None
if transport == "stdio":
if oauth is True:
raise click.ClickException(
"stdio transport does not support OAuth mode. "
"Use single-user BasicAuth with NEXTCLOUD_HOST, "
"NEXTCLOUD_USERNAME, and NEXTCLOUD_PASSWORD."
)
from .stdio import get_stdio_mcp # noqa: PLC0415
try:
mcp = get_stdio_mcp(enabled_apps=enabled_apps)
except ValueError as e:
raise click.ClickException(str(e)) from e
mcp.run(transport="stdio")
return
app = get_app(transport=transport, enabled_apps=enabled_apps)
# Get observability settings and create uvicorn logging config
settings = get_settings()
uvicorn_log_config = get_uvicorn_logging_config(
log_format=settings.log_format,
log_level=settings.log_level,
include_trace_context=settings.log_include_trace_context,
)
uvicorn.run(
app=app,
host=host,
port=port,
log_level=log_level,
log_config=uvicorn_log_config,
)
@click.group()
def db():
"""Database migration management commands."""
pass
def _resolve_db_url(database_url: str | None, database_path: str | None) -> str:
"""Pick the database URL for a CLI subcommand.
Priority: explicit ``--database-url`` > legacy ``--database-path``
(treated as a SQLite file) > :func:`get_database_url` (honors
``DATABASE_URL`` env or falls back to the ephemeral SQLite tempfile).
"""
if database_url:
return database_url
if database_path:
return f"sqlite+aiosqlite:///{database_path}"
return get_database_url()
def _warn_if_ephemeral(database_url: str) -> None:
"""Warn when the resolved URL is the per-process SQLite tempfile."""
if not database_url.startswith(
"sqlite+aiosqlite:///"
) and not database_url.startswith("sqlite:///"):
return
path = database_url.split("///", 1)[1]
if is_ephemeral_token_db(path):
click.echo(
click.style(
f"⚠ Using ephemeral tempfile {path}; changes "
"will be lost on exit. Pass --database-url / --database-path "
"or set DATABASE_URL / TOKEN_STORAGE_DB to operate on a "
"persistent database.",
fg="yellow",
),
err=True,
)
def _db_target_options(fn):
"""Attach the shared ``--database-url`` / ``--database-path`` options.
Using a decorator factory rather than ``**kwargs`` dict-expansion so
static type checkers (ty) see ``click.option`` called with literal
keyword arguments, which is the only form it's typed to accept.
"""
fn = click.option(
"--database-path",
"-d",
envvar="TOKEN_STORAGE_DB",
default=None,
help="SQLite database file path. Equivalent to "
"--database-url sqlite+aiosqlite:///<path>.",
)(fn)
fn = click.option(
"--database-url",
"-u",
envvar="DATABASE_URL",
default=None,
help="SQLAlchemy URL (e.g. postgresql+asyncpg://...). Wins over --database-path.",
)(fn)
return fn
@db.command()
@_db_target_options
@click.option(
"--revision",
"-r",
default="head",
show_default=True,
help="Target revision (default: head for latest)",
)
def upgrade(database_url: str | None, database_path: str | None, revision: str):
"""Upgrade database to a specific revision.
\b
Examples:
# Upgrade to latest version
$ nextcloud-mcp-server db upgrade
# Upgrade a Postgres backend
$ nextcloud-mcp-server db upgrade -u postgresql+asyncpg://mcp:mcp@db/mcp
# Use custom SQLite path
$ nextcloud-mcp-server db upgrade -d /path/to/tokens.db
"""
url = _resolve_db_url(database_url, database_path)
_warn_if_ephemeral(url)
try:
click.echo(f"Upgrading database to revision: {revision}")
upgrade_database(url, revision)
click.echo(click.style("✓ Database upgraded successfully", fg="green"))
except Exception as e:
click.echo(click.style(f"✗ Upgrade failed: {e}", fg="red"), err=True)
raise click.ClickException(str(e))
@db.command()
@_db_target_options
@click.option(
"--revision",
"-r",
default="-1",
show_default=True,
help="Target revision (default: -1 for previous version)",
)
@click.confirmation_option(
prompt="Are you sure you want to downgrade the database? This may result in data loss."
)
def downgrade(database_url: str | None, database_path: str | None, revision: str):
"""Downgrade database to a specific revision.
WARNING: This may result in data loss! Use with caution.
"""
url = _resolve_db_url(database_url, database_path)
_warn_if_ephemeral(url)
try:
click.echo(f"Downgrading database to revision: {revision}")
downgrade_database(url, revision)
click.echo(click.style("✓ Database downgraded successfully", fg="green"))
except Exception as e:
click.echo(click.style(f"✗ Downgrade failed: {e}", fg="red"), err=True)
raise click.ClickException(str(e))
@db.command()
@_db_target_options
def current(database_url: str | None, database_path: str | None):
"""Show current database revision."""
url = _resolve_db_url(database_url, database_path)
_warn_if_ephemeral(url)
try:
revision = get_current_revision(url)
if revision:
click.echo(f"Current revision: {click.style(revision, fg='cyan')}")
else:
click.echo(
click.style(
"Database is not versioned (no alembic_version table)", fg="yellow"
)
)
except Exception as e:
click.echo(
click.style(f"✗ Failed to get current revision: {e}", fg="red"), err=True
)
raise click.ClickException(str(e))
@db.command()
@_db_target_options
def history(database_url: str | None, database_path: str | None):
"""Show migration history."""
url = _resolve_db_url(database_url, database_path)
_warn_if_ephemeral(url)
try:
click.echo("Migration history:")
show_migration_history(url)
except Exception as e:
click.echo(click.style(f"✗ Failed to show history: {e}", fg="red"), err=True)
raise click.ClickException(str(e))
@db.command()
@click.argument("message")
def migrate(message: str):
"""Create a new migration script (developers only).
The MESSAGE argument describes the changes in this migration.
\b
Examples:
$ nextcloud-mcp-server db migrate "add user preferences table"
$ nextcloud-mcp-server db migrate "add index on refresh_tokens.user_id"
Note: You must manually edit the generated migration file to add SQL statements.
"""
try:
click.echo(f"Creating new migration: {message}")
create_migration(message)
click.echo(click.style("✓ Migration created successfully", fg="green"))
click.echo(
"Edit the migration file in alembic/versions/ to add upgrade/downgrade SQL."
)
except Exception as e:
click.echo(
click.style(f"✗ Failed to create migration: {e}", fg="red"), err=True
)
raise click.ClickException(str(e))
# Create CLI group with subcommands
@click.group()
@click.version_option(
version=version("nextcloud-mcp-server"), prog_name="nextcloud-mcp-server"
)
def cli():
pass
cli.add_command(run)
cli.add_command(db)
if __name__ == "__main__":
cli()