admin/mcp-nextcloud
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(storage): pluggable database backend via DATABASE_URL (ADR-026)

Browse Source 
Adds a `DATABASE_URL` setting that lets `RefreshTokenStorage` run against
any SQLAlchemy async backend, primarily `postgresql+asyncpg://...` for
HA k8s deployments. Default behavior is unchanged: when `DATABASE_URL` is
unset the server falls back to the existing `TOKEN_STORAGE_DB` path /
ephemeral SQLite tempfile.

Why
---
Today every MCP pod needs its own PVC to hold the SQLite file, which
pins the Deployment to one replica and blocks horizontal scaling. With
this change, operators can point all replicas at a shared Postgres
(CNPG, RDS, etc.) and the pods become stateless. Encryption stays in
Python (Fernet); the database only sees ciphertext.

What changed
------------
- `config.get_database_url()` resolves DATABASE_URL → TOKEN_STORAGE_DB →
  ephemeral tempfile in that priority order.
- `RefreshTokenStorage` builds a process-shared `AsyncEngine` in
  `initialize()`. SQLite gets NullPool; Postgres gets pool_size=10,
  max_overflow=20, pool_pre_ping=True. 30 aiosqlite call sites adapted
  via a thin `_DBConn` / `_Cursor` / `_Row` / `_ExecuteCtx` shim so
  existing method bodies need no churn beyond the connection
  context-manager swap.
- 7 `INSERT OR REPLACE` statements rewritten as portable
  `INSERT ... ON CONFLICT (...) DO UPDATE` (SQLite ≥ 3.24, Postgres ≥ 9.5).
- `sqlite_master` legacy-detection lookup replaced with SQLAlchemy
  inspector so the path works against either backend.
- File-permission hardening + parent-dir creation gated on
  `is_sqlite_url(...)` — centralized backends manage their own filesystem.
- Alembic migrations 001/002/003/005 converted from raw `op.execute(SQL)`
  to portable `op.create_table()` / `op.create_index()` with SQLAlchemy
  types. All timestamp columns are `sa.BigInteger` so Postgres allocates
  BIGINT (unix epochs don't fit in INT4). SQLite treats BIGINT as
  INTEGER, so existing deployments at revision 006 see no schema drift.
- `migrations.py` + CLI take URLs; `db {upgrade,downgrade,current,history}`
  gain `--database-url / -u` alongside the legacy `--database-path / -d`.
  `get_current_revision()` uses SQLAlchemy inspector instead of raw
  sqlite3, so the CLI works against Postgres too.
- `docker-compose.yml` adds a `postgres-test` service under the
  `postgres` profile (pinned `postgres:16-alpine` digest) for
  integration testing.
- Unit storage tests parametrized over backends via shared
  `tests/fixtures/storage_backend.py` — every test in
  `test_app_password_storage.py` and `test_webhook_storage.py` runs
  once per backend that is available. Postgres is opted in by
  `TEST_DATABASE_URL`.
- New `tests/integration/test_storage_postgres.py` (5 tests, marked
  `postgres` + `integration`) covers refresh-token, app-password,
  OAuth-session, webhook, and audit-log paths end-to-end on Postgres.
- New `docs/ADR-026-pluggable-database-backend.md` records the decision;
  `docs/configuration.md` documents `DATABASE_URL` with examples.

Out of scope
------------
- No SQLite → Postgres data migration tool (clean cutover; tokens reissue
  on next login, webhooks re-register on next sync tick).
- This repo does not provision Postgres. The matching helm chart change
  lives in cbcoutinho/helm-charts (database.url / existingSecret values).

Verification
------------
- `uv run pytest tests/unit/` — 1012 passed, SQLite path unchanged.
- `docker compose --profile postgres up -d postgres-test`
- `TEST_DATABASE_URL=... uv run pytest tests/integration/test_storage_postgres.py -m postgres -v`
  — 5 passed.
- `TEST_DATABASE_URL=... uv run pytest tests/unit/test_app_password_storage.py
  tests/unit/test_webhook_storage.py` — 50 passed (25 per backend).
- `uv run ruff check && uv run ruff format --check && uv run ty check -- nextcloud_mcp_server` — clean.

Tracked on Astrolabe Cloud POC board, card #99.

---

_This PR was generated with the help of AI, and reviewed by a Human_

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Chris Coutinho
2026-05-16 18:06:42 +02:00
co-authored by Claude Opus 4.7
parent 15a7e680a6
commit 292cbb3292
19 changed files with 1402 additions and 588 deletions
Download Patch File Download Diff File Expand all files Collapse all files
nextcloud_mcp_server/alembic/versions/20251217_2200_001_initial_schema.py
+105 -140
View File
@@ -8,12 +8,19 @@ This migration creates the initial database schema including:
- registered_webhooks: Webhook registration tracking (both OAuth and BasicAuth)
- schema_version: Legacy schema version tracking (deprecated, use alembic_version)
Uses Alembic's portable schema-DDL helpers (``op.create_table`` /
``op.create_index``) with SQLAlchemy types so the DDL is emitted correctly
for both SQLite (BLOB / INTEGER PRIMARY KEY AUTOINCREMENT) and Postgres
(BYTEA / SERIAL). See ADR-026.
Revision ID: 001
Revises:
Create Date: 2025-12-17 22:00:00.000000
"""
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
@@ -24,143 +31,104 @@ depends_on = None
def upgrade() -> None:
"""Create initial database schema."""
"""Create initial database schema.
# Refresh tokens table (OAuth mode only, for background jobs)
op.execute(
"""
CREATE TABLE IF NOT EXISTS refresh_tokens (
user_id TEXT PRIMARY KEY,
encrypted_token BLOB NOT NULL,
expires_at INTEGER,
created_at INTEGER NOT NULL,
updated_at INTEGER NOT NULL,
-- ADR-004 Progressive Consent fields
flow_type TEXT DEFAULT 'hybrid',
token_audience TEXT DEFAULT 'nextcloud',
provisioned_at INTEGER,
provisioning_client_id TEXT,
scopes TEXT,
-- Browser session profile cache
user_profile TEXT,
profile_cached_at INTEGER
)
"""
All ``*_at`` / expiration / timestamp columns use :class:`sa.BigInteger`
so Postgres allocates BIGINT (8-byte) and unix epoch values don't
overflow the 32-bit int32 INTEGER range on long-lived sessions. SQLite
treats BIGINT and INTEGER identically (dynamic typing), so this is
backwards compatible.
"""
op.create_table(
"refresh_tokens",
sa.Column("user_id", sa.Text, primary_key=True),
sa.Column("encrypted_token", sa.LargeBinary, nullable=False),
sa.Column("expires_at", sa.BigInteger),
sa.Column("created_at", sa.BigInteger, nullable=False),
sa.Column("updated_at", sa.BigInteger, nullable=False),
# ADR-004 Progressive Consent fields
sa.Column("flow_type", sa.Text, server_default="hybrid"),
sa.Column("token_audience", sa.Text, server_default="nextcloud"),
sa.Column("provisioned_at", sa.BigInteger),
sa.Column("provisioning_client_id", sa.Text),
sa.Column("scopes", sa.Text),
# Browser session profile cache
sa.Column("user_profile", sa.Text),
sa.Column("profile_cached_at", sa.BigInteger),
)
# Audit logs table (both OAuth and BasicAuth modes)
op.execute(
"""
CREATE TABLE IF NOT EXISTS audit_logs (
id INTEGER PRIMARY KEY AUTOINCREMENT,
timestamp INTEGER NOT NULL,
event TEXT NOT NULL,
user_id TEXT NOT NULL,
resource_type TEXT,
resource_id TEXT,
auth_method TEXT,
hostname TEXT
)
"""
op.create_table(
"audit_logs",
sa.Column("id", sa.Integer, primary_key=True, autoincrement=True),
sa.Column("timestamp", sa.BigInteger, nullable=False),
sa.Column("event", sa.Text, nullable=False),
sa.Column("user_id", sa.Text, nullable=False),
sa.Column("resource_type", sa.Text),
sa.Column("resource_id", sa.Text),
sa.Column("auth_method", sa.Text),
sa.Column("hostname", sa.Text),
)
op.create_index("idx_audit_user_timestamp", "audit_logs", ["user_id", "timestamp"])
op.create_table(
"oauth_clients",
sa.Column("id", sa.Integer, primary_key=True, autoincrement=False),
sa.Column("client_id", sa.Text, nullable=False, unique=True),
sa.Column("encrypted_client_secret", sa.LargeBinary, nullable=False),
sa.Column("client_id_issued_at", sa.BigInteger, nullable=False),
sa.Column("client_secret_expires_at", sa.BigInteger, nullable=False),
sa.Column("redirect_uris", sa.Text, nullable=False),
sa.Column("encrypted_registration_access_token", sa.LargeBinary),
sa.Column("registration_client_uri", sa.Text),
sa.Column("created_at", sa.BigInteger, nullable=False),
sa.Column("updated_at", sa.BigInteger, nullable=False),
)
# Index on audit logs for efficient queries
op.execute(
"""
CREATE INDEX IF NOT EXISTS idx_audit_user_timestamp
ON audit_logs(user_id, timestamp)
"""
op.create_table(
"oauth_sessions",
sa.Column("session_id", sa.Text, primary_key=True),
sa.Column("client_id", sa.Text),
sa.Column("client_redirect_uri", sa.Text, nullable=False),
sa.Column("state", sa.Text),
sa.Column("code_challenge", sa.Text),
sa.Column("code_challenge_method", sa.Text),
sa.Column("mcp_authorization_code", sa.Text, unique=True),
sa.Column("idp_access_token", sa.Text),
sa.Column("idp_refresh_token", sa.Text),
sa.Column("user_id", sa.Text),
sa.Column("created_at", sa.BigInteger, nullable=False),
sa.Column("expires_at", sa.BigInteger, nullable=False),
# ADR-004 Progressive Consent fields
sa.Column("flow_type", sa.Text, server_default="hybrid"),
sa.Column("requested_scopes", sa.Text),
sa.Column("granted_scopes", sa.Text),
sa.Column("is_provisioning", sa.Boolean, server_default=sa.false()),
)
op.create_index(
"idx_oauth_sessions_mcp_code",
"oauth_sessions",
["mcp_authorization_code"],
)
# OAuth client credentials storage (OAuth mode only)
op.execute(
"""
CREATE TABLE IF NOT EXISTS oauth_clients (
id INTEGER PRIMARY KEY,
client_id TEXT UNIQUE NOT NULL,
encrypted_client_secret BLOB NOT NULL,
client_id_issued_at INTEGER NOT NULL,
client_secret_expires_at INTEGER NOT NULL,
redirect_uris TEXT NOT NULL,
encrypted_registration_access_token BLOB,
registration_client_uri TEXT,
created_at INTEGER NOT NULL,
updated_at INTEGER NOT NULL
)
"""
# Legacy schema-version table; superseded by alembic_version. Retained
# so pre-Alembic databases that get stamped into the migration chain
# still match the schema fingerprint they had on disk.
op.create_table(
"schema_version",
sa.Column("version", sa.Integer, primary_key=True, autoincrement=False),
sa.Column("applied_at", sa.Float, nullable=False),
)
# OAuth flow sessions (ADR-004 Progressive Consent)
op.execute(
"""
CREATE TABLE IF NOT EXISTS oauth_sessions (
session_id TEXT PRIMARY KEY,
client_id TEXT,
client_redirect_uri TEXT NOT NULL,
state TEXT,
code_challenge TEXT,
code_challenge_method TEXT,
mcp_authorization_code TEXT UNIQUE,
idp_access_token TEXT,
idp_refresh_token TEXT,
user_id TEXT,
created_at INTEGER NOT NULL,
expires_at INTEGER NOT NULL,
-- ADR-004 Progressive Consent fields
flow_type TEXT DEFAULT 'hybrid',
requested_scopes TEXT,
granted_scopes TEXT,
is_provisioning BOOLEAN DEFAULT FALSE
)
"""
)
# Index for MCP authorization code lookups
op.execute(
"""
CREATE INDEX IF NOT EXISTS idx_oauth_sessions_mcp_code
ON oauth_sessions(mcp_authorization_code)
"""
)
# Legacy schema version tracking table
# NOTE: This is deprecated in favor of Alembic's alembic_version table
# Kept for backward compatibility with pre-Alembic databases
op.execute(
"""
CREATE TABLE IF NOT EXISTS schema_version (
version INTEGER PRIMARY KEY,
applied_at REAL NOT NULL
)
"""
)
# Registered webhooks tracking (both BasicAuth and OAuth modes)
op.execute(
"""
CREATE TABLE IF NOT EXISTS registered_webhooks (
id INTEGER PRIMARY KEY AUTOINCREMENT,
webhook_id INTEGER NOT NULL UNIQUE,
preset_id TEXT NOT NULL,
created_at REAL NOT NULL
)
"""
)
# Indexes for efficient webhook queries
op.execute(
"""
CREATE INDEX IF NOT EXISTS idx_webhooks_preset
ON registered_webhooks(preset_id)
"""
)
op.execute(
"""
CREATE INDEX IF NOT EXISTS idx_webhooks_created
ON registered_webhooks(created_at)
"""
op.create_table(
"registered_webhooks",
sa.Column("id", sa.Integer, primary_key=True, autoincrement=True),
sa.Column("webhook_id", sa.Integer, nullable=False, unique=True),
sa.Column("preset_id", sa.Text, nullable=False),
sa.Column("created_at", sa.Float, nullable=False),
)
op.create_index("idx_webhooks_preset", "registered_webhooks", ["preset_id"])
op.create_index("idx_webhooks_created", "registered_webhooks", ["created_at"])
def downgrade() -> None:
@@ -170,16 +138,13 @@ def downgrade() -> None:
Use with extreme caution.
"""
# Drop indexes first
op.execute("DROP INDEX IF EXISTS idx_webhooks_created")
op.execute("DROP INDEX IF EXISTS idx_webhooks_preset")
op.execute("DROP INDEX IF EXISTS idx_oauth_sessions_mcp_code")
op.execute("DROP INDEX IF EXISTS idx_audit_user_timestamp")
# Drop tables
op.execute("DROP TABLE IF EXISTS registered_webhooks")
op.execute("DROP TABLE IF EXISTS schema_version")
op.execute("DROP TABLE IF EXISTS oauth_sessions")
op.execute("DROP TABLE IF EXISTS oauth_clients")
op.execute("DROP TABLE IF EXISTS audit_logs")
op.execute("DROP TABLE IF EXISTS refresh_tokens")
op.drop_index("idx_webhooks_created", table_name="registered_webhooks")
op.drop_index("idx_webhooks_preset", table_name="registered_webhooks")
op.drop_table("registered_webhooks")
op.drop_table("schema_version")
op.drop_index("idx_oauth_sessions_mcp_code", table_name="oauth_sessions")
op.drop_table("oauth_sessions")
op.drop_table("oauth_clients")
op.drop_index("idx_audit_user_timestamp", table_name="audit_logs")
op.drop_table("audit_logs")
op.drop_table("refresh_tokens")
nextcloud_mcp_server/alembic/versions/20260113_1200_002_add_app_passwords.py
+12 -20
View File
@@ -10,6 +10,8 @@ Create Date: 2026-01-13 12:00:00.000000
"""
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
@@ -22,29 +24,19 @@ depends_on = None
def upgrade() -> None:
"""Add app_passwords table for multi-user BasicAuth mode."""
# App passwords table for multi-user BasicAuth background sync
op.execute(
"""
CREATE TABLE IF NOT EXISTS app_passwords (
user_id TEXT PRIMARY KEY,
encrypted_password BLOB NOT NULL,
created_at INTEGER NOT NULL,
updated_at INTEGER NOT NULL
)
"""
)
# Index for efficient user lookups
op.execute(
"""
CREATE INDEX IF NOT EXISTS idx_app_passwords_updated
ON app_passwords(updated_at)
"""
op.create_table(
"app_passwords",
sa.Column("user_id", sa.Text, primary_key=True),
sa.Column("encrypted_password", sa.LargeBinary, nullable=False),
# BigInteger to keep unix epochs in range on Postgres (see 001).
sa.Column("created_at", sa.BigInteger, nullable=False),
sa.Column("updated_at", sa.BigInteger, nullable=False),
)
op.create_index("idx_app_passwords_updated", "app_passwords", ["updated_at"])
def downgrade() -> None:
"""Drop app_passwords table."""
op.execute("DROP INDEX IF EXISTS idx_app_passwords_updated")
op.execute("DROP TABLE IF EXISTS app_passwords")
op.drop_index("idx_app_passwords_updated", table_name="app_passwords")
op.drop_table("app_passwords")
nextcloud_mcp_server/alembic/versions/20260227_1200_003_add_scopes_and_login_flow.py
+28 -61
View File
@@ -12,6 +12,8 @@ Create Date: 2026-02-27 12:00:00.000000
"""
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
@@ -24,72 +26,37 @@ depends_on = None
def upgrade() -> None:
"""Add scopes/username to app_passwords and create login_flow_sessions."""
# Add scopes column (nullable JSON array, NULL = all scopes allowed)
op.execute(
"""
ALTER TABLE app_passwords ADD COLUMN scopes TEXT
"""
)
# Nullable scope columns on the existing app_passwords table.
op.add_column("app_passwords", sa.Column("scopes", sa.Text))
op.add_column("app_passwords", sa.Column("username", sa.Text))
# Add username column (Nextcloud loginName from Login Flow v2)
op.execute(
"""
ALTER TABLE app_passwords ADD COLUMN username TEXT
"""
op.create_table(
"login_flow_sessions",
sa.Column("user_id", sa.Text, primary_key=True),
sa.Column("encrypted_poll_token", sa.LargeBinary, nullable=False),
sa.Column("poll_endpoint", sa.Text, nullable=False),
sa.Column("requested_scopes", sa.Text),
# BigInteger to keep unix epochs in range on Postgres (see 001).
sa.Column("created_at", sa.BigInteger, nullable=False),
sa.Column("expires_at", sa.BigInteger, nullable=False),
)
# Login Flow v2 session tracking
op.execute(
"""
CREATE TABLE IF NOT EXISTS login_flow_sessions (
user_id TEXT PRIMARY KEY,
encrypted_poll_token BLOB NOT NULL,
poll_endpoint TEXT NOT NULL,
requested_scopes TEXT,
created_at INTEGER NOT NULL,
expires_at INTEGER NOT NULL
)
"""
)
# Index for efficient cleanup of expired sessions
op.execute(
"""
CREATE INDEX IF NOT EXISTS idx_login_flow_sessions_expires
ON login_flow_sessions(expires_at)
"""
op.create_index(
"idx_login_flow_sessions_expires",
"login_flow_sessions",
["expires_at"],
)
def downgrade() -> None:
"""Drop login_flow_sessions and remove added columns."""
"""Drop login_flow_sessions and remove added columns.
op.execute("DROP INDEX IF EXISTS idx_login_flow_sessions_expires")
op.execute("DROP TABLE IF EXISTS login_flow_sessions")
``batch_alter_table`` handles SQLite's pre-3.35 lack of ``DROP COLUMN``
by recreating the table; on Postgres it issues a native ``DROP COLUMN``.
"""
# SQLite doesn't support DROP COLUMN before 3.35.0
# Recreate app_passwords without the new columns
op.execute(
"""
CREATE TABLE app_passwords_backup (
user_id TEXT PRIMARY KEY,
encrypted_password BLOB NOT NULL,
created_at INTEGER NOT NULL,
updated_at INTEGER NOT NULL
)
"""
)
op.execute(
"""
INSERT INTO app_passwords_backup (user_id, encrypted_password, created_at, updated_at)
SELECT user_id, encrypted_password, created_at, updated_at FROM app_passwords
"""
)
op.execute("DROP TABLE app_passwords")
op.execute("ALTER TABLE app_passwords_backup RENAME TO app_passwords")
op.execute(
"""
CREATE INDEX IF NOT EXISTS idx_app_passwords_updated
ON app_passwords(updated_at)
"""
)
op.drop_index("idx_login_flow_sessions_expires", table_name="login_flow_sessions")
op.drop_table("login_flow_sessions")
with op.batch_alter_table("app_passwords") as batch_op:
batch_op.drop_column("username")
batch_op.drop_column("scopes")
nextcloud_mcp_server/alembic/versions/20260502_1500_005_add_browser_sessions.py
+14 -24
View File
@@ -10,6 +10,8 @@ Revises: 004
Create Date: 2026-05-02 15:00:00.000000
"""
import sqlalchemy as sa
from alembic import op
revision = "005"
@@ -19,31 +21,19 @@ depends_on = None
def upgrade() -> None:
op.execute(
"""
CREATE TABLE IF NOT EXISTS browser_sessions (
session_id TEXT PRIMARY KEY,
user_id TEXT NOT NULL,
created_at INTEGER NOT NULL,
expires_at INTEGER NOT NULL
)
"""
)
op.execute(
"""
CREATE INDEX IF NOT EXISTS idx_browser_sessions_user
ON browser_sessions(user_id)
"""
)
op.execute(
"""
CREATE INDEX IF NOT EXISTS idx_browser_sessions_expires
ON browser_sessions(expires_at)
"""
op.create_table(
"browser_sessions",
sa.Column("session_id", sa.Text, primary_key=True),
sa.Column("user_id", sa.Text, nullable=False),
# BigInteger to keep unix epochs in range on Postgres (see 001).
sa.Column("created_at", sa.BigInteger, nullable=False),
sa.Column("expires_at", sa.BigInteger, nullable=False),
)
op.create_index("idx_browser_sessions_user", "browser_sessions", ["user_id"])
op.create_index("idx_browser_sessions_expires", "browser_sessions", ["expires_at"])
def downgrade() -> None:
op.execute("DROP INDEX IF EXISTS idx_browser_sessions_expires")
op.execute("DROP INDEX IF EXISTS idx_browser_sessions_user")
op.execute("DROP TABLE IF EXISTS browser_sessions")
op.drop_index("idx_browser_sessions_expires", table_name="browser_sessions")
op.drop_index("idx_browser_sessions_user", table_name="browser_sessions")
op.drop_table("browser_sessions")
nextcloud_mcp_server/auth/storage.py
+454 -164
View File
File diff suppressed because it is too large Load Diff
nextcloud_mcp_server/cli.py
+77 -76
View File
@@ -5,8 +5,8 @@ import click
import uvicorn
from nextcloud_mcp_server.config import (
get_database_url,
get_settings,
get_token_db_path,
is_ephemeral_token_db,
)
from nextcloud_mcp_server.migrations import (
@@ -287,27 +287,67 @@ def db():
pass
def _warn_if_ephemeral(database_path: str) -> None:
if is_ephemeral_token_db(database_path):
def _resolve_db_url(database_url: str | None, database_path: str | None) -> str:
"""Pick the database URL for a CLI subcommand.
Priority: explicit ``--database-url`` > legacy ``--database-path``
(treated as a SQLite file) > :func:`get_database_url` (honors
``DATABASE_URL`` env or falls back to the ephemeral SQLite tempfile).
"""
if database_url:
return database_url
if database_path:
return f"sqlite+aiosqlite:///{database_path}"
return get_database_url()
def _warn_if_ephemeral(database_url: str) -> None:
"""Warn when the resolved URL is the per-process SQLite tempfile."""
if not database_url.startswith(
"sqlite+aiosqlite:///"
) and not database_url.startswith("sqlite:///"):
return
path = database_url.split("///", 1)[1]
if is_ephemeral_token_db(path):
click.echo(
click.style(
f"⚠ Using ephemeral tempfile {database_path}; changes "
"will be lost on exit. Pass --database-path or set "
"TOKEN_STORAGE_DB to operate on a persistent database.",
f"⚠ Using ephemeral tempfile {path}; changes "
"will be lost on exit. Pass --database-url / --database-path "
"or set DATABASE_URL / TOKEN_STORAGE_DB to operate on a "
"persistent database.",
fg="yellow",
),
err=True,
)
def _db_target_options(fn):
"""Attach the shared ``--database-url`` / ``--database-path`` options.
Using a decorator factory rather than ``**kwargs`` dict-expansion so
static type checkers (ty) see ``click.option`` called with literal
keyword arguments, which is the only form it's typed to accept.
"""
fn = click.option(
"--database-path",
"-d",
envvar="TOKEN_STORAGE_DB",
default=None,
help="SQLite database file path. Equivalent to "
"--database-url sqlite+aiosqlite:///<path>.",
)(fn)
fn = click.option(
"--database-url",
"-u",
envvar="DATABASE_URL",
default=None,
help="SQLAlchemy URL (e.g. postgresql+asyncpg://...). Wins over --database-path.",
)(fn)
return fn
@db.command()
@click.option(
"--database-path",
"-d",
envvar="TOKEN_STORAGE_DB",
default=None,
help="Path to token storage database (can also use TOKEN_STORAGE_DB env var)",
)
@_db_target_options
@click.option(
"--revision",
"-r",
@@ -315,7 +355,7 @@ def _warn_if_ephemeral(database_path: str) -> None:
show_default=True,
help="Target revision (default: head for latest)",
)
def upgrade(database_path: str | None, revision: str):
def upgrade(database_url: str | None, database_path: str | None, revision: str):
"""Upgrade database to a specific revision.
\b
@@ -323,17 +363,17 @@ def upgrade(database_path: str | None, revision: str):
# Upgrade to latest version
$ nextcloud-mcp-server db upgrade
# Upgrade to specific revision
$ nextcloud-mcp-server db upgrade --revision 001
# Upgrade a Postgres backend
$ nextcloud-mcp-server db upgrade -u postgresql+asyncpg://mcp:mcp@db/mcp
# Use custom database path
# Use custom SQLite path
$ nextcloud-mcp-server db upgrade -d /path/to/tokens.db
"""
database_path = database_path or get_token_db_path()
_warn_if_ephemeral(database_path)
url = _resolve_db_url(database_url, database_path)
_warn_if_ephemeral(url)
try:
click.echo(f"Upgrading database to revision: {revision}")
upgrade_database(database_path, revision)
upgrade_database(url, revision)
click.echo(click.style("✓ Database upgraded successfully", fg="green"))
except Exception as e:
click.echo(click.style(f"✗ Upgrade failed: {e}", fg="red"), err=True)
@@ -341,13 +381,7 @@ def upgrade(database_path: str | None, revision: str):
@db.command()
@click.option(
"--database-path",
"-d",
envvar="TOKEN_STORAGE_DB",
default=None,
help="Path to token storage database",
)
@_db_target_options
@click.option(
"--revision",
"-r",
@@ -358,27 +392,16 @@ def upgrade(database_path: str | None, revision: str):
@click.confirmation_option(
prompt="Are you sure you want to downgrade the database? This may result in data loss."
)
def downgrade(database_path: str | None, revision: str):
def downgrade(database_url: str | None, database_path: str | None, revision: str):
"""Downgrade database to a specific revision.
WARNING: This may result in data loss! Use with caution.
\b
Examples:
# Downgrade by one version
$ nextcloud-mcp-server db downgrade
# Downgrade to specific revision
$ nextcloud-mcp-server db downgrade --revision 001
# Downgrade to base (empty database)
$ nextcloud-mcp-server db downgrade --revision base
"""
database_path = database_path or get_token_db_path()
_warn_if_ephemeral(database_path)
url = _resolve_db_url(database_url, database_path)
_warn_if_ephemeral(url)
try:
click.echo(f"Downgrading database to revision: {revision}")
downgrade_database(database_path, revision)
downgrade_database(url, revision)
click.echo(click.style("✓ Database downgraded successfully", fg="green"))
except Exception as e:
click.echo(click.style(f"✗ Downgrade failed: {e}", fg="red"), err=True)
@@ -386,24 +409,13 @@ def downgrade(database_path: str | None, revision: str):
@db.command()
@click.option(
"--database-path",
"-d",
envvar="TOKEN_STORAGE_DB",
default=None,
help="Path to token storage database",
)
def current(database_path: str | None):
"""Show current database revision.
\b
Example:
$ nextcloud-mcp-server db current
"""
database_path = database_path or get_token_db_path()
_warn_if_ephemeral(database_path)
@_db_target_options
def current(database_url: str | None, database_path: str | None):
"""Show current database revision."""
url = _resolve_db_url(database_url, database_path)
_warn_if_ephemeral(url)
try:
revision = get_current_revision(database_path)
revision = get_current_revision(url)
if revision:
click.echo(f"Current revision: {click.style(revision, fg='cyan')}")
else:
@@ -420,25 +432,14 @@ def current(database_path: str | None):
@db.command()
@click.option(
"--database-path",
"-d",
envvar="TOKEN_STORAGE_DB",
default=None,
help="Path to token storage database",
)
def history(database_path: str | None):
"""Show migration history.
\b
Example:
$ nextcloud-mcp-server db history
"""
database_path = database_path or get_token_db_path()
_warn_if_ephemeral(database_path)
@_db_target_options
def history(database_url: str | None, database_path: str | None):
"""Show migration history."""
url = _resolve_db_url(database_url, database_path)
_warn_if_ephemeral(url)
try:
click.echo("Migration history:")
show_migration_history(database_path)
show_migration_history(url)
except Exception as e:
click.echo(click.style(f"✗ Failed to show history: {e}", fg="red"), err=True)
raise click.ClickException(str(e))
nextcloud_mcp_server/config.py
+28
View File
@@ -56,6 +56,10 @@ _DEFAULTS: dict[str, Any] = {
# None = ephemeral per-process tempfile (see get_token_db_path()).
# Set TOKEN_STORAGE_DB to persist tokens across restarts.
"token_storage_db": None,
# Centralized backend (any SQLAlchemy URL). Wins over TOKEN_STORAGE_DB
# when set. Use postgresql+asyncpg://user:pw@host/db for HA k8s
# deployments so pods can be stateless. See ADR-026.
"database_url": None,
# Webhook delivery authentication (ADR-010): when set, registrations
# tell NC to add `Authorization: Bearer <secret>` to webhook deliveries
# and the receiver rejects unauthenticated requests.
@@ -279,6 +283,30 @@ def is_ephemeral_token_db(path: str) -> bool:
return path == _ephemeral_db_path
def get_database_url() -> str:
"""Resolve the SQLAlchemy database URL for token storage.
Priority:
1. ``DATABASE_URL`` if set — any SQLAlchemy URL is accepted; the primary
supported backends are ``postgresql+asyncpg://...`` for HA k8s
deployments and ``sqlite+aiosqlite:///...`` for development.
2. Otherwise build ``sqlite+aiosqlite:///{get_token_db_path()}`` so the
legacy ``TOKEN_STORAGE_DB`` env var and the ephemeral-tempfile
fallback both keep working unchanged.
"""
explicit = _dynaconf.get("DATABASE_URL")
if explicit:
return str(explicit)
return f"sqlite+aiosqlite:///{get_token_db_path()}"
def is_sqlite_url(url: str) -> bool:
"""Return True for SQLite SQLAlchemy URLs (used to gate sqlite-only logic
like file-permission hardening and ``sqlite_master`` legacy lookups).
"""
return url.startswith("sqlite")
LOGGING_CONFIG = {
"version": 1,
"disable_existing_loggers": False,
nextcloud_mcp_server/migrations.py
+76 -90
View File
@@ -3,22 +3,49 @@
This module provides helper functions for managing Alembic database migrations
programmatically. It enables automatic migration on application startup and
provides CLI integration.
All helpers accept a SQLAlchemy URL (``sqlite+aiosqlite:///...`` or
``postgresql+asyncpg://...``). When called without an explicit URL they fall
back to :func:`nextcloud_mcp_server.config.get_database_url`.
"""
import logging
import sqlite3
from pathlib import Path
from alembic.config import Config
from sqlalchemy import create_engine, inspect, text
import nextcloud_mcp_server.alembic as alembic_package
from alembic import command
from nextcloud_mcp_server.config import get_token_db_path
from nextcloud_mcp_server.config import get_database_url
logger = logging.getLogger(__name__)
def get_alembic_config(database_path: str | Path | None = None) -> Config:
def _coerce_url(database_url: str | Path | None) -> str:
"""Accept either a URL string, a Path (legacy SQLite path), or None.
A bare ``Path`` is interpreted as a SQLite database file for backward
compatibility with the prior path-based API.
"""
if database_url is None:
return get_database_url()
if isinstance(database_url, Path):
return f"sqlite+aiosqlite:///{database_url.resolve()}"
return database_url
def _to_sync_url(database_url: str) -> str:
"""Map an async driver URL to its sync equivalent for blocking inspection.
SQLAlchemy's :func:`inspect` and :func:`create_engine` used below are
synchronous APIs. The runtime uses async drivers (``aiosqlite``,
``asyncpg``) but Alembic and these utility queries don't need them.
"""
return database_url.replace("+aiosqlite", "").replace("+asyncpg", "")
def get_alembic_config(database_url: str | Path | None = None) -> Config:
"""
Get Alembic configuration for programmatic use.
@@ -26,145 +53,102 @@ def get_alembic_config(database_path: str | Path | None = None) -> Config:
package location instead of alembic.ini file.
Args:
database_path: Path to SQLite database file. If None, resolves via
config.get_token_db_path() (ephemeral tempfile unless
TOKEN_STORAGE_DB is set).
database_url: SQLAlchemy URL. If None, resolves via
:func:`get_database_url` (DATABASE_URL env var, falling back
to the ephemeral SQLite tempfile under ``TOKEN_STORAGE_DB``).
For backward compatibility a ``Path`` is treated as a SQLite
file path.
Returns:
Alembic Config object configured for the specified database
Alembic Config object configured for the resolved URL.
"""
# Use package location (works in both editable and installed modes)
if alembic_package.__file__ is None:
raise RuntimeError("alembic package __file__ is None")
script_location = Path(alembic_package.__file__).parent
# Create config programmatically (no alembic.ini needed at runtime)
config = Config()
config.set_main_option("script_location", str(script_location))
config.set_main_option("path_separator", "os") # Suppress deprecation warning
config.set_main_option("path_separator", "os")
# Set database URL
if database_path:
db_path = Path(database_path).resolve()
else:
db_path = Path(get_token_db_path()).resolve()
url = f"sqlite+aiosqlite:///{db_path}"
url = _coerce_url(database_url)
config.set_main_option("sqlalchemy.url", url)
logger.debug("Alembic script location: %s", script_location)
logger.debug("Database: %s", db_path)
logger.debug("Database URL: %s", url)
return config
def upgrade_database(
database_path: str | Path | None = None, revision: str = "head"
database_url: str | Path | None = None, revision: str = "head"
) -> None:
"""
Upgrade database to a specific revision.
Args:
database_path: Path to SQLite database file
revision: Target revision (default: "head" for latest)
"""
config = get_alembic_config(database_path)
"""Upgrade database to a specific revision (default: latest)."""
config = get_alembic_config(database_url)
logger.info("Upgrading database to revision: %s", revision)
command.upgrade(config, revision)
logger.info("Database upgrade completed successfully")
def downgrade_database(
database_path: str | Path | None = None, revision: str = "-1"
database_url: str | Path | None = None, revision: str = "-1"
) -> None:
"""
Downgrade database to a specific revision.
Args:
database_path: Path to SQLite database file
revision: Target revision (default: "-1" for previous version)
"""
config = get_alembic_config(database_path)
"""Downgrade database to a specific revision (default: previous)."""
config = get_alembic_config(database_url)
logger.warning("Downgrading database to revision: %s", revision)
command.downgrade(config, revision)
logger.info("Database downgrade completed successfully")
def get_current_revision(database_path: str | Path | None = None) -> str | None:
def get_current_revision(database_url: str | Path | None = None) -> str | None:
"""
Get the current database revision by directly querying the alembic_version table.
Get the current database revision by reading the ``alembic_version`` table.
Args:
database_path: Path to SQLite database file
Returns:
Current revision ID or None if not versioned
Returns ``None`` when the database does not exist or has no
``alembic_version`` table (i.e. has never been migrated).
"""
url = _to_sync_url(_coerce_url(database_url))
if database_path is None:
database_path = get_token_db_path()
db_path = Path(database_path).resolve()
if not db_path.exists():
logger.debug("Database does not exist: %s", db_path)
return None
try:
# Query alembic_version table directly
conn = sqlite3.connect(str(db_path))
cursor = conn.cursor()
# Check if alembic_version table exists
cursor.execute(
"SELECT name FROM sqlite_master WHERE type='table' AND name='alembic_version'"
)
has_table = cursor.fetchone() is not None
if not has_table:
conn.close()
if url.startswith("sqlite:///"):
path = url[len("sqlite:///") :]
if path and not Path(path).exists():
logger.debug("Database does not exist: %s", path)
return None
# Get current version
cursor.execute("SELECT version_num FROM alembic_version")
row = cursor.fetchone()
conn.close()
return row[0] if row else None
try:
engine = create_engine(url, future=True)
try:
inspector = inspect(engine)
if not inspector.has_table("alembic_version"):
return None
with engine.connect() as conn:
row = conn.execute(
text("SELECT version_num FROM alembic_version")
).fetchone()
return row[0] if row else None
finally:
engine.dispose()
except Exception as e:
logger.error("Failed to get current revision: %s", e)
return None
def stamp_database(
database_path: str | Path | None = None, revision: str = "head"
database_url: str | Path | None = None, revision: str = "head"
) -> None:
"""
Stamp database with a specific revision without running migrations.
This is useful for marking existing databases that were created before
Alembic was introduced. It tells Alembic "this database is at revision X"
without actually running the migration.
Args:
database_path: Path to SQLite database file
revision: Revision to stamp (default: "head" for latest)
Useful for marking pre-Alembic databases as already at a known revision.
"""
config = get_alembic_config(database_path)
config = get_alembic_config(database_url)
logger.info("Stamping database with revision: %s", revision)
command.stamp(config, revision)
logger.info("Database stamped successfully")
def show_migration_history(database_path: str | Path | None = None) -> None:
"""
Display migration history.
Args:
database_path: Path to SQLite database file
"""
config = get_alembic_config(database_path)
def show_migration_history(database_url: str | Path | None = None) -> None:
"""Display migration history."""
config = get_alembic_config(database_url)
command.history(config, verbose=True)
@@ -178,7 +162,9 @@ def create_migration(message: str, autogenerate: bool = False) -> None:
Note:
Since we don't use SQLAlchemy models, autogenerate will be disabled
and migrations must be written manually.
and migrations must be written manually using portable Alembic
operations (``op.create_table``, ``op.add_column`` …) rather than
raw SQL so they work on both SQLite and Postgres.
"""
config = get_alembic_config()
logger.info("Creating new migration: %s", message)