mcp-nextcloud/docs/ADR-026-pluggable-database-backend.md

ADR-026: Pluggable database backend (DATABASE_URL)

Status

Accepted — 2026-05-16

Context

RefreshTokenStorage (in nextcloud_mcp_server/auth/storage.py) holds all of the MCP server's persistent state: refresh tokens, OAuth client credentials, OAuth sessions, browser sessions, app passwords, login-flow sessions, audit logs, and webhook registrations. Until this ADR it was backed by a single SQLite file, with the path configured by TOKEN_STORAGE_DB.

This works well for single-user deployments but blocks horizontal scaling in Kubernetes:

• Every pod needs its own PVC (ReadWriteOnce) or a ReadWriteMany volume.
• Tokens stored on pod A are invisible to pod B, so a Service can only route traffic to one pod at a time.
• Restart / re-deploy cycles either drop the volume (token loss) or require coordinated PVC handling.
• Backup, encryption-at-rest, and multi-region replication become per-pod concerns rather than centrally managed DB concerns.

We needed a way for pods to be stateless and share a centralized store without giving up the zero-config SQLite path that single-user installs and local development rely on.

Decision

Introduce a DATABASE_URL setting that accepts any SQLAlchemy async URL, with sqlite+aiosqlite:///... remaining the default. The runtime keeps a single linear migration history and a single RefreshTokenStorage class — the backend is selected purely by the URL.

Resolution order

get_database_url() (in nextcloud_mcp_server/config.py) returns:

1. DATABASE_URL if set — wins over everything.
2. Otherwise sqlite+aiosqlite:///{get_token_db_path()}, so the legacy TOKEN_STORAGE_DB env var and the process-local ephemeral tempfile fallback both keep working unchanged.

Why SQLAlchemy Core + async engine, not an ABC with parallel drivers

Two alternatives were considered:

Option	Why rejected
Define a Storage ABC with SQLiteStorage (aiosqlite) and PostgresStorage (asyncpg) implementations	Doubles the surface area — every schema change has to land in two backends, with two sets of migrations, two SQL dialects, two upsert idioms. Diverges over time.
Switch to a full SQLAlchemy ORM (declarative models)	Larger refactor; the existing explicit-SQL style is intentional and well-understood by reviewers.
Keep RefreshTokenStorage and put SQLAlchemy Core under it (chosen)	One method body per operation, one migration history (Alembic is already SQLAlchemy-based). The URL drives dialect, pool, and DDL.

A thin compatibility shim (_DBConn / _Cursor / _Row in storage.py) adapts the async with aiosqlite.connect(...) as db: async with db.execute(...) as cursor: ... idiom to SQLAlchemy AsyncEngine / AsyncConnection. Existing method bodies needed only their connection context-manager swapped; ? placeholders are rewritten to named binds on the fly. The seven INSERT OR REPLACE statements were rewritten as portable INSERT ... ON CONFLICT (...) DO UPDATE (SQLite ≥ 3.24, Postgres ≥ 9.5; we already require SQLite ≥ 3.35 elsewhere).

Encryption stays in Python (Fernet), not the DB

The DB only ever sees ciphertext for sensitive columns (encrypted_token, encrypted_client_secret, encrypted_password, encrypted_poll_token). The Fernet key remains a TOKEN_ENCRYPTION_KEY env var, applied in Python before INSERT and after SELECT. This means:

• Switching backends does not invalidate or re-key existing data.
• Postgres-level features like pgcrypto are not required.
• Operators rotating the encryption key still go through the existing Python path.

DDL portability

All Alembic migrations were rewritten from raw op.execute("CREATE TABLE ...") strings to op.create_table() / op.create_index() calls with SQLAlchemy types. Notable choices:

• All *_at / expiration / timestamp columns use sa.BigInteger — Postgres INTEGER is 32-bit and unix epochs are already past that range. SQLite treats BIGINT and INTEGER identically (dynamic typing) so this is backwards compatible.
• BLOB → sa.LargeBinary (becomes BYTEA on Postgres).
• BOOLEAN DEFAULT FALSE → sa.Boolean, server_default=sa.false().
• Existing SQLite deployments are at revision 006 and skip the rewritten migrations entirely — content rewrites are safe.

No data migration, no shipped Postgres

Two scope decisions worth recording:

1. Clean cutover, no SQLite → Postgres data migration tool. Tokens are reissued on the next login; webhooks re-register on the next sync tick. Acceptable because the ephemeral-default already implies this, and the data being preserved (audit logs, OAuth sessions) is either short-lived or reconstructible.
2. Bring-your-own database. The MCP server consumes a DATABASE_URL; it does not provision Postgres itself. Operators use CNPG, RDS, the project's existing Helm chart with a sub-chart, etc. The postgres-test service in docker-compose.yml exists only for integration tests and manual HA smoke testing — it is gated on the postgres profile and is not the recommended production pattern.

CLI changes

The nextcloud-mcp-server db {upgrade,downgrade,current,history} commands gain a --database-url / -u flag (env DATABASE_URL) alongside the existing --database-path / -d (env TOKEN_STORAGE_DB). -u wins over -d; both fall back to get_database_url().

Consequences

Positive

• MCP server pods become stateless. A Kubernetes Deployment can run with replicas: 3 behind a Service, with all pods pointed at the same Postgres URL — tokens written by pod A are immediately visible to pod B.
• Centralized DB operations (backup, restore, replication, encryption at rest, monitoring) are handled by the operator's existing Postgres infrastructure rather than duplicated per-pod.
• No regression for single-user / local-development / docker-compose installs — the SQLite tempfile path is unchanged and remains the default when DATABASE_URL is unset.
• Test coverage doubles automatically: every test that uses the temp_storage fixture now runs against both SQLite and Postgres when TEST_DATABASE_URL is exported.

Negative

• One more thing operators have to think about for HA deployments (Postgres connection string, credentials secret, network policies).
• Adds SQLAlchemy + asyncpg to the runtime dependency set. SQLAlchemy was already transitively present via Alembic; asyncpg is genuinely new.
• The compatibility shim in storage.py is a small piece of bespoke code that future contributors need to understand. The alternative — rewriting every method body to SQLAlchemy idioms — was rejected as too risky for this PR but might be revisited.

Neutral

• The Alembic migration history was content-rewritten but its revision graph is unchanged (still 001 → 006), so existing SQLite deployments do not re-run anything.
• TOKEN_STORAGE_DB still works exactly as before; deployments that already set it require no changes.

Related

• ADR-022 Login Flow v2 — defines the per-user app password storage that this ADR centralizes.
• ADR-002 Vector sync authentication — explains the offline-access tokens that benefit most from HA storage.

Verification

1. uv run pytest tests/unit/ — SQLite path unchanged (1012 tests).
2. docker compose --profile postgres up -d postgres-test then TEST_DATABASE_URL=postgresql+asyncpg://mcp:mcp@localhost:5433/mcp uv run pytest tests/unit/test_app_password_storage.py tests/unit/test_webhook_storage.py — every test runs once per backend.
3. Manual end-to-end smoke against mcp-login-flow with a Postgres URL (commands in /home/chris/.claude/plans/spicy-enchanting-flurry.md → Verification).
4. k8s HA validation (after merge in homelab-argocd): replicas: 3, confirm session continuity through the Service.