feat(storage): pluggable database backend via DATABASE_URL (ADR-026)
Adds a `DATABASE_URL` setting that lets `RefreshTokenStorage` run against
any SQLAlchemy async backend, primarily `postgresql+asyncpg://...` for
HA k8s deployments. Default behavior is unchanged: when `DATABASE_URL` is
unset the server falls back to the existing `TOKEN_STORAGE_DB` path /
ephemeral SQLite tempfile.
Why
---
Today every MCP pod needs its own PVC to hold the SQLite file, which
pins the Deployment to one replica and blocks horizontal scaling. With
this change, operators can point all replicas at a shared Postgres
(CNPG, RDS, etc.) and the pods become stateless. Encryption stays in
Python (Fernet); the database only sees ciphertext.
What changed
------------
- `config.get_database_url()` resolves DATABASE_URL → TOKEN_STORAGE_DB →
ephemeral tempfile in that priority order.
- `RefreshTokenStorage` builds a process-shared `AsyncEngine` in
`initialize()`. SQLite gets NullPool; Postgres gets pool_size=10,
max_overflow=20, pool_pre_ping=True. 30 aiosqlite call sites adapted
via a thin `_DBConn` / `_Cursor` / `_Row` / `_ExecuteCtx` shim so
existing method bodies need no churn beyond the connection
context-manager swap.
- 7 `INSERT OR REPLACE` statements rewritten as portable
`INSERT ... ON CONFLICT (...) DO UPDATE` (SQLite ≥ 3.24, Postgres ≥ 9.5).
- `sqlite_master` legacy-detection lookup replaced with SQLAlchemy
inspector so the path works against either backend.
- File-permission hardening + parent-dir creation gated on
`is_sqlite_url(...)` — centralized backends manage their own filesystem.
- Alembic migrations 001/002/003/005 converted from raw `op.execute(SQL)`
to portable `op.create_table()` / `op.create_index()` with SQLAlchemy
types. All timestamp columns are `sa.BigInteger` so Postgres allocates
BIGINT (unix epochs don't fit in INT4). SQLite treats BIGINT as
INTEGER, so existing deployments at revision 006 see no schema drift.
- `migrations.py` + CLI take URLs; `db {upgrade,downgrade,current,history}`
gain `--database-url / -u` alongside the legacy `--database-path / -d`.
`get_current_revision()` uses SQLAlchemy inspector instead of raw
sqlite3, so the CLI works against Postgres too.
- `docker-compose.yml` adds a `postgres-test` service under the
`postgres` profile (pinned `postgres:16-alpine` digest) for
integration testing.
- Unit storage tests parametrized over backends via shared
`tests/fixtures/storage_backend.py` — every test in
`test_app_password_storage.py` and `test_webhook_storage.py` runs
once per backend that is available. Postgres is opted in by
`TEST_DATABASE_URL`.
- New `tests/integration/test_storage_postgres.py` (5 tests, marked
`postgres` + `integration`) covers refresh-token, app-password,
OAuth-session, webhook, and audit-log paths end-to-end on Postgres.
- New `docs/ADR-026-pluggable-database-backend.md` records the decision;
`docs/configuration.md` documents `DATABASE_URL` with examples.
Out of scope
------------
- No SQLite → Postgres data migration tool (clean cutover; tokens reissue
on next login, webhooks re-register on next sync tick).
- This repo does not provision Postgres. The matching helm chart change
lives in cbcoutinho/helm-charts (database.url / existingSecret values).
Verification
------------
- `uv run pytest tests/unit/` — 1012 passed, SQLite path unchanged.
- `docker compose --profile postgres up -d postgres-test`
- `TEST_DATABASE_URL=... uv run pytest tests/integration/test_storage_postgres.py -m postgres -v`
— 5 passed.
- `TEST_DATABASE_URL=... uv run pytest tests/unit/test_app_password_storage.py
tests/unit/test_webhook_storage.py` — 50 passed (25 per backend).
- `uv run ruff check && uv run ruff format --check && uv run ty check -- nextcloud_mcp_server` — clean.
Tracked on Astrolabe Cloud POC board, card #99.
---
_This PR was generated with the help of AI, and reviewed by a Human_
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.7
parent
15a7e680a6
commit
292cbb3292
@@ -118,6 +118,48 @@ See [Login Flow v2](login-flow-v2.md) for full setup, scope reference, and troub
|
||||
|
||||
---
|
||||
|
||||
## Centralized Token Storage (DATABASE_URL, Optional)
|
||||
|
||||
By default the MCP server stores tokens / sessions / app passwords in a
|
||||
local SQLite file (`TOKEN_STORAGE_DB`, falling back to a per-process
|
||||
tempfile). For HA Kubernetes deployments where you need multiple
|
||||
stateless pods to share state, point the server at a centralized
|
||||
database via `DATABASE_URL`.
|
||||
|
||||
```env
|
||||
# Centralized Postgres backend (HA k8s deployments)
|
||||
DATABASE_URL=postgresql+asyncpg://mcp:secret@postgres.svc.cluster.local:5432/mcp
|
||||
TOKEN_ENCRYPTION_KEY=<fernet-key>
|
||||
```
|
||||
|
||||
| Variable | Required | Description |
|
||||
|----------|----------|-------------|
|
||||
| `DATABASE_URL` | Optional | SQLAlchemy async URL for any supported backend. When set, wins over `TOKEN_STORAGE_DB`. Primary supported targets: `postgresql+asyncpg://...` (recommended for HA) and `sqlite+aiosqlite:///...` (development). |
|
||||
| `TOKEN_STORAGE_DB` | Optional | Legacy SQLite-only path. Used when `DATABASE_URL` is unset. Falls back to a per-process ephemeral tempfile when both are unset. |
|
||||
|
||||
Notes:
|
||||
|
||||
- **Bring-your-own DB.** The MCP server doesn't provision the database;
|
||||
it just consumes the URL. Use CNPG, RDS, your existing Helm chart's
|
||||
Postgres sub-chart, etc.
|
||||
- **Encryption stays in the app.** `TOKEN_ENCRYPTION_KEY` (Fernet) is
|
||||
applied in Python; the database only ever sees ciphertext for
|
||||
sensitive columns. You don't need `pgcrypto`.
|
||||
- **Schema is managed automatically.** On startup the server runs
|
||||
Alembic migrations against the configured backend. Existing SQLite
|
||||
deployments are stamped at the current revision and skip re-execution.
|
||||
- **No data migration tool.** Moving from SQLite to Postgres is a clean
|
||||
cutover — tokens are reissued on the next login, webhooks
|
||||
re-register on the next sync tick.
|
||||
- **Testing a Postgres backend locally:** `docker compose --profile
|
||||
postgres up -d postgres-test` then export
|
||||
`DATABASE_URL=postgresql+asyncpg://mcp:mcp@localhost:5433/mcp`.
|
||||
|
||||
See [ADR-026 Pluggable database backend](ADR-026-pluggable-database-backend.md)
|
||||
for the architecture rationale.
|
||||
|
||||
---
|
||||
|
||||
## SSL/TLS Configuration (Optional)
|
||||
|
||||
If your Nextcloud instance uses a self-signed certificate or a private CA (common with reverse proxies like Traefik or Caddy), the MCP server will reject the connection by default. Use these settings to configure certificate verification.
|
||||
|
||||
Reference in New Issue
Block a user