Adds a `DATABASE_URL` setting that lets `RefreshTokenStorage` run against
any SQLAlchemy async backend, primarily `postgresql+asyncpg://...` for
HA k8s deployments. Default behavior is unchanged: when `DATABASE_URL` is
unset the server falls back to the existing `TOKEN_STORAGE_DB` path /
ephemeral SQLite tempfile.
Why
---
Today every MCP pod needs its own PVC to hold the SQLite file, which
pins the Deployment to one replica and blocks horizontal scaling. With
this change, operators can point all replicas at a shared Postgres
(CNPG, RDS, etc.) and the pods become stateless. Encryption stays in
Python (Fernet); the database only sees ciphertext.
What changed
------------
- `config.get_database_url()` resolves DATABASE_URL → TOKEN_STORAGE_DB →
ephemeral tempfile in that priority order.
- `RefreshTokenStorage` builds a process-shared `AsyncEngine` in
`initialize()`. SQLite gets NullPool; Postgres gets pool_size=10,
max_overflow=20, pool_pre_ping=True. 30 aiosqlite call sites adapted
via a thin `_DBConn` / `_Cursor` / `_Row` / `_ExecuteCtx` shim so
existing method bodies need no churn beyond the connection
context-manager swap.
- 7 `INSERT OR REPLACE` statements rewritten as portable
`INSERT ... ON CONFLICT (...) DO UPDATE` (SQLite ≥ 3.24, Postgres ≥ 9.5).
- `sqlite_master` legacy-detection lookup replaced with SQLAlchemy
inspector so the path works against either backend.
- File-permission hardening + parent-dir creation gated on
`is_sqlite_url(...)` — centralized backends manage their own filesystem.
- Alembic migrations 001/002/003/005 converted from raw `op.execute(SQL)`
to portable `op.create_table()` / `op.create_index()` with SQLAlchemy
types. All timestamp columns are `sa.BigInteger` so Postgres allocates
BIGINT (unix epochs don't fit in INT4). SQLite treats BIGINT as
INTEGER, so existing deployments at revision 006 see no schema drift.
- `migrations.py` + CLI take URLs; `db {upgrade,downgrade,current,history}`
gain `--database-url / -u` alongside the legacy `--database-path / -d`.
`get_current_revision()` uses SQLAlchemy inspector instead of raw
sqlite3, so the CLI works against Postgres too.
- `docker-compose.yml` adds a `postgres-test` service under the
`postgres` profile (pinned `postgres:16-alpine` digest) for
integration testing.
- Unit storage tests parametrized over backends via shared
`tests/fixtures/storage_backend.py` — every test in
`test_app_password_storage.py` and `test_webhook_storage.py` runs
once per backend that is available. Postgres is opted in by
`TEST_DATABASE_URL`.
- New `tests/integration/test_storage_postgres.py` (5 tests, marked
`postgres` + `integration`) covers refresh-token, app-password,
OAuth-session, webhook, and audit-log paths end-to-end on Postgres.
- New `docs/ADR-026-pluggable-database-backend.md` records the decision;
`docs/configuration.md` documents `DATABASE_URL` with examples.
Out of scope
------------
- No SQLite → Postgres data migration tool (clean cutover; tokens reissue
on next login, webhooks re-register on next sync tick).
- This repo does not provision Postgres. The matching helm chart change
lives in cbcoutinho/helm-charts (database.url / existingSecret values).
Verification
------------
- `uv run pytest tests/unit/` — 1012 passed, SQLite path unchanged.
- `docker compose --profile postgres up -d postgres-test`
- `TEST_DATABASE_URL=... uv run pytest tests/integration/test_storage_postgres.py -m postgres -v`
— 5 passed.
- `TEST_DATABASE_URL=... uv run pytest tests/unit/test_app_password_storage.py
tests/unit/test_webhook_storage.py` — 50 passed (25 per backend).
- `uv run ruff check && uv run ruff format --check && uv run ty check -- nextcloud_mcp_server` — clean.
Tracked on Astrolabe Cloud POC board, card #99.
---
_This PR was generated with the help of AI, and reviewed by a Human_
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
251 lines
8.9 KiB
Python
251 lines
8.9 KiB
Python
"""
|
|
Unit tests for App Password Storage functionality.
|
|
|
|
Tests the app password methods in RefreshTokenStorage for multi-user
|
|
BasicAuth mode background sync.
|
|
|
|
These tests are parametrized over both supported backends so the storage
|
|
layer is exercised against SQLite (default, always runs) and Postgres (gated
|
|
on ``TEST_DATABASE_URL``; bring up ``docker compose --profile postgres up
|
|
-d postgres-test`` and export ``TEST_DATABASE_URL=postgresql+asyncpg://mcp:mcp@localhost:5433/mcp``
|
|
to opt in).
|
|
"""
|
|
|
|
import tempfile
|
|
from pathlib import Path
|
|
|
|
import pytest
|
|
from cryptography.fernet import Fernet
|
|
|
|
from nextcloud_mcp_server.auth.storage import RefreshTokenStorage
|
|
|
|
pytestmark = pytest.mark.unit
|
|
|
|
|
|
@pytest.fixture
|
|
def encryption_key():
|
|
"""Generate a test encryption key."""
|
|
return Fernet.generate_key().decode()
|
|
|
|
|
|
@pytest.fixture
|
|
async def temp_storage(encryption_key, storage_backend):
|
|
"""Create a storage instance backed by either SQLite or Postgres.
|
|
|
|
The ``storage_backend`` fixture is parametrized by pytest, so every test
|
|
that uses ``temp_storage`` runs once per backend that is available in
|
|
the current environment.
|
|
"""
|
|
if storage_backend["kind"] == "sqlite":
|
|
with tempfile.TemporaryDirectory() as tmpdir:
|
|
db_path = Path(tmpdir) / "test_app_passwords.db"
|
|
storage = RefreshTokenStorage(
|
|
db_path=str(db_path), encryption_key=encryption_key
|
|
)
|
|
await storage.initialize()
|
|
yield storage
|
|
else:
|
|
storage = RefreshTokenStorage(
|
|
database_url=storage_backend["url"], encryption_key=encryption_key
|
|
)
|
|
await storage.initialize()
|
|
try:
|
|
yield storage
|
|
finally:
|
|
# Each test gets an isolated schema; tear it down so the next
|
|
# parametrized run starts clean.
|
|
await storage_backend["reset"]()
|
|
|
|
|
|
async def test_store_app_password(temp_storage):
|
|
"""Test storing an app password."""
|
|
await temp_storage.store_app_password(
|
|
user_id="testuser",
|
|
app_password="JHWzB-ZYgLZ-3qBDj-ZQe5o-LdKpB",
|
|
)
|
|
|
|
# Verify it can be retrieved
|
|
retrieved = await temp_storage.get_app_password("testuser")
|
|
assert retrieved == "JHWzB-ZYgLZ-3qBDj-ZQe5o-LdKpB"
|
|
|
|
|
|
async def test_store_app_password_replaces_existing(temp_storage):
|
|
"""Test that storing a new app password replaces the existing one."""
|
|
await temp_storage.store_app_password(
|
|
user_id="testuser",
|
|
app_password="aaaaa-bbbbb-ccccc-ddddd-eeeee",
|
|
)
|
|
await temp_storage.store_app_password(
|
|
user_id="testuser",
|
|
app_password="fffff-ggggg-hhhhh-iiiii-jjjjj",
|
|
)
|
|
|
|
retrieved = await temp_storage.get_app_password("testuser")
|
|
assert retrieved == "fffff-ggggg-hhhhh-iiiii-jjjjj"
|
|
|
|
|
|
async def test_get_app_password_nonexistent(temp_storage):
|
|
"""Test retrieving app password for non-existent user."""
|
|
retrieved = await temp_storage.get_app_password("nonexistent")
|
|
assert retrieved is None
|
|
|
|
|
|
async def test_delete_app_password(temp_storage):
|
|
"""Test deleting an app password."""
|
|
await temp_storage.store_app_password(
|
|
user_id="testuser",
|
|
app_password="JHWzB-ZYgLZ-3qBDj-ZQe5o-LdKpB",
|
|
)
|
|
|
|
deleted = await temp_storage.delete_app_password("testuser")
|
|
assert deleted is True
|
|
|
|
# Verify it's gone
|
|
retrieved = await temp_storage.get_app_password("testuser")
|
|
assert retrieved is None
|
|
|
|
|
|
async def test_delete_app_password_nonexistent(temp_storage):
|
|
"""Test deleting non-existent app password."""
|
|
deleted = await temp_storage.delete_app_password("nonexistent")
|
|
assert deleted is False
|
|
|
|
|
|
async def test_get_all_app_password_user_ids(temp_storage):
|
|
"""Test listing all users with app passwords."""
|
|
await temp_storage.store_app_password("alice", "aaaaa-aaaaa-aaaaa-aaaaa-aaaaa")
|
|
await temp_storage.store_app_password("bob", "bbbbb-bbbbb-bbbbb-bbbbb-bbbbb")
|
|
await temp_storage.store_app_password("charlie", "ccccc-ccccc-ccccc-ccccc-ccccc")
|
|
|
|
user_ids = await temp_storage.get_all_app_password_user_ids()
|
|
assert len(user_ids) == 3
|
|
assert "alice" in user_ids
|
|
assert "bob" in user_ids
|
|
assert "charlie" in user_ids
|
|
|
|
|
|
async def test_get_all_app_password_user_ids_empty(temp_storage):
|
|
"""Test listing users when none have app passwords."""
|
|
user_ids = await temp_storage.get_all_app_password_user_ids()
|
|
assert len(user_ids) == 0
|
|
|
|
|
|
async def test_app_password_encryption(encryption_key):
|
|
"""Test that app passwords are encrypted at rest."""
|
|
with tempfile.TemporaryDirectory() as tmpdir:
|
|
db_path = Path(tmpdir) / "test_encryption.db"
|
|
storage = RefreshTokenStorage(
|
|
db_path=str(db_path), encryption_key=encryption_key
|
|
)
|
|
await storage.initialize()
|
|
|
|
# Store a password
|
|
test_password = "JHWzB-ZYgLZ-3qBDj-ZQe5o-LdKpB"
|
|
await storage.store_app_password("testuser", test_password)
|
|
|
|
# Read directly from database to verify encryption
|
|
import aiosqlite
|
|
|
|
async with aiosqlite.connect(str(db_path)) as db:
|
|
async with db.execute(
|
|
"SELECT encrypted_password FROM app_passwords WHERE user_id = ?",
|
|
("testuser",),
|
|
) as cursor:
|
|
row = await cursor.fetchone()
|
|
|
|
# The stored value should be encrypted (not plain text)
|
|
encrypted_bytes = row[0]
|
|
assert encrypted_bytes != test_password.encode()
|
|
# Encrypted data should be longer due to Fernet overhead
|
|
assert len(encrypted_bytes) > len(test_password)
|
|
|
|
|
|
async def test_app_password_requires_encryption_key():
|
|
"""Test that app password operations require encryption key."""
|
|
with tempfile.TemporaryDirectory() as tmpdir:
|
|
db_path = Path(tmpdir) / "test_no_key.db"
|
|
storage = RefreshTokenStorage(db_path=str(db_path), encryption_key=None)
|
|
await storage.initialize()
|
|
|
|
# Storing should fail without encryption key
|
|
with pytest.raises(RuntimeError, match="Encryption key not configured"):
|
|
await storage.store_app_password(
|
|
"testuser", "aaaaa-bbbbb-ccccc-ddddd-eeeee"
|
|
)
|
|
|
|
# Getting should also fail without encryption key
|
|
with pytest.raises(RuntimeError, match="Encryption key not configured"):
|
|
await storage.get_app_password("testuser")
|
|
|
|
|
|
async def test_multiple_users_independence(temp_storage):
|
|
"""Test that different users maintain independent app passwords."""
|
|
users = ["alice", "bob", "charlie", "diana"]
|
|
|
|
# Store unique passwords for each user
|
|
for i, user in enumerate(users):
|
|
password = (
|
|
f"{user[0]}{user[0]}{user[0]}{user[0]}{user[0]}-" * 4
|
|
+ f"{user[0]}{user[0]}{user[0]}{user[0]}{user[0]}"
|
|
)
|
|
await temp_storage.store_app_password(user, password)
|
|
|
|
# Verify each user has their correct password
|
|
for user in users:
|
|
expected = (
|
|
f"{user[0]}{user[0]}{user[0]}{user[0]}{user[0]}-" * 4
|
|
+ f"{user[0]}{user[0]}{user[0]}{user[0]}{user[0]}"
|
|
)
|
|
retrieved = await temp_storage.get_app_password(user)
|
|
assert retrieved == expected
|
|
|
|
# Delete one user's password
|
|
await temp_storage.delete_app_password("bob")
|
|
|
|
# Verify other users unchanged
|
|
for user in ["alice", "charlie", "diana"]:
|
|
retrieved = await temp_storage.get_app_password(user)
|
|
assert retrieved is not None
|
|
|
|
# Verify bob's password is gone
|
|
assert await temp_storage.get_app_password("bob") is None
|
|
|
|
|
|
async def test_app_password_with_special_characters(temp_storage):
|
|
"""Test storing passwords with various alphanumeric patterns."""
|
|
# Nextcloud app passwords use alphanumeric characters
|
|
passwords = [
|
|
"AAAAA-BBBBB-CCCCC-DDDDD-EEEEE", # uppercase
|
|
"aaaaa-bbbbb-ccccc-ddddd-eeeee", # lowercase
|
|
"12345-67890-12345-67890-12345", # numbers
|
|
"aB1cD-eF2gH-iJ3kL-mN4oP-qR5sT", # mixed
|
|
]
|
|
|
|
for i, password in enumerate(passwords):
|
|
user = f"user{i}"
|
|
await temp_storage.store_app_password(user, password)
|
|
retrieved = await temp_storage.get_app_password(user)
|
|
assert retrieved == password
|
|
|
|
|
|
async def test_decryption_with_wrong_key(encryption_key):
|
|
"""Test that decryption fails with wrong key."""
|
|
with tempfile.TemporaryDirectory() as tmpdir:
|
|
db_path = Path(tmpdir) / "test_wrong_key.db"
|
|
|
|
# Store with original key
|
|
storage1 = RefreshTokenStorage(
|
|
db_path=str(db_path), encryption_key=encryption_key
|
|
)
|
|
await storage1.initialize()
|
|
await storage1.store_app_password("testuser", "JHWzB-ZYgLZ-3qBDj-ZQe5o-LdKpB")
|
|
|
|
# Try to read with different key
|
|
wrong_key = Fernet.generate_key().decode()
|
|
storage2 = RefreshTokenStorage(db_path=str(db_path), encryption_key=wrong_key)
|
|
await storage2.initialize()
|
|
|
|
# Decryption should fail and return None (graceful handling)
|
|
retrieved = await storage2.get_app_password("testuser")
|
|
assert retrieved is None
|