Files
mcp-nextcloud/tests/unit/test_app_password_storage.py
Chris CoutinhoandClaude Opus 4.7 292cbb3292 feat(storage): pluggable database backend via DATABASE_URL (ADR-026)
Adds a `DATABASE_URL` setting that lets `RefreshTokenStorage` run against
any SQLAlchemy async backend, primarily `postgresql+asyncpg://...` for
HA k8s deployments. Default behavior is unchanged: when `DATABASE_URL` is
unset the server falls back to the existing `TOKEN_STORAGE_DB` path /
ephemeral SQLite tempfile.

Why
---
Today every MCP pod needs its own PVC to hold the SQLite file, which
pins the Deployment to one replica and blocks horizontal scaling. With
this change, operators can point all replicas at a shared Postgres
(CNPG, RDS, etc.) and the pods become stateless. Encryption stays in
Python (Fernet); the database only sees ciphertext.

What changed
------------
- `config.get_database_url()` resolves DATABASE_URL → TOKEN_STORAGE_DB →
  ephemeral tempfile in that priority order.
- `RefreshTokenStorage` builds a process-shared `AsyncEngine` in
  `initialize()`. SQLite gets NullPool; Postgres gets pool_size=10,
  max_overflow=20, pool_pre_ping=True. 30 aiosqlite call sites adapted
  via a thin `_DBConn` / `_Cursor` / `_Row` / `_ExecuteCtx` shim so
  existing method bodies need no churn beyond the connection
  context-manager swap.
- 7 `INSERT OR REPLACE` statements rewritten as portable
  `INSERT ... ON CONFLICT (...) DO UPDATE` (SQLite ≥ 3.24, Postgres ≥ 9.5).
- `sqlite_master` legacy-detection lookup replaced with SQLAlchemy
  inspector so the path works against either backend.
- File-permission hardening + parent-dir creation gated on
  `is_sqlite_url(...)` — centralized backends manage their own filesystem.
- Alembic migrations 001/002/003/005 converted from raw `op.execute(SQL)`
  to portable `op.create_table()` / `op.create_index()` with SQLAlchemy
  types. All timestamp columns are `sa.BigInteger` so Postgres allocates
  BIGINT (unix epochs don't fit in INT4). SQLite treats BIGINT as
  INTEGER, so existing deployments at revision 006 see no schema drift.
- `migrations.py` + CLI take URLs; `db {upgrade,downgrade,current,history}`
  gain `--database-url / -u` alongside the legacy `--database-path / -d`.
  `get_current_revision()` uses SQLAlchemy inspector instead of raw
  sqlite3, so the CLI works against Postgres too.
- `docker-compose.yml` adds a `postgres-test` service under the
  `postgres` profile (pinned `postgres:16-alpine` digest) for
  integration testing.
- Unit storage tests parametrized over backends via shared
  `tests/fixtures/storage_backend.py` — every test in
  `test_app_password_storage.py` and `test_webhook_storage.py` runs
  once per backend that is available. Postgres is opted in by
  `TEST_DATABASE_URL`.
- New `tests/integration/test_storage_postgres.py` (5 tests, marked
  `postgres` + `integration`) covers refresh-token, app-password,
  OAuth-session, webhook, and audit-log paths end-to-end on Postgres.
- New `docs/ADR-026-pluggable-database-backend.md` records the decision;
  `docs/configuration.md` documents `DATABASE_URL` with examples.

Out of scope
------------
- No SQLite → Postgres data migration tool (clean cutover; tokens reissue
  on next login, webhooks re-register on next sync tick).
- This repo does not provision Postgres. The matching helm chart change
  lives in cbcoutinho/helm-charts (database.url / existingSecret values).

Verification
------------
- `uv run pytest tests/unit/` — 1012 passed, SQLite path unchanged.
- `docker compose --profile postgres up -d postgres-test`
- `TEST_DATABASE_URL=... uv run pytest tests/integration/test_storage_postgres.py -m postgres -v`
  — 5 passed.
- `TEST_DATABASE_URL=... uv run pytest tests/unit/test_app_password_storage.py
  tests/unit/test_webhook_storage.py` — 50 passed (25 per backend).
- `uv run ruff check && uv run ruff format --check && uv run ty check -- nextcloud_mcp_server` — clean.

Tracked on Astrolabe Cloud POC board, card #99.

---

_This PR was generated with the help of AI, and reviewed by a Human_

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-16 18:06:42 +02:00

251 lines
8.9 KiB
Python

"""
Unit tests for App Password Storage functionality.
Tests the app password methods in RefreshTokenStorage for multi-user
BasicAuth mode background sync.
These tests are parametrized over both supported backends so the storage
layer is exercised against SQLite (default, always runs) and Postgres (gated
on ``TEST_DATABASE_URL``; bring up ``docker compose --profile postgres up
-d postgres-test`` and export ``TEST_DATABASE_URL=postgresql+asyncpg://mcp:mcp@localhost:5433/mcp``
to opt in).
"""
import tempfile
from pathlib import Path
import pytest
from cryptography.fernet import Fernet
from nextcloud_mcp_server.auth.storage import RefreshTokenStorage
pytestmark = pytest.mark.unit
@pytest.fixture
def encryption_key():
"""Generate a test encryption key."""
return Fernet.generate_key().decode()
@pytest.fixture
async def temp_storage(encryption_key, storage_backend):
"""Create a storage instance backed by either SQLite or Postgres.
The ``storage_backend`` fixture is parametrized by pytest, so every test
that uses ``temp_storage`` runs once per backend that is available in
the current environment.
"""
if storage_backend["kind"] == "sqlite":
with tempfile.TemporaryDirectory() as tmpdir:
db_path = Path(tmpdir) / "test_app_passwords.db"
storage = RefreshTokenStorage(
db_path=str(db_path), encryption_key=encryption_key
)
await storage.initialize()
yield storage
else:
storage = RefreshTokenStorage(
database_url=storage_backend["url"], encryption_key=encryption_key
)
await storage.initialize()
try:
yield storage
finally:
# Each test gets an isolated schema; tear it down so the next
# parametrized run starts clean.
await storage_backend["reset"]()
async def test_store_app_password(temp_storage):
"""Test storing an app password."""
await temp_storage.store_app_password(
user_id="testuser",
app_password="JHWzB-ZYgLZ-3qBDj-ZQe5o-LdKpB",
)
# Verify it can be retrieved
retrieved = await temp_storage.get_app_password("testuser")
assert retrieved == "JHWzB-ZYgLZ-3qBDj-ZQe5o-LdKpB"
async def test_store_app_password_replaces_existing(temp_storage):
"""Test that storing a new app password replaces the existing one."""
await temp_storage.store_app_password(
user_id="testuser",
app_password="aaaaa-bbbbb-ccccc-ddddd-eeeee",
)
await temp_storage.store_app_password(
user_id="testuser",
app_password="fffff-ggggg-hhhhh-iiiii-jjjjj",
)
retrieved = await temp_storage.get_app_password("testuser")
assert retrieved == "fffff-ggggg-hhhhh-iiiii-jjjjj"
async def test_get_app_password_nonexistent(temp_storage):
"""Test retrieving app password for non-existent user."""
retrieved = await temp_storage.get_app_password("nonexistent")
assert retrieved is None
async def test_delete_app_password(temp_storage):
"""Test deleting an app password."""
await temp_storage.store_app_password(
user_id="testuser",
app_password="JHWzB-ZYgLZ-3qBDj-ZQe5o-LdKpB",
)
deleted = await temp_storage.delete_app_password("testuser")
assert deleted is True
# Verify it's gone
retrieved = await temp_storage.get_app_password("testuser")
assert retrieved is None
async def test_delete_app_password_nonexistent(temp_storage):
"""Test deleting non-existent app password."""
deleted = await temp_storage.delete_app_password("nonexistent")
assert deleted is False
async def test_get_all_app_password_user_ids(temp_storage):
"""Test listing all users with app passwords."""
await temp_storage.store_app_password("alice", "aaaaa-aaaaa-aaaaa-aaaaa-aaaaa")
await temp_storage.store_app_password("bob", "bbbbb-bbbbb-bbbbb-bbbbb-bbbbb")
await temp_storage.store_app_password("charlie", "ccccc-ccccc-ccccc-ccccc-ccccc")
user_ids = await temp_storage.get_all_app_password_user_ids()
assert len(user_ids) == 3
assert "alice" in user_ids
assert "bob" in user_ids
assert "charlie" in user_ids
async def test_get_all_app_password_user_ids_empty(temp_storage):
"""Test listing users when none have app passwords."""
user_ids = await temp_storage.get_all_app_password_user_ids()
assert len(user_ids) == 0
async def test_app_password_encryption(encryption_key):
"""Test that app passwords are encrypted at rest."""
with tempfile.TemporaryDirectory() as tmpdir:
db_path = Path(tmpdir) / "test_encryption.db"
storage = RefreshTokenStorage(
db_path=str(db_path), encryption_key=encryption_key
)
await storage.initialize()
# Store a password
test_password = "JHWzB-ZYgLZ-3qBDj-ZQe5o-LdKpB"
await storage.store_app_password("testuser", test_password)
# Read directly from database to verify encryption
import aiosqlite
async with aiosqlite.connect(str(db_path)) as db:
async with db.execute(
"SELECT encrypted_password FROM app_passwords WHERE user_id = ?",
("testuser",),
) as cursor:
row = await cursor.fetchone()
# The stored value should be encrypted (not plain text)
encrypted_bytes = row[0]
assert encrypted_bytes != test_password.encode()
# Encrypted data should be longer due to Fernet overhead
assert len(encrypted_bytes) > len(test_password)
async def test_app_password_requires_encryption_key():
"""Test that app password operations require encryption key."""
with tempfile.TemporaryDirectory() as tmpdir:
db_path = Path(tmpdir) / "test_no_key.db"
storage = RefreshTokenStorage(db_path=str(db_path), encryption_key=None)
await storage.initialize()
# Storing should fail without encryption key
with pytest.raises(RuntimeError, match="Encryption key not configured"):
await storage.store_app_password(
"testuser", "aaaaa-bbbbb-ccccc-ddddd-eeeee"
)
# Getting should also fail without encryption key
with pytest.raises(RuntimeError, match="Encryption key not configured"):
await storage.get_app_password("testuser")
async def test_multiple_users_independence(temp_storage):
"""Test that different users maintain independent app passwords."""
users = ["alice", "bob", "charlie", "diana"]
# Store unique passwords for each user
for i, user in enumerate(users):
password = (
f"{user[0]}{user[0]}{user[0]}{user[0]}{user[0]}-" * 4
+ f"{user[0]}{user[0]}{user[0]}{user[0]}{user[0]}"
)
await temp_storage.store_app_password(user, password)
# Verify each user has their correct password
for user in users:
expected = (
f"{user[0]}{user[0]}{user[0]}{user[0]}{user[0]}-" * 4
+ f"{user[0]}{user[0]}{user[0]}{user[0]}{user[0]}"
)
retrieved = await temp_storage.get_app_password(user)
assert retrieved == expected
# Delete one user's password
await temp_storage.delete_app_password("bob")
# Verify other users unchanged
for user in ["alice", "charlie", "diana"]:
retrieved = await temp_storage.get_app_password(user)
assert retrieved is not None
# Verify bob's password is gone
assert await temp_storage.get_app_password("bob") is None
async def test_app_password_with_special_characters(temp_storage):
"""Test storing passwords with various alphanumeric patterns."""
# Nextcloud app passwords use alphanumeric characters
passwords = [
"AAAAA-BBBBB-CCCCC-DDDDD-EEEEE", # uppercase
"aaaaa-bbbbb-ccccc-ddddd-eeeee", # lowercase
"12345-67890-12345-67890-12345", # numbers
"aB1cD-eF2gH-iJ3kL-mN4oP-qR5sT", # mixed
]
for i, password in enumerate(passwords):
user = f"user{i}"
await temp_storage.store_app_password(user, password)
retrieved = await temp_storage.get_app_password(user)
assert retrieved == password
async def test_decryption_with_wrong_key(encryption_key):
"""Test that decryption fails with wrong key."""
with tempfile.TemporaryDirectory() as tmpdir:
db_path = Path(tmpdir) / "test_wrong_key.db"
# Store with original key
storage1 = RefreshTokenStorage(
db_path=str(db_path), encryption_key=encryption_key
)
await storage1.initialize()
await storage1.store_app_password("testuser", "JHWzB-ZYgLZ-3qBDj-ZQe5o-LdKpB")
# Try to read with different key
wrong_key = Fernet.generate_key().decode()
storage2 = RefreshTokenStorage(db_path=str(db_path), encryption_key=wrong_key)
await storage2.initialize()
# Decryption should fail and return None (graceful handling)
retrieved = await storage2.get_app_password("testuser")
assert retrieved is None