Round-2 fixes after the bot review on PR #798 plus two user follow-ups
(self-signed Postgres support; asyncpg should be a PyPI extra). Folded
into the same PR rather than a follow-up since the work is still
unmerged.
Security
--------
- Mask database credentials in all 5 log call sites (storage.py × 4,
migrations.py × 1) via a new `mask_db_password()` helper in config.py.
Uses SQLAlchemy's `make_url(...).render_as_string(hide_password=True)`
with a regex fallback so the masking path never raises.
- New `tests/unit/test_storage_logging.py` asserts a sentinel password
never appears in `caplog` during `RefreshTokenStorage.initialize()`.
Distribution
------------
- `asyncpg` moved to `[project.optional-dependencies] postgres` so a
vanilla `pip install nextcloud-mcp-server` no longer pulls in the
~5 MB C extension. The Docker image runs `uv sync --extra postgres`,
so containerized deployments are unchanged.
- When `DATABASE_URL=postgresql+asyncpg://...` is set on a venv missing
the extra, `RefreshTokenStorage.initialize()` raises a friendly
RuntimeError pointing at `[postgres]` rather than the generic
ModuleNotFoundError.
TLS for the Postgres backend
----------------------------
- New `DATABASE_VERIFY_SSL` + `DATABASE_CA_BUNDLE` env vars mirror the
existing `NEXTCLOUD_VERIFY_SSL` / `NEXTCLOUD_CA_BUNDLE` pattern
(validators in Settings.__post_init__, `get_database_ssl()` helper
alongside `get_nextcloud_ssl_verify()`). `DATABASE_VERIFY_SSL=false`
wins over `DATABASE_CA_BUNDLE` for incident-response convenience.
- Default is **None** rather than True — keeps PR #798's behavior
intact for cluster-internal Postgres that runs without TLS. Operators
opt into verify-full or supply a private CA. ADR-026 records the
reasoning vs the Nextcloud HTTPS default.
- Engine factory in `storage.py` passes `ssl` via `connect_args` only
when `get_database_ssl()` returns non-None; otherwise asyncpg's
default (`prefer`) applies.
- Storage logs which TLS mode is active at INFO (no secret material).
Configurable connection pool
----------------------------
- `DATABASE_POOL_SIZE` (default 10) and `DATABASE_MAX_OVERFLOW`
(default 20) replace the hardcoded engine values. With many replicas
this can blow past managed-Postgres `max_connections=100`; tune down
for large fleets.
- gte-1 / gte-0 validators in __post_init__ reject 0/negative pool
sizes at startup with the offending value in the error.
Consistency polish
------------------
- Migration 006: convert raw `op.execute("ALTER TABLE ... ADD COLUMN")`
to `op.batch_alter_table(...).add_column(sa.Column("nonce", sa.Text))`
for stylistic consistency with the rewritten 001-005. Downgrade now
drops the column instead of being a no-op.
- `registered_webhooks.created_at` standardized from `sa.Float` to
`sa.BigInteger` (all other `*_at` columns); `store_webhook()` casts
`time.time()` → `int`.
- `is_sqlite_url()` made case-insensitive.
Testing
-------
- New `tests/integration/test_storage_postgres.py::test_cleanup_expired_roundtrip`
exercises `cleanup_expired_tokens`, `cleanup_expired_sessions`, and
`cleanup_expired_browser_sessions` — relies on DELETE rowcount,
historically dialect-tricky.
- `tests/unit/test_ssl_config.py` extended with `TestDatabaseSSLSettings`
+ `TestGetDatabaseSSL` classes (9 new tests) mirroring the existing
Nextcloud SSL tests one-for-one.
Docs
----
- `docs/configuration.md` Centralized-Storage section grew the four new
env vars + a homelab example with a private CA.
- `docs/ADR-026` grew Distribution, TLS, and `alembic/env.py` async-pattern
subsections explaining the non-obvious design choices.
Helm chart counterpart in cbcoutinho/helm-charts PR #34 (separate
commit on `feat/nextcloud-mcp-server-database-url`).
Verification
------------
- `uv run pytest tests/unit/` — 1025 passed.
- `TEST_DATABASE_URL=... uv run pytest tests/integration/test_storage_postgres.py -m postgres` — 6 passed (including new cleanup test).
- `uv run ruff check && uv run ruff format --check && uv run ty check -- nextcloud_mcp_server` — clean.
Tracked on Astrolabe Cloud POC board, card #99.
---
_This PR was generated with the help of AI, and reviewed by a Human_
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When enable_login_flow is true, also check the app_passwords table
for provisioned users. Previously, OAuth mode only queried the
refresh_tokens table, missing users who were provisioned via
Login Flow v2 (which stores app passwords, not refresh tokens).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Move alembic/ directory to nextcloud_mcp_server/alembic/ subpackage
- Update migrations.py to use package location instead of alembic.ini
- Update env.py to set script_location dynamically
- Update alembic.ini for development CLI usage
- Fix Dockerfile typo: .vnev -> .venv
This fixes FileNotFoundError when running in Docker with non-editable
install. The alembic package is now installed with the main package,
making it work in both development and production environments.
Resolves: Docker startup error 'alembic.ini not found at
/app/.venv/lib/python3.12/site-packages/alembic.ini'
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Updated docs/running.md to use Docker container examples instead of
direct Python commands. This aligns with the CLI change to require
explicit 'run' subcommand while maintaining backward compatibility
for Docker users (ENTRYPOINT includes 'run').
Key changes:
- Quick Start: Use Docker commands instead of uv run
- Running Locally → Running with Docker: All examples use Docker
- Development Mode: Added CLI subcommands documentation (run/db)
- Database Migrations: Documented Alembic integration for developers
- Server Options: Docker port mapping instead of --host/--port flags
- Process Management: Simplified to Docker Compose only (removed systemd)
- Performance Tuning: Production Docker Compose with resource limits
- Troubleshooting: Docker logs and debug commands
Updated Dockerfile ENTRYPOINT:
- Changed from: ["/app/.venv/bin/nextcloud-mcp-server", "--host", "0.0.0.0"]
- Changed to: ["/app/.venv/bin/nextcloud-mcp-server", "run", "--host", "0.0.0.0"]
No breaking changes for Docker/Helm users - container interface unchanged.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Implements Alembic for managing token storage database schema versions.
Migrations run automatically on startup with full backward compatibility.
**Changes:**
- Add Alembic dependency (1.14.0+) and SQLAlchemy (auto-installed)
- Create migration infrastructure in alembic/ directory
- Add initial migration (001) capturing current schema
- Modify RefreshTokenStorage.initialize() to run migrations via anyio
- Add CLI commands: db upgrade, current, history, downgrade, migrate
- Add comprehensive migration documentation
**Backward Compatibility:**
- Pre-Alembic databases automatically stamped with revision 001
- No schema changes for existing databases
- Automatic upgrade on first startup after update
**Migration Strategy:**
Three scenarios handled:
1. New database → Run migrations from scratch
2. Pre-Alembic database → Stamp with 001 (no changes)
3. Alembic-managed → Upgrade to latest
**Architecture:**
- Uses anyio.to_thread.run_sync() for structured concurrency
- Alembic env.py runs with anyio.run() in worker thread
- SQLite-friendly migration patterns documented
- No ThreadPoolExecutor needed (anyio handles it)
**CLI Usage:**
```bash
nextcloud-mcp-server db upgrade # Upgrade to latest
nextcloud-mcp-server db current # Show version
nextcloud-mcp-server db history # View changelog
nextcloud-mcp-server db downgrade # Rollback (with confirmation)
nextcloud-mcp-server db migrate "description" # Create migration
```
**Testing:**
- All 13 webhook storage tests pass
- New/pre-Alembic database scenarios validated
- anyio integration tested
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Drawing directly with ImageDraw on RGBA mode doesn't blend alpha
properly. Use Image.alpha_composite() with a transparent overlay
to achieve correct semi-transparent highlight fills.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Introduces a placeholder-based state tracking system to prevent duplicate
document processing during the gap between scanner queuing and processor
completion.
**Key Changes:**
1. **Placeholder Helper Functions** (`vector/placeholder.py`):
- `write_placeholder_point()` - Creates zero-vector placeholder when queuing
- `query_document_metadata()` - Queries for existing entry (placeholder or real)
- `delete_placeholder_point()` - Removes placeholder before writing real vectors
- `get_placeholder_filter()` - Filters placeholders from user-facing queries
2. **Scanner Updates** (`vector/scanner.py`):
- Replace `indexed_at` comparison with `modified_at` comparison
- Write placeholder before queuing each document
- Query per-document metadata instead of bulk-querying indexed_at
- Fixes bug where files were resubmitted every scan cycle
3. **Processor Updates** (`vector/processor.py`):
- Delete placeholder before upserting real vectors
- Ensures no duplicate points in Qdrant
4. **Query Filters** (all search files):
- Add `get_placeholder_filter()` to all user-facing queries
- Ensures placeholders never appear in search results or visualizations
- Applied to: bm25_hybrid.py, semantic.py, viz_routes.py, algorithms.py
**Architecture:**
- Placeholders use zero vectors with dimension from embedding service
- Payload includes `is_placeholder: True` flag for filtering
- Status field tracks: "pending", "processing", "completed", "failed"
- Deterministic UUIDs using uuid5 for consistent point IDs
**Impact:**
- Eliminates duplicate processing of same documents
- Fixes race condition where long-running documents get queued multiple times
- Prevents scanner from resubmitting files every scan cycle
- Maintains clean separation between in-flight and indexed documents
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
