ea108140ab55dfa0f8f1ed697a52ed3aa9054863
168
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Chris CoutinhoandClaude Opus 4.8
|
cd243ed6c3 |
fix(search): address review feedback on multi-folder path filter
- visualization.py: drop the CSV string-split branch. The Astrolabe PHP client sends path_prefixes as a JSON array, so only a list is accepted; any other shape is ignored rather than comma-split (which would corrupt folder names containing commas). - viz_routes.py: split the path_prefixes query param on newline (a comma is a valid POSIX path char; a newline is not) and pass None instead of [""] when the param is absent. - access_filter.py: widen build_base_filter_conditions' path_prefixes to Iterable[str] for consistency with normalize_path_prefixes. - ADR-027: document the newline delimiter (frontend/viz route) and JSON array (PHP->MCP body), and the PHP-side cap on list width. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Chris CoutinhoandClaude Opus 4.8
|
de6c4b360d |
feat(search): support multiple folders in the semantic-search path filter
Extend the ADR-027 Phase 2 path filter from a single path_prefix to a list of folders. The new normalize_path_prefixes() helper is the single source of truth for trimming, dropping blanks, and de-duplicating, and folds the legacy single path_prefix into the list for backward compatibility. build_base_filter_conditions() adds one MatchText to the must clause for a single folder (unchanged shape) and OR-s multiple folders via a nested Filter(should=[...]) so a file under any selected folder matches while still AND-ing against the ACL/doc_type/date conditions. path_prefixes is threaded through every search surface: the nc_semantic_search MCP tool, the visualization API (JSON body), and the viz route (CSV query param). The Astrolabe frontend folder picker that produces these lists ships in a companion astrolabe PR. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Chris CoutinhoandGitHub
|
9adbf9e8a5 |
Merge pull request #834 from cbcoutinho/fix/verify-on-read-tag-gate
fix(search): gate verify-on-read file results on vector-index tag membership |
||
|
Chris CoutinhoandClaude Opus 4.8
|
ab128bef5b |
feat(search): ADR-027 Phase 2 — file-path filter
Add a path_prefix filter to semantic search, honoured on both the MCP tool and the dense-only visualization/API paths through the shared filter contract. - build_base_filter_conditions: append FieldCondition(file_path, MatchText(path_prefix)) when set. file_path is only on doc_type == "file" points, so a non-empty path_prefix implicitly restricts to files. - Promote path_prefix to an explicit keyword param on the SearchAlgorithm ABC and both algorithms; thread it through nc_semantic_search (blank ⇒ no filter), the /api/v1 search endpoints, and the viz route. - Add a file_path TEXT payload index to _PAYLOAD_INDEX_FIELDS (no content re-index; idempotent startup migration). MatchText tokenizes on server Qdrant and matches by substring on local/embedded qdrant-client — both serve folder scoping. - Update ADR-027 (Phase 2 implemented; readiness table; semantics note). Tests. Refs ADR-027 Phase 2. Deck #177. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Chris CoutinhoandClaude Opus 4.8
|
c2c8dc1a08 |
feat(search): ADR-027 Phase 1 — modified-date range filter
Add a modified_after/modified_before date-range filter to semantic search, honoured on both the MCP tool path (BM25HybridSearchAlgorithm) and the dense-only visualization/API path (SemanticSearchAlgorithm) through one shared contract. - Promote modified_after/modified_before to explicit keyword params on the SearchAlgorithm ABC and both concrete algorithms; factor the shared placeholder+ownership+doc_type+date filter into access_filter.build_base_filter_conditions so new filters land in one place. - nc_semantic_search: accept RFC 3339 / ISO 8601 (or Unix seconds) bounds via utils.validation.parse_modified_timestamp; Annotated/Field constraints on the numeric args; explicit McpError guard for after > before. Thread the parsed bounds through the cross-app and per-doc_type dispatch. - /api/v1 search endpoints + viz route parse the same formats and 400 on bad or inverted ranges. - Add a modified_at INTEGER payload index to _PAYLOAD_INDEX_FIELDS; the idempotent _ensure_payload_indexes() startup path migrates existing collections with no content re-index. - Update ADR-027 to resolve the review feedback (validation placement, shared algorithm contract, deferral of nc_semantic_search_answer, payload index, RFC-3339-at-the-boundary rationale). Add unit tests. Refs ADR-027. Deck #177. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Chris CoutinhoandClaude Opus 4.8
|
7033c64393 |
fix(search): address PR #834 re-review (403/404 coverage + docs)
Resolve the blocking + important findings from the claude bot's re-review: - test (blocking): pin the file verifier's fail-open contract for definitive 403/404 on the tag REPORT, not just transient 503/429. A disabled systemtags endpoint commonly 403s; unlike the per-access verifiers (where 403/404 = drop), the batch file verifier must keep all results since the whole set hinges on one REPORT. Adds _http_error(403)/_http_error(404) to test_verify_files_tag_fetch_failure_keeps_all and documents the asymmetry. - docs (important): migration caveat — if vector-index was created as user_visible=False (manual occ tag:add, or pre-release), an owner's tag won't surface in a recipient's REPORT and shared-file results are silently dropped after upgrade. Note that the MCP server's get_or_create_tag defaults to user_visible=True, and how to verify/fix an existing tag. - docs (important): note the file verifier's latency scales with both the Depth:infinity folder expansion and the EXCLUDED_TAGS lookup (~2 WebDAV calls per excluded tag, fanned out under one slot); suggest lowering VERIFICATION_CONCURRENCY for large excluded-tag lists / deeply tagged trees. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Chris CoutinhoandClaude Opus 4.8
|
d4dbf01b0a |
fix(search): gate verify-on-read file results on vector-index tag membership
Verify-on-read only checked file *accessibility* (file_accessible_by_id), never tag membership, so a file removed from the `vector-index` tag (but still readable) kept surfacing in semantic search, and stale points only got evicted when they happened to rank in a search's top-K. Rework `_verify_files` to gate on current `vector-index` tag membership via a single batch `find_files_by_tag(tag, mime_type_filter="application/pdf")` REPORT per search (plus a one-shot EXCLUDED_TAGS lookup for exclusion-wins parity) — exactly what the scanner indexes. A file is kept iff it is in that set, so untagged / deleted / excluded files drop out immediately and the existing eviction wiring reclaims their Qdrant points. The gate is strict for all file results, own and shared. Mirrors the batch-fetch-and-intersect shape of `_verify_news_items` (one semaphore slot, fail-open on fetch error, malformed-id keep). - Promote the tag name to a `vector_sync_pdf_tag` Settings field (dynaconf env mapping VECTOR_SYNC_PDF_TAG) used by both scanner and verifier; drop the scanner's direct os.getenv. - Expose `find_files_by_tag` on NextcloudClientProtocol. - Rewrite the file-verifier unit tests (tagged/untagged/deleted/excluded/ fail-open/non-numeric); update the ACL + verify-on-read integration tests to seed tagged PDFs. - Amend ADR-019 and the configuration.md verify-on-read latency budget. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Chris CoutinhoandClaude Opus 4.8
|
f6ab04b2d9 |
docs: add ADR-027 for rich search filters
Add ADR-027 describing rich, chip-style filters for Astrolabe semantic search (modified-date range, doc type, path, tags). Generalises the existing doc_type filter contract (tool param -> search() kwarg -> Qdrant FieldCondition applied pre-fusion and pre-verify-on-read) and phases the rollout by payload readiness: date range ships now, path and tags defer behind a payload index / re-index. Tracking: Astrolabe Cloud POC Deck card #177 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Chris CoutinhoandClaude Opus 4.8
|
d2da195172 |
refactor(deck): address PR #826 review feedback
- Modernize the new models (DeckCardSummary, DeckCommentSummary, StackOverview, BoardOverviewResponse + the loosened unions) to PEP 604 syntax (list[...] / X | None), per CLAUDE.md. - Make status="done" exclude archived cards so open/done/archived partition the board with no overlap (a done+archived card is reported only as "archived"); document the semantics in docstrings and docs/deck.md, add a partition unit test. - deck_get_archived_stacks: pass through label/assigned_to filters (status stays archived-only by definition); note the limitation in the docstring. - Rename _validate_description_max_length → _validate_positive_length (now a generic positive-length guard). - Soften deck_get_board_overview docstring: it views board state and omits the ACL/user/label-management fields deck_get_board exposes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Chris CoutinhoandClaude Opus 4.8
|
b11103064c |
feat(deck): compact card/comment retrieval (summaries, filters, board overview)
Deck read tools returned too many tokens to be usable as boards grow — even deck_get_stacks(description_max_length=1) exceeded the MCP token limit because every card was fully serialized in list views. - Add compact projection models (DeckCardSummary, DeckCommentSummary, StackOverview, BoardOverviewResponse) and a uniform detail="summary"|"full" knob (summary default) on deck_get_cards / get_stacks / get_stack / get_archived_stacks. - Add pre-serialization filtering: status (open/done/archived/all), label, assigned_to. - Add deck_get_board_overview: board title + label legend + stacks with compact card rows + counts in a single call. - Compact comments: detail / message_max_length / newest-first order on deck_get_card_comments. - Docs + unit/integration tests. BREAKING CHANGE: deck list tools now default to detail="summary" and status="open". The include_archived_cards parameter is replaced by status (use status="all" to include archived cards); pass detail="full" to restore the previous per-card shape. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Chris CoutinhoandClaude Opus 4.8
|
b5ed1e3b4d |
fix: address PR #814 review + SonarCloud gate
SonarCloud: - Resolve 6 S5332 hotspots (http→https in test fixture URLs). - S6418: hoist the unauthenticated AsyncOpenAI placeholder to a named constant + NOSONAR (genuine non-secret; gateway ignores it when unauthenticated). - Fix two reliability bugs: None-index guard in the gateway token-cache test (S2259) and float `> 0.0` instead of `!= 0.0` in the sentinel test (S1244). - status.py idle path sleeps 0.1s instead of sleep(0) (S7491); NOSONAR on the protocol-required async no-await aclose() stubs (S7503). Claude review: - Remove three leftover debug print() calls in app.py (logger.info already covers them). - payload_backfill: drop parsed_at from the backfilled-keys docstring (it is per-document state, not a deployment scalar); add a clean 404 precondition for BasicAuth deployments without an OAuth token verifier. - status.py: task_status typed TaskStatus | None (drop type: ignore). - nats.py: TODO to thread etags for file/deck/news; note etag default → None. - factory: warn on unknown INGEST_BUS_URL scheme; raise ValueError instead of assert for the external-mode preconditions. - docs/configuration.md: document the decomposition hook-point env vars + that nats-py ships core (lazy-imported) and external+bus uses two NATS connections. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Chris CoutinhoandClaude Opus 4.7
|
e98903c502 |
fix(storage): address review on PR #799 (stale comments, docs deprecation, unit test)
claude-review on #799 flagged: 1. Stale inline comment in ``initialize()`` (line 466) still said "Postgres uses a small bounded pool". Updated to reflect both backends now use NullPool. 2. Stale ``close()`` docstring referenced pool-size starving max_connections — irrelevant with NullPool. Replaced with the NullPool-aware rationale (dispose still tears down in-flight asyncpg connections cleanly). 3. ``docs/configuration.md`` actively directed operators to tune DATABASE_POOL_SIZE / DATABASE_MAX_OVERFLOW, with worked examples and pool math. Both are now deprecated no-ops; the table entries explain the deprecation and link to PR #799. Operators reading the docs will no longer be confused into tuning settings that don't do anything. 4. ``config.py`` comment for the deprecated fields updated to record the deprecation. Validators are intentionally kept (still reject < 1 / < 0) so misconfigured deploys fail loudly rather than silently — the reviewer flagged this as a minor UX wart but explicitly "not a blocker"; the docs change in (3) keeps operators away from the config altogether. 5. New ``tests/unit/test_storage_engine.py`` with three tests: - ``test_postgres_engine_uses_nullpool`` — pins ``isinstance( engine.pool, NullPool)`` so a refactor back to QueuePool / SingletonThreadPool can't silently re-introduce the cross- event-loop crashes. - ``test_postgres_engine_ignores_pool_sizing_settings`` — setting DATABASE_POOL_SIZE / DATABASE_MAX_OVERFLOW to huge values must not change pool type (proves the deprecated fields are wired-up no-ops). - ``test_postgres_engine_missing_asyncpg_driver_message`` — guards the existing actionable-error branch when the optional ``[postgres]`` extra isn't installed. Verified: - ``uv run pytest tests/unit/`` — 1028 passed - ``uv run ruff check`` clean on the touched python files Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Chris CoutinhoandClaude Opus 4.7
|
d717c64750 |
fix(storage): address PR #798 round-4 review (NOSONAR syntax + pg_advisory_lock + engine dispose + nits)
Addresses all 8 items in the round-4 bot review plus 4 remaining SonarQube OPEN issues that were silently broken by round 3's malformed NOSONAR markers. NOSONAR syntax fix (clears the remaining 4 OPEN SQ issues) ---------------------------------------------------------- Round 3 used ``# NOSONAR S<rule_key>`` form. SonarQube Python doesn't recognize the rule-key suffix — it treats the whole thing as a malformed suppression directive (S7632) AND lets the underlying rule keep firing (S7503 on ``_Cursor.__aenter__/__aexit__``). Switch every marker to bare ``# NOSONAR``, with the rationale moved into a preceding comment block. Affected sites: - storage.py: ``_Cursor.__aenter__``, ``_Cursor.__aexit__`` - config.py: ``get_database_ssl()`` ``return False`` + ``ssl.create_default_context()`` - test_storage_logging.py: ``SENTINEL_PASSWORD_FRAGMENT`` constant - test_storage_postgres.py: three ``bob_pw_v1`` / ``bob_pw_v2`` / ``carol_pw`` literals Bot 🔴#1 — defensive NOSONAR on get_database_ssl `return False` -------------------------------------------------------------- Bot predicted S4830 fires on the operator-opt-out path. SQ output shows it doesn't currently fire, but bare NOSONAR added defensively with rationale comment. Bot 🔴#2 — defensive NOSONAR on f-string SQL -------------------------------------------- ``update_oauth_session`` builds its SET clause via ``f"{', '.join(update_fields)}"``; ``get_audit_logs`` builds its WHERE clause via string concatenation. Both are safe (the fragments only come from this function's own branches, no user input), but the patterns trip taint analysers. Annotated both with bare NOSONAR + safety comment explaining the hardcoded-fragments invariant. Note: S2077 doesn't currently fire on these; defensive. Bot 🟡#3 — pg_advisory_lock for concurrent migrations ----------------------------------------------------- Without coordination, two pods rolling-updating simultaneously can both observe ``has_alembic=False`` and both try to apply migrations from scratch — the second crashes with "relation already exists". New ``_migration_lock()`` async context manager: - On Postgres: ``SELECT pg_advisory_lock(:lock_id)`` on a fresh connection (separate from the engine pool so it survives the ``to_thread.run_sync`` worker), held across BOTH the schema-inspect AND the migration call. Without that span, two pods could each observe "no alembic_version" before either started migrating, defeating the lock. - On SQLite: yields immediately (file-level locking serializes writes natively). Lock ID derived from ``sha256(b"nextcloud-mcp-server:migrations")[:8]`` as a stable signed int64 so we can't collide with other apps sharing the same Postgres. Bot 🟡#4 — RefreshTokenStorage.close() + lifespan wiring -------------------------------------------------------- New idempotent ``close()`` method calls ``await engine.dispose()``, nulls the engine, resets ``_initialized``. Wired into both ``app_lifespan_basic`` (BasicAuth) and the OAuth lifespan teardown, each wrapped in ``try/except Exception`` with ``logger.warning`` so a buggy dispose can't block SIGTERM. Without this, pooled asyncpg connections leak server-side slots until ``idle_in_transaction_session_timeout`` reaps them — with small pool defaults and frequent k8s rolling restarts this can starve ``max_connections``. Bot 🟢#5 — is_sqlite_url docstring on :memory: ---------------------------------------------- Updated docstring to note both file-backed and in-memory forms are recognized; caller is responsible for ``:memory:`` magic. Bot 🟢#6 — db_path via make_url(...).database --------------------------------------------- Replaced ``database_url.split("///", 1)[1]`` hack with SQLAlchemy's own URL parsing. Naturally handles in-memory (``.database is None`` → falls back to ``""``). Same lazy-import pattern as the existing ``mask_db_password`` to avoid module-import-time cost. Bot 🟢#7 — _to_sync_url unrecognized-driver guard ------------------------------------------------- Pulled ``_KNOWN_ASYNC_DRIVERS = ("aiosqlite", "asyncpg")`` into a module constant. When an unrecognized ``+<driver>`` token survives the strip, emits ``logger.warning`` with the known-supported list. Behavior unchanged for valid URLs. Bot 🟢#8 — get_audit_logs SELECT * → explicit columns ----------------------------------------------------- Replaced ``SELECT *`` with explicit column list. Future schema additions stay out of the dict return. New tests --------- - ``test_close_disposes_engine``: pins the public contract — engine nulled, state reset, second call is a no-op. - ``test_concurrent_initialize_serialized_by_advisory_lock``: spawns 3 concurrent inits against a fresh schema; asserts no "relation already exists" and exactly one ``alembic_version`` row at the end. Without the lock, this reliably fails on the second concurrent task. Docs ---- - ADR-026: new "Concurrent migrations across pods" subsection documents the advisory-lock approach + lock-ID derivation. Verification ------------ - ``uv run pytest tests/unit/`` — 1025 passed. - ``TEST_DATABASE_URL=… uv run pytest tests/integration/test_storage_postgres.py -m postgres`` — 9 passed (was 7). - ``ruff check && ruff format --check && ty check`` — clean. Expected post-push: SQ scan reports 0 OPEN issues (was 4). Tracked on Astrolabe Cloud POC board, card #99. --- _This PR was generated with the help of AI, and reviewed by a Human_ Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Chris CoutinhoandClaude Opus 4.7
|
51419329b0 |
fix(storage): address PR #798 round-3 review (SonarQube + pool sizing + RETURNING test)
Round-3 fixes. Two threads: - 9 OPEN SonarQube issues caused the "E Security Rating on New Code" gate failure. The bot's diagnosis (sa.text(text_sql) → SQL injection) was a wrong guess; the actual SQ rules firing were different. - Bot's substantive concerns: pool defaults too aggressive, delete_browser_session RETURNING path untested on Postgres, schema_version legacy table created on Postgres, stale module docstring. - User's underlying question on the pool: "isn't 1 connection enough?" Right-sized to 2+5 and documented the concurrency model in ADR-026 so the rationale is durable. SonarQube quality-gate fixes (clears all 9 OPEN issues) ------------------------------------------------------- - BLOCKER S6418: rename `SECRET` constant in test_storage_logging.py to `SENTINEL_PASSWORD_FRAGMENT` + NOSONAR with rationale. - CRITICAL S3776: extract `_build_postgres_engine()` from `initialize()` (was complexity 26 > 15); incidentally creates a clean unit-test seam for engine args. - CRITICAL S4423: `ssl.create_default_context(cafile=...)` is flagged as "weak protocol" — Python 3.10+ already negotiates the strongest available protocol. Explicitly pass `purpose=ssl.Purpose.SERVER_AUTH` and NOSONAR with the Python-version rationale. - MAJOR S3358: split the TLS-mode nested ternary in the engine factory into a `_describe_ssl_arg()` helper. - MAJOR S2068 ×3: bind test app-password literals to local vars and put `# NOSONAR S2068` on the same line as the literal (anchoring requirement) instead of on the closing paren. - MINOR S7503 ×2: `# NOSONAR S7503` on `_Cursor.__aenter__/__aexit__` — they MUST be `async` per the context-manager protocol. Pool sizing right-sized (answers "why so many connections?") ------------------------------------------------------------ - `DATABASE_POOL_SIZE` default 10 → **2**. - `DATABASE_MAX_OVERFLOW` default 20 → **5**. - Per-pod max drops from 30 to 7. With 3 replicas, total = 21 connections (was 90) — well under managed-Postgres `max_connections=100`. - New INFO log at startup: `Postgres engine ready: pool_size=N max_overflow=M (per-pod max K connections)`. Surfaces the active sizing without grepping config. - New ADR-026 § "Concurrency model and pool sizing" explains asyncpg's single-flight connection semantics, the MCP workload shape (read-mostly point lookups), why-not-1 (multi-user serialization), and the tune-up/tune-down recipe. - `docs/configuration.md` table updated with new defaults + homelab-vs-prod tuning guidance, linking the ADR. RETURNING path covered on Postgres ---------------------------------- - New `test_browser_session_delete_returning` exercises the `DELETE … RETURNING user_id` path — the only RETURNING clause in the storage layer and the most dialect-sensitive SQL in this PR. Asserts both present-row (returns True, row gone) and absent-row (returns False) branches. Schema portability polish ------------------------- - `alembic 001`: gate `schema_version` table creation on `op.get_bind().dialect.name == "sqlite"`. The table exists purely to match the fingerprint of pre-Alembic SQLite databases; fresh Postgres installs no longer carry the dead legacy table. Misc polish ----------- - Module docstring: "SQLite-based" → "SQL-backed", with a sentence on the DATABASE_URL opt-in and an ADR-026 link. - Comment on `_wrap_row` noting `row._mapping` is the documented RowMapping accessor in SQLAlchemy 2.x despite the underscore. Skipped (rationale in PR reply) ------------------------------- - `_qmark_to_named` SQL-comment handling: docstring already notes the limitation; no `?` in storage SQL comments today. - Module-level `anyio.Lock()`: established precedent confirmed by the bot itself. - `get_audit_logs` `SELECT *`: pre-existing pattern, out of scope. Verification ------------ - `uv run pytest tests/unit/` — 1025 passed. - `TEST_DATABASE_URL=… uv run pytest tests/integration/test_storage_postgres.py -m postgres` — 7 passed. - `ruff check && ruff format --check && ty check` — clean. - Confirmed `schema_version` absent on fresh Postgres, still present on fresh SQLite. Tracked on Astrolabe Cloud POC board, card #99. --- _This PR was generated with the help of AI, and reviewed by a Human_ Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Chris CoutinhoandClaude Opus 4.7
|
f2b7bf132f |
fix(storage): address PR #798 review feedback (credentials, asyncpg extra, TLS, pool)
Round-2 fixes after the bot review on PR #798 plus two user follow-ups (self-signed Postgres support; asyncpg should be a PyPI extra). Folded into the same PR rather than a follow-up since the work is still unmerged. Security -------- - Mask database credentials in all 5 log call sites (storage.py × 4, migrations.py × 1) via a new `mask_db_password()` helper in config.py. Uses SQLAlchemy's `make_url(...).render_as_string(hide_password=True)` with a regex fallback so the masking path never raises. - New `tests/unit/test_storage_logging.py` asserts a sentinel password never appears in `caplog` during `RefreshTokenStorage.initialize()`. Distribution ------------ - `asyncpg` moved to `[project.optional-dependencies] postgres` so a vanilla `pip install nextcloud-mcp-server` no longer pulls in the ~5 MB C extension. The Docker image runs `uv sync --extra postgres`, so containerized deployments are unchanged. - When `DATABASE_URL=postgresql+asyncpg://...` is set on a venv missing the extra, `RefreshTokenStorage.initialize()` raises a friendly RuntimeError pointing at `[postgres]` rather than the generic ModuleNotFoundError. TLS for the Postgres backend ---------------------------- - New `DATABASE_VERIFY_SSL` + `DATABASE_CA_BUNDLE` env vars mirror the existing `NEXTCLOUD_VERIFY_SSL` / `NEXTCLOUD_CA_BUNDLE` pattern (validators in Settings.__post_init__, `get_database_ssl()` helper alongside `get_nextcloud_ssl_verify()`). `DATABASE_VERIFY_SSL=false` wins over `DATABASE_CA_BUNDLE` for incident-response convenience. - Default is **None** rather than True — keeps PR #798's behavior intact for cluster-internal Postgres that runs without TLS. Operators opt into verify-full or supply a private CA. ADR-026 records the reasoning vs the Nextcloud HTTPS default. - Engine factory in `storage.py` passes `ssl` via `connect_args` only when `get_database_ssl()` returns non-None; otherwise asyncpg's default (`prefer`) applies. - Storage logs which TLS mode is active at INFO (no secret material). Configurable connection pool ---------------------------- - `DATABASE_POOL_SIZE` (default 10) and `DATABASE_MAX_OVERFLOW` (default 20) replace the hardcoded engine values. With many replicas this can blow past managed-Postgres `max_connections=100`; tune down for large fleets. - gte-1 / gte-0 validators in __post_init__ reject 0/negative pool sizes at startup with the offending value in the error. Consistency polish ------------------ - Migration 006: convert raw `op.execute("ALTER TABLE ... ADD COLUMN")` to `op.batch_alter_table(...).add_column(sa.Column("nonce", sa.Text))` for stylistic consistency with the rewritten 001-005. Downgrade now drops the column instead of being a no-op. - `registered_webhooks.created_at` standardized from `sa.Float` to `sa.BigInteger` (all other `*_at` columns); `store_webhook()` casts `time.time()` → `int`. - `is_sqlite_url()` made case-insensitive. Testing ------- - New `tests/integration/test_storage_postgres.py::test_cleanup_expired_roundtrip` exercises `cleanup_expired_tokens`, `cleanup_expired_sessions`, and `cleanup_expired_browser_sessions` — relies on DELETE rowcount, historically dialect-tricky. - `tests/unit/test_ssl_config.py` extended with `TestDatabaseSSLSettings` + `TestGetDatabaseSSL` classes (9 new tests) mirroring the existing Nextcloud SSL tests one-for-one. Docs ---- - `docs/configuration.md` Centralized-Storage section grew the four new env vars + a homelab example with a private CA. - `docs/ADR-026` grew Distribution, TLS, and `alembic/env.py` async-pattern subsections explaining the non-obvious design choices. Helm chart counterpart in cbcoutinho/helm-charts PR #34 (separate commit on `feat/nextcloud-mcp-server-database-url`). Verification ------------ - `uv run pytest tests/unit/` — 1025 passed. - `TEST_DATABASE_URL=... uv run pytest tests/integration/test_storage_postgres.py -m postgres` — 6 passed (including new cleanup test). - `uv run ruff check && uv run ruff format --check && uv run ty check -- nextcloud_mcp_server` — clean. Tracked on Astrolabe Cloud POC board, card #99. --- _This PR was generated with the help of AI, and reviewed by a Human_ Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Chris CoutinhoandClaude Opus 4.7
|
292cbb3292 |
feat(storage): pluggable database backend via DATABASE_URL (ADR-026)
Adds a `DATABASE_URL` setting that lets `RefreshTokenStorage` run against
any SQLAlchemy async backend, primarily `postgresql+asyncpg://...` for
HA k8s deployments. Default behavior is unchanged: when `DATABASE_URL` is
unset the server falls back to the existing `TOKEN_STORAGE_DB` path /
ephemeral SQLite tempfile.
Why
---
Today every MCP pod needs its own PVC to hold the SQLite file, which
pins the Deployment to one replica and blocks horizontal scaling. With
this change, operators can point all replicas at a shared Postgres
(CNPG, RDS, etc.) and the pods become stateless. Encryption stays in
Python (Fernet); the database only sees ciphertext.
What changed
------------
- `config.get_database_url()` resolves DATABASE_URL → TOKEN_STORAGE_DB →
ephemeral tempfile in that priority order.
- `RefreshTokenStorage` builds a process-shared `AsyncEngine` in
`initialize()`. SQLite gets NullPool; Postgres gets pool_size=10,
max_overflow=20, pool_pre_ping=True. 30 aiosqlite call sites adapted
via a thin `_DBConn` / `_Cursor` / `_Row` / `_ExecuteCtx` shim so
existing method bodies need no churn beyond the connection
context-manager swap.
- 7 `INSERT OR REPLACE` statements rewritten as portable
`INSERT ... ON CONFLICT (...) DO UPDATE` (SQLite ≥ 3.24, Postgres ≥ 9.5).
- `sqlite_master` legacy-detection lookup replaced with SQLAlchemy
inspector so the path works against either backend.
- File-permission hardening + parent-dir creation gated on
`is_sqlite_url(...)` — centralized backends manage their own filesystem.
- Alembic migrations 001/002/003/005 converted from raw `op.execute(SQL)`
to portable `op.create_table()` / `op.create_index()` with SQLAlchemy
types. All timestamp columns are `sa.BigInteger` so Postgres allocates
BIGINT (unix epochs don't fit in INT4). SQLite treats BIGINT as
INTEGER, so existing deployments at revision 006 see no schema drift.
- `migrations.py` + CLI take URLs; `db {upgrade,downgrade,current,history}`
gain `--database-url / -u` alongside the legacy `--database-path / -d`.
`get_current_revision()` uses SQLAlchemy inspector instead of raw
sqlite3, so the CLI works against Postgres too.
- `docker-compose.yml` adds a `postgres-test` service under the
`postgres` profile (pinned `postgres:16-alpine` digest) for
integration testing.
- Unit storage tests parametrized over backends via shared
`tests/fixtures/storage_backend.py` — every test in
`test_app_password_storage.py` and `test_webhook_storage.py` runs
once per backend that is available. Postgres is opted in by
`TEST_DATABASE_URL`.
- New `tests/integration/test_storage_postgres.py` (5 tests, marked
`postgres` + `integration`) covers refresh-token, app-password,
OAuth-session, webhook, and audit-log paths end-to-end on Postgres.
- New `docs/ADR-026-pluggable-database-backend.md` records the decision;
`docs/configuration.md` documents `DATABASE_URL` with examples.
Out of scope
------------
- No SQLite → Postgres data migration tool (clean cutover; tokens reissue
on next login, webhooks re-register on next sync tick).
- This repo does not provision Postgres. The matching helm chart change
lives in cbcoutinho/helm-charts (database.url / existingSecret values).
Verification
------------
- `uv run pytest tests/unit/` — 1012 passed, SQLite path unchanged.
- `docker compose --profile postgres up -d postgres-test`
- `TEST_DATABASE_URL=... uv run pytest tests/integration/test_storage_postgres.py -m postgres -v`
— 5 passed.
- `TEST_DATABASE_URL=... uv run pytest tests/unit/test_app_password_storage.py
tests/unit/test_webhook_storage.py` — 50 passed (25 per backend).
- `uv run ruff check && uv run ruff format --check && uv run ty check -- nextcloud_mcp_server` — clean.
Tracked on Astrolabe Cloud POC board, card #99.
---
_This PR was generated with the help of AI, and reviewed by a Human_
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Chris CoutinhoandClaude Opus 4.7
|
bf424135a5 |
docs(adr-025): remove duplicate [login_flow] section in TOML example
Commit 4 renamed `[oauth_single_audience]` → `[login_flow]` via a global sed pass, but ADR-025's example settings.toml already had a `[login_flow]` section just above the `[keycloak]` block. The rename produced two back-to-back `[login_flow]` headers with identical contents — TOML parsers either reject the file or silently override, and a reader copying the example would land on either outcome. Dropped the now-duplicate second `[login_flow]` section (the renamed one). The earlier `[login_flow]` section retains the same content + a comment explaining the ADR-022 derivation, so no information is lost. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Chris CoutinhoandClaude Opus 4.7
|
1fa4c82fd2 |
chore: address review-round-4 nits — stale delenv, upgrade hint, in-sync notes
Four small follow-ups from the reviewer's latest pass:
- tests/unit/test_stdio.py:18: the single_user_env fixture used
monkeypatch.delenv("ENABLE_MULTI_USER_BASIC_AUTH", ...). That env var
is no longer read after the ADR-022 follow-up; switched to delenv of
MCP_DEPLOYMENT_MODE which is the canonical mode-selection input today.
Comment updated to match.
- config_validators.py: when detect_auth_mode rejects an invalid
MCP_DEPLOYMENT_MODE, surface a one-line ADR-022 migration hint if the
rejected value is exactly "oauth_single_audience" (the most common
upgrade pain — users carrying that value over from ADR-021 .env files).
Other invalid values get the regular "Valid values: …" message
unchanged.
- config.py + config_validators.py: added cross-reference comments on
both mode-resolution sites (Settings.__post_init__ and
detect_auth_mode) noting that they each compute the canonical mode
independently and must be kept in sync when a new mode is added.
Surfaces the parallel-duplication intentionally so the next maintainer
doesn't have to discover it.
- docs/ADR-021-configuration-consolidation.md:92: appended a trailing
comment to the historical "valid values" example, marking
oauth_single_audience and oauth_token_exchange as removed in ADR-022.
ADR-021 stays as the historical record; the trailer points future
readers at the current state.
No functional changes; 1009 unit tests still pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Chris CoutinhoandClaude Opus 4.7
|
ade42b55dc |
docs: clear review-round-3 nits — stale login_flow_v2, duplicates, field comments
Five small findings from the reviewer's third round, plus a SonarCloud
quality-gate failure on a test fixture.
- docs/troubleshooting.md, docs/configuration.md: six pre-PR references
to a non-existent `login_flow_v2` mode value (the actual enum value is
`login_flow`). They predated this PR but became actively misleading
once `detect_auth_mode` started raising ValueError for anything not in
the mode_map. Replaced with `login_flow` via sed.
- docs/configuration-migration-v2.md: removed a duplicate
`MCP_DEPLOYMENT_MODE=multi_user_basic` line in the troubleshooting
section (around line 447) — same shape as the round-2 duplicate
caught earlier in the migration-steps section. Also dropped the
`oauth_token_exchange` row from the mode-value table around line 364
(that enum value was removed in
|
||
|
Chris CoutinhoandClaude Opus 4.7
|
6e7c821761 |
fix(config): derive mode flags in Settings.__post_init__; address review round 2
The integration jobs for `mcp-multi-user-basic` and `mcp-login-flow`
were failing with HTTP 500s. Root cause: `get_settings()` builds a
fresh Settings on every call (not cached). Commits 3 and 4 set the
derived `enable_login_flow` / `enable_multi_user_basic_auth` flags as
a side effect of `detect_auth_mode`. detect_auth_mode runs once at
startup, against the Settings instance owned by `validate_configuration`.
Every per-request call site that does `settings = get_settings()` got
a fresh Settings with both flags at their default `False` (since the
env-var aliases were dropped), causing the multi-user dispatcher in
`context.py` to take the wrong branch and crash.
Fix: move the derivation into `Settings.__post_init__`. Every Settings
instance now carries correct flags from the moment it's constructed —
no caching needed, no mutation-after-construction race. detect_auth_mode
becomes a pure reader of the already-derived state.
The legacy env-var deprecation check moves with it. It also picks up
the reviewer's truthy-string fix: previously `os.getenv(legacy)` fired
for the literal string "false" (a non-empty Python string is truthy),
which would have errored on any user with a leftover
`ENABLE_LOGIN_FLOW=false` in their `.env`. The check now only fires
when the value lowercases to one of {"1", "true", "yes", "on"}.
- nextcloud_mcp_server/config.py: extend Settings.__post_init__ with
the legacy-deprecation block and the derived-flag derivation
(resolve mode from deployment_mode + username/password, set flags).
- nextcloud_mcp_server/config_validators.py: drop the
`_sync_derived_flags` helper (superseded by __post_init__). Drop the
legacy-env-var deprecation block (moved). `detect_auth_mode` is now
pure — no mutation. Drop the now-unused `import os`.
- tests/unit/test_config_validators.py: legacy-env-var tests now
expect `ValueError` at `Settings(...)` construction (via `get_settings()`),
not at `detect_auth_mode` call. Added two new tests:
* `test_legacy_env_var_check_ignores_falsy_strings` — pins the
truthy-string fix (reviewer round 2 finding).
* `test_derived_flags_stable_across_get_settings_calls` — regression
test pinning the integration-test fix (two consecutive
`get_settings()` calls return Settings instances with the same
derived flags).
Also reworked `test_login_flow_mode_auto_derives_enable_login_flow_flag`
to assert at-construction derivation (not the old mutation pattern).
- docs/configuration-migration-v2.md: dropped the duplicate
`MCP_DEPLOYMENT_MODE=multi_user_basic` line (review round 2 nit — a
sed artifact from commit 4).
- docs/ADR-021-configuration-consolidation.md: sed-replaced the in-body
`MCP_DEPLOYMENT_MODE=oauth_single_audience` examples with `login_flow`
(review round 2 nit — only the status header was updated in commit 4).
- tests/conftest.py: docstring comment for the multi-user-basic fixture
switched from `ENABLE_MULTI_USER_BASIC_AUTH=true` to
`MCP_DEPLOYMENT_MODE=multi_user_basic` (review round 2 nit).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Chris CoutinhoandClaude Opus 4.7
|
282c245da1 |
refactor(config)!: drop ENABLE_MULTI_USER_BASIC_AUTH env var, fail loud on legacy aliases
Same pattern as the ENABLE_LOGIN_FLOW removal in the previous commit:
the deployment mode (MCP_DEPLOYMENT_MODE) is the single source of truth
for selecting an auth flow. The ENABLE_MULTI_USER_BASIC_AUTH env-var
alias is redundant with `MCP_DEPLOYMENT_MODE=multi_user_basic`.
Unlike the ENABLE_LOGIN_FLOW removal — where silent removal was safe
because Login Flow v2 is the auto-detection default — silent removal
here would be a surprise: a user with only ENABLE_MULTI_USER_BASIC_AUTH=true
in their .env would auto-detect into LOGIN_FLOW after upgrade (wrong
runtime mode). Mitigation: detect_auth_mode now reads os.environ
directly for both legacy aliases and raises ValueError with a one-line
migration message if either is set. Applied retroactively to
ENABLE_LOGIN_FLOW as well — loud is better than silent.
- nextcloud_mcp_server/config.py:
- Drop the dynaconf env-var alias entry for ENABLE_MULTI_USER_BASIC_AUTH.
- Update the `enable_multi_user_basic_auth` field docstring to mark it
as derived / not user-settable.
- `_is_multi_user_mode()` (early-config helper, runs before Settings
is built) switched to checking MCP_DEPLOYMENT_MODE directly. Now
consistent with the canonical detection in detect_auth_mode.
- nextcloud_mcp_server/config_validators.py:
- Drop the auto-detection branch (`if settings.enable_multi_user_basic_auth`).
Selection of MULTI_USER_BASIC is now exclusively via the explicit
MCP_DEPLOYMENT_MODE branch.
- Add `enable_multi_user_basic_auth` to `_sync_derived_flags` alongside
`enable_login_flow` — both flags are now derived from the resolved mode.
- Drop `enable_multi_user_basic_auth` from
`MODE_REQUIREMENTS[MULTI_USER_BASIC].required` and from the
`forbidden` lists of SINGLE_USER_BASIC and LOGIN_FLOW (no longer
user input → no meaningful forbidden check).
- Add loud-deprecation `ValueError` block at the top of detect_auth_mode
that errors with a clear migration message when ENABLE_MULTI_USER_BASIC_AUTH
or ENABLE_LOGIN_FLOW is found in os.environ.
- tests/unit/test_config_validators.py:
- Switch ~10 fixtures from `enable_multi_user_basic_auth=True` to
`deployment_mode="multi_user_basic"` (mirrors `enable_login_flow`
treatment from the previous commit).
- Switch two `patch.dict(os.environ, {"ENABLE_MULTI_USER_BASIC_AUTH": "true"})`
blocks to use MCP_DEPLOYMENT_MODE.
- Rename `test_forbidden_multi_user_basic_auth` to
`test_forbidden_multi_user_basic_when_credentials_present` — the
scenario is now an explicit-mode + credentials conflict, not an
env-var-flag conflict.
- Add `test_legacy_enable_multi_user_basic_auth_env_var_errors` and
`test_legacy_enable_login_flow_env_var_errors` to exercise the new
loud-deprecation ValueError path.
- docker-compose.yml: mcp-multi-user-basic profile switched to
`MCP_DEPLOYMENT_MODE=multi_user_basic`.
- env.sample: replaced `#ENABLE_MULTI_USER_BASIC_AUTH=true` example with
`#MCP_DEPLOYMENT_MODE=multi_user_basic`.
- docs/authentication.md, configuration.md, troubleshooting.md,
auth-flows.md, webhook-management-guide.md,
configuration-migration-v2.md, ADR-025: replaced env-var examples
with the canonical MCP_DEPLOYMENT_MODE form.
- docs/ADR-020: marked partly superseded by ADR-022.
- CLAUDE.md: Multi-User BasicAuth section updated to set
MCP_DEPLOYMENT_MODE.
- nextcloud_mcp_server/vector/oauth_sync.py: module docstring updated.
BREAKING CHANGE: ENABLE_MULTI_USER_BASIC_AUTH is no longer read from
the environment, and setting it now raises a startup ValueError with
a migration message. Replace `ENABLE_MULTI_USER_BASIC_AUTH=true` with
`MCP_DEPLOYMENT_MODE=multi_user_basic`. The same loud-deprecation
check is also applied to the recently-removed ENABLE_LOGIN_FLOW —
replace with `MCP_DEPLOYMENT_MODE=login_flow` (or drop both;
`login_flow` is the auto-detect default when no other auth env vars
are set).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Chris CoutinhoandClaude Opus 4.7
|
df4994e860 |
refactor(config)!: derive enable_login_flow from mode, remove ENABLE_LOGIN_FLOW env var
Once OAUTH_SINGLE_AUDIENCE was renamed to LOGIN_FLOW and the validation
gate ensured the only meaningful configuration was
`MCP_DEPLOYMENT_MODE=login_flow + ENABLE_LOGIN_FLOW=true`, the two
controls became redundant. Setting the mode is sufficient; the
ENABLE_LOGIN_FLOW env var doesn't add information.
This commit makes the deployment mode the single source of truth for
the Login Flow v2 toggle:
- `nextcloud_mcp_server/config.py`: drop the `ENABLE_LOGIN_FLOW`
dynaconf env-var alias. The `enable_login_flow` field stays as an
internal attribute so the 6 runtime call sites (app.py x4,
context.py, auth/scope_authorization.py) keep working unchanged.
Updated field docstring to flag it as derived.
- `nextcloud_mcp_server/config_validators.py`:
- Drop `enable_login_flow` from `MODE_REQUIREMENTS[LOGIN_FLOW].required`.
- Drop the validation gate that required ENABLE_LOGIN_FLOW=true for
LOGIN_FLOW mode (no longer possible to misconfigure — the flag is
derived, not user input).
- Add `_sync_derived_flags()` helper called at every return path of
`detect_auth_mode` to set `settings.enable_login_flow` from the
resolved mode.
- `tests/unit/test_config_validators.py`: drop `enable_login_flow=True`
from happy-path fixtures (no longer needed — detection sets it).
Repurpose `test_login_flow_requires_enable_login_flow_flag` into
`test_login_flow_mode_auto_derives_enable_login_flow_flag` which
asserts the new auto-derivation behaviour for both LOGIN_FLOW and a
non-LOGIN_FLOW mode.
- `docker-compose.yml`: remove `ENABLE_LOGIN_FLOW=true` from the
`mcp-login-flow` and `mcp-keycloak` profiles.
- `env.sample`: remove the ENABLE_LOGIN_FLOW reference; the comment
on `MCP_DEPLOYMENT_MODE` now notes the derived flag.
- `docs/configuration.md`, `docs/authentication.md`,
`docs/login-flow-v2.md`, `docs/auth-flows.md`,
`docs/troubleshooting.md`, `docs/ADR-025-*.md`: replace
ENABLE_LOGIN_FLOW=true examples and references with
MCP_DEPLOYMENT_MODE=login_flow.
BREAKING CHANGE: `ENABLE_LOGIN_FLOW` is no longer read from the
environment. Anyone who relied on `ENABLE_LOGIN_FLOW=true` to activate
Login Flow v2 should set `MCP_DEPLOYMENT_MODE=login_flow` instead (or
rely on it being the default when no other auth env vars are set).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Chris CoutinhoandClaude Opus 4.7
|
c74ef014ee |
docs(adr-022): mark Accepted, update env/compose/migration docs for LOGIN_FLOW rename
Follow-up to the LOGIN_FLOW rename. The user-facing surface area — env.sample, docker-compose.yml mcp-login-flow profile, migration guide, ADR statuses, and the running.md boot-log examples — all need to refer to `login_flow` rather than the deprecated `oauth_single_audience` string. - docker-compose.yml: add explicit MCP_DEPLOYMENT_MODE=login_flow to the mcp-login-flow profile (no longer relying on auto-detection). - env.sample: update the deployment-mode list and example, dropping the removed `oauth_token_exchange` and pointing at ADR-022 for the rename rationale. - docs/ADR-022: flip Status to Accepted with a note that this PR implements step 1 (rename + validation gate). - docs/ADR-021: note that it has been partly superseded by ADR-022 (the oauth_single_audience naming is no longer accurate); cross-link. - docs/ADR-025: drop oauth_single_audience/keycloak from the dynaconf validator example and the [oauth_single_audience] TOML section. - docs/configuration-migration-v2.md: bulk-replace oauth_single_audience → login_flow throughout (sed -i). - docs/running.md: re-collapse the per-mode boot-log subsections (added during the closed PR #786 workaround) back into a uniform "<mode>"-substitution block — now correct after this PR's logging cleanup at app.py:1172. No code changes in this commit. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Chris CoutinhoandClaude Opus 4.7
|
5f01312cc4 |
docs: correct OAuth enum value and split boot-log block by mode
Reviewer found two accuracy issues in the rewritten "Check Deployment Mode" section: - The AuthMode.OAUTH_SINGLE_AUDIENCE enum value is `oauth_single`, not `oauth_single_audience` (config_validators.py:28). A user grepping their container logs would have found nothing. - The "Configuring MCP server for <mode> mode" line was presented as a uniform <mode> substitution, but app.py:1170 hardcodes the literal string `OAuth mode` for OAuth, while app.py:1239 uses the enum value for the two BasicAuth modes. Split the boot-time block into per-mode subsections so each one shows the actual literal text users will see, and add a one-line note calling out the OAuth string difference. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Chris CoutinhoandClaude Opus 4.7
|
e78e191818 |
docs: fix /health → /health/live, refresh stale mode-detection log examples (#766)
Issue #766 reported that the running.md quick-start tells users to `curl http://localhost:8000/health`, which returns 404 — the server only registers `/health/live` and `/health/ready` (K8s-style probes). The same section also listed BasicAuth and OAuth startup log lines (`BasicAuth mode detected …`, `OAuth mode detected …`) that no longer exist anywhere in the codebase. Update running.md and troubleshooting.md to point at the real endpoints, explain liveness vs readiness, and replace the fictional log examples with messages the server actually emits today. Also clarify that the per-session BasicAuth messages only appear after the first MCP client connects, which is the second symptom the reporter hit. Docs-only change; code paths and endpoint surface unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Chris CoutinhoandClaude Opus 4.7
|
8246d9a088 |
fix(vector): address PR review round 17 + local-mode collection-creation regression
Round 17 reviewer (🟡 Important): 1. docs/configuration.md degraded-migration runbook said `doc_id backfill failed on …` but the actual log line in qdrant_client.py:415 is `doc_id backfill scroll failed on …`. Operators grepping the runbook string would have missed it. Insert the `scroll` qualifier. 2. _create_one_payload_index returned True on the 400 schema-conflict path, so a wrong-type index discovered at create time skipped the consolidated `Payload index creation incomplete` summary — but a wrong-type index discovered via the existing-schema check at line 195-206 did fire it. Tenants whose payload_schema is hidden from their JWT (Qdrant Cloud collection-scoped tokens) only ever observe the create-time path, so they never saw the operator-level summary. Return False so the summary fires in both cases. 3. docs/configuration.md said the upgrade-time delay was `proportional to point count while writes are issued` — overstating the cost. Writes are proportional to int-typed points only; the scroll itself is proportional to total point count. Reword. Local-mode collection-creation regression (root-cause of failing single-user / login-flow / multi-user-basic CI jobs): PR #779 changed the existence probe in get_qdrant_client from collection_exists() (returned bool in both modes) to get_collection() + except UnexpectedResponse(status_code=404). The HTTP-mode client raises UnexpectedResponse with a 404 body, but the local/in-memory client raises ValueError(f"Collection {name} not found") — see qdrant_client/local/async_qdrant_local.py. The narrow except clause let the ValueError propagate, app.py's lifespan re-raised as RuntimeError, and the mcp container crashed on first start. Catch ValueError too, with a `not found` substring guard so genuine programming bugs (bad collection_name, etc.) still surface. Tests: extend the existing 400-path test to assert the new failed_fields contract; add two get_qdrant_client unit tests pinning the local-mode VE catch (positive case + propagation case). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Chris CoutinhoandClaude Opus 4.7
|
b97ac23228 |
fix(vector): address PR review round 4 — backfill resilience + degraded-mode docs
- Remove three stale `# Use numeric file ID` / `# Pass file path` comments in scanner.py. file_id is already normalized to str() above each call site, so the inline comments mislead readers. - Wrap `_backfill_doc_id_to_string` scroll loop + sentinel upsert in try/except Exception. The qdrant_client singleton is assigned before this migration runs, so a transient scroll failure was leaving the process holding a usable client with int payloads permanently unbackfilled until the next restart. Catch broadly, log ERROR with exc_info, and return without writing the sentinel — next process restart retries from scratch. - Note `:memory:` mode behavior near the sentinel constants so future readers don't read the every-start scroll as a bug. - Document the two degraded-migration ERROR log signals in docs/configuration.md so operators know when a clean restart is required to recover indexing. - Add unit test asserting scroll-time exceptions are logged and swallowed without writing the sentinel. Closes round-4 review feedback on PR #773. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Chris Coutinho
|
02744a50e0 | Merge remote-tracking branch 'origin/master' into fix/qdrant-doc-id-keyword-index | ||
|
Chris CoutinhoandClaude Opus 4.7
|
b5b4025bb4 |
fix(vector): address PR review round 2 — status branching, doc_id guard, doc restore
- _ensure_keyword_payload_indexes: distinguish 400 (schema conflict, warning)
from other status codes (5xx/network, error) so a transient outage doesn't
silently leave the collection unindexed.
- build_search_result_from_point: use .get("doc_id") + return None on missing
instead of KeyError-crashing the search; reverse metadata merge order so
payload-derived chunk_index/total_chunks win over caller-supplied extras.
- docs/configuration.md: restore the OpenAI/Mistral/Bedrock/Simple provider
sections + reference-table rows that were dropped in the rebase. Reword
the "Startup migrations" bullet to describe what the code actually does
(no sampling — full scroll, zero writes when clean). Add operator note
about the SemanticSearchResult.id TypeError path.
- tests: pytest.approx for float equality (Sonar python:S1244); coverage
for non-400 → ERROR, payload={doc_id: None}, and missing doc_id key.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Chris CoutinhoandClaude Opus 4.7
|
719b3b5034 |
fix(vector): normalize doc_id to str + add Qdrant keyword payload indexes
Production was logging two cascading classes of Qdrant errors against the
welcomed-malamute deployment:
1. HTTP 400 — "Bad request: Index required but not found for \"doc_id\" of
one of the following types: [keyword]". The collection was created via
create_collection() with no payload indexes, so any FieldCondition
filter on doc_id failed at the Qdrant layer (placeholder writes/reads,
eviction, search context lookups).
2. Compounding the missing index, producers wrote a mix of int and str
doc_ids: webhook_parser stringified node_id, scanner stringified note
IDs, news IDs, and deck card IDs — but the file scanner passed the
numeric file_id through unchanged. A keyword index would not have
covered both kinds even if it had existed.
This change:
- Normalizes doc_id to str at every producer site (scanner.py:459,
DocumentTask.doc_id, indexed_*_ids reads from Qdrant).
- Tightens str|int annotations to str across placeholder.py,
eviction.py, search/verification.py, search/context.py,
SearchResult.id, and the auth/api visualization endpoints.
- Defensive str() coercion on doc_id reads in semantic.py /
bm25_hybrid.py / vector/visualization.py for the transition window
before the backfill runs.
- Adds an idempotent startup migration in get_qdrant_client():
- _ensure_keyword_payload_indexes creates KEYWORD indexes for
doc_id, user_id, and doc_type (tolerates "already exists" 400s).
- _backfill_doc_id_to_string scrolls the collection once and rewrites
int doc_ids to str. Skipped after a quick sample shows no legacy
int payloads.
- Public API preserved: SemanticSearchResult.id stays int via explicit
int(r.id) narrowing in server/semantic.py — surfaces a TypeError with
actionable context if a future doc_type ships non-numeric ids.
- Documents the startup migration in docs/configuration.md.
Tests: 11 new unit tests in tests/unit/vector/test_qdrant_client.py
covering happy path / already-exists / unrelated-400 for the index
helpers, and sample-skip / mixed-batch rewrite / payload=None edge cases
for the backfill. 889 unit tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Chris CoutinhoandClaude Opus 4.7
|
adcf13f082 |
refactor(providers): address PR #772 review round 3 — hermetic test, lazy logging, defensive-guard tests
- test_registry.py: stub `mistralai.client.Mistral` in `test_registry_mistral_wins_over_ollama`, mirroring the sibling picker test, so the test doesn't depend on the SDK accepting arbitrary keys. - openai.py: convert remaining f-string `logger.info(...)` calls to lazy `%s` formatting, aligning with the pattern in mistral.py and the repo's logging convention. - test_mistral.py: add four tests covering the defensive RuntimeError guards in `embed()` and `_embed_batch_request()` — empty response.data, single null embedding, batch null embedding, and count-mismatch. - docs/configuration.md: add `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` rows to the env-var reference table; they were already mentioned in prose but missing from the table. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Chris CoutinhoandClaude Opus 4.7
|
20f1770794 |
refactor(providers): address PR #772 review round 2 — guard, naming, docs, tests
- _retry.py: replace `assert last_error is not None` with explicit `if last_error is None: raise RuntimeError(...)` so the original rate-limit error is preserved under `python -O`. - openai.py: drop the `_retry_factory` alias chain; rename the bound decorator to `_retry_429` to match the pattern in mistral.py. - mistral.py: comment the imports so future reviewers understand why `from mistralai.client import …` is the canonical path on 2.x (no top-level `__init__.py`; no `mistralai.models` subpackage either). - docs/configuration.md: add `OPENAI_GENERATION_MODEL` and `OLLAMA_GENERATION_MODEL` rows to the env-var reference table. - test_mistral.py: add direct unit test for the `_is_rate_limit` predicate (429 → True, 500 → False, missing-attr → False). - test_registry.py: stub `mistralai.client.Mistral` in the registry picker test, mirroring the Ollama sibling, so the test doesn't depend on the SDK accepting arbitrary keys. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Chris CoutinhoandClaude Opus 4.7
|
3268a13d11 |
feat(providers): add Mistral embedding provider, route registry through dynaconf
Adds a hosted Mistral embedding option (mistral-embed, 1024-dim) alongside the existing Bedrock / OpenAI / Ollama / Simple providers. Implementation mirrors OpenAIProvider: lazy dimension detection with a known-models lookup, chunked batch requests, defensive index sort, and a 429-aware retry decorator. In the same change, ProviderRegistry switches from os.getenv to the dynaconf-backed Settings dataclass so all five providers share a single configuration path. config.py gains the previously-uncovered Bedrock keys, the new Mistral keys, the missing OPENAI_GENERATION_MODEL / OLLAMA_GENERATION_MODEL, and SIMPLE_EMBEDDING_DIMENSION. Auto-detection priority: Bedrock → OpenAI → Mistral → Ollama → Simple. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Chris CoutinhoandClaude Opus 4.7
|
22ed9e99a0 |
feat(webdav): add tag-based file exclusion (#710)
Hide sensitive files/folders from the WebDAV MCP tool surface by tagging them with a configured Nextcloud system tag. Defence-in-depth control for users who connect LLMs to accounts holding contracts, medical records, credentials, etc. A new EXCLUDED_TAGS env var (comma-separated tag names, empty by default) gates an exclusion layer that runs at the start of every WebDAV tool call: tag names are resolved to tag IDs, those IDs are expanded to the set of tagged paths, then listings/searches are filtered and read/write/delete/move/copy operations on excluded paths raise ToolError. Tagged folders exclude their descendants via prefix match. Empty EXCLUDED_TAGS disables the feature entirely. The threat model is preventing accidental data exfiltration via the LLM tool surface — not hiding files from a determined operator. The docs explicitly recommend creating exclusion tags with user_assignable=false so the credentials the MCP server uses cannot remove the tag. Implementation: - config.py: add `excluded_tags` to _DEFAULTS, Settings, and the _field_map alongside other comma-separated env vars. - client/webdav.py: get_files_by_tag now requests <d:resourcetype/> and surfaces is_directory so tagged directories can recursively exclude descendants. - server/tag_exclusion.py (new): get_excluded_tag_names, get_excluded_file_paths, is_path_excluded. - server/webdav.py: exclusion guards in all 11 WebDAV tools; read/write/create/delete/move/copy raise ToolError, list/search tools silently filter excluded entries. Existing f-string log calls converted to lazy %-style. - tests: 17 new unit tests covering path-matching edge cases (shared-prefix non-match, descendants of excluded dirs), tag-name parsing, and get_excluded_file_paths with mocked WebDAV; 1 new client test asserting <d:resourcetype/> -> is_directory parsing. - docs/configuration.md: new "Tag-Based File Exclusion" section with per-tool effect table, security guidance, and per-call cost note. - README.md: feature mention under Key Features. Closes #710. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Chris CoutinhoandClaude Opus 4.7
|
e2955e8246 |
fix(auth): address PR #758 round-5 medium/low review
Three findings from the latest review on #758 (1 medium, 2 low): Medium: - browser_oauth_routes.oauth_logout: move delete_browser_session into a finally block so an error from delete_refresh_token can no longer leave an orphan browser_sessions row. The orphan was not exploitable (SessionAuthBackend rejects sessions without a live refresh token), but it lingered until the hourly cleanup cron — a correctness gap. New regression test pins the fix. Low: - oauth_callback_nextcloud: drop redundant ``or None`` from ``expected_nonce=nonce``. ``nonce`` is already ``str | None`` and ``secrets.token_urlsafe`` never produces an empty string, so the coercion was a no-op that could mislead future readers into thinking empty-string was a valid skip-the-check path. - storage.RefreshTokenStorage.initialize: fail fast at startup when SQLite < 3.35, since ``DELETE ... RETURNING`` (used in ``delete_browser_session``) needs that minimum. Ubuntu 20.04 ships 3.31 and would otherwise hit OperationalError on every logout. Prerequisite also documented in docs/installation.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Chris CoutinhoandClaude Opus 4.7
|
da60322597 |
docs(login-flow): add external-IdP setup section
Calls out the apps-to-install matrix (user_oidc required, oidc skip, astrolabe optional), the OIDC clients to register and what each is for, the per-app scope advertisement requirement on the IdP side, and the "OAuth succeeded but Nextcloud returns 401" diagnosis path. Mined from the cbcoutinho/nextcloud-mcp-server#752 thread. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Chris CoutinhoandClaude Opus 4.7
|
8a2626da6c |
refactor(search): address PR #750 round 8 review feedback
- Rename `verified_count` → `verified_chunk_count` to make the count granularity explicit at the field name (chunks vs unique docs). - News verifier now fails open *per-item* on non-numeric stored doc_ids (matches notes/files/deck shape); a single bad id no longer rescues definitively-missing siblings from eviction. - Update note-verifier integration test to use string doc_ids end-to-end to match production storage (scanner.py:241 stringifies note ids). - Add regression test for the closed-task-group race guard in `verify_search_results` so the RuntimeError swallow is locked in. - Convert remaining f-string logger calls in `server/semantic.py` to lazy %-style formatting (per repo convention). - Document `evict_on_missing` as a developer/test flag (no env var) and flag the `get_file_info` 404→raise contract change in its docstring. - Add a TODO(ADR-019) breadcrumb for the hardcoded 2× over-fetch so future tuning has a clear hook. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Chris CoutinhoandClaude Opus 4.7
|
e8df6003c5 |
refactor(search): address PR #750 round 6 review feedback
Closes out the remaining nits flagged in the round-6 review. Critical: - _verify_files contract comment now enumerates all None-return cases (404 + malformed PROPFIND XML) and documents the false-eviction trade-off; self-healing via re-indexing recovers - int(r.id) cast at the SemanticSearchResult boundary now raises a TypeError with explicit doc_type/value context instead of bubbling up as an opaque "Search failed: ..." McpError Design observations: - nc_semantic_search_answer docstring documents the per-note round-trip cost from the post-verification race guard - News verification latency hint added to configuration.md - SemanticSearchResponse exposes verified_count + dropped_count so short result pages on high-ghost-density indexes are distinguishable from genuine scarcity. verify_search_results now returns (kept, dropped_count); production caller and tests updated Minor: - Comment clarifies the .get() fallback in verify_search_results is defensive only (run_verifier always populates the entry) - Eviction task-group guard narrowed from except Exception to except RuntimeError (the only documented failure mode of TaskGroup.start_soon on a closed group) - Indexer logs a warning when a deck_card task is missing board_id/stack_id, surfacing data-quality issues at index time rather than at verification time - New unit test covers the news verifier's non-numeric-id fail-open path (one bad doc_id keeps the entire batch) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Chris CoutinhoandClaude Opus 4.7
|
ffcca23a7b |
refactor(search): address PR #750 round 5 review feedback
Tightens verifier consistency, closes test gaps, hardens the fire-and-forget eviction snapshot, and routes the new concurrency knob through Settings. - Pre-flight ``int()`` guard in ``_verify_notes`` mirrors ``_verify_deck_cards``, so a non-numeric note id produces a type-specific log line instead of falling through to the generic "unexpected error" branch. - Adds explicit 403 tests for the file and news verifiers (symmetry with the existing notes/deck 403 tests) plus a ``non_numeric_id_keeps`` test. - ``AppContext`` and ``OAuthAppContext`` no longer snapshot ``_vector_sync_state.eviction_task_group`` at lifespan-yield time. Both expose it as a ``@property`` that reads the singleton dynamically, removing the order-sensitive race where a future startup-ordering change could silently degrade fire-and-forget eviction to inline forever. - Adds ``verification_concurrency`` (env var ``VERIFICATION_CONCURRENCY``, default 20) to ``Settings`` with a dynaconf validator; ``verify_search_results`` resolves the cap lazily from settings when the caller doesn't override it. - Enriches the news verifier TODO to call out that ``batch_size=-1`` is intentional — a numeric ceiling would silently break correctness because any item beyond the cap would be missing from ``present_ids`` and dropped. - Updates ``Optional[TaskGroup]`` to ``TaskGroup | None`` per project style. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Chris CoutinhoandClaude Opus 4.7
|
926722b09d |
refactor(search): address PR #750 round 4 review feedback
- Guard eviction_task_group.start_soon against shutdown race so a RuntimeError on a closed group never surfaces as a search error. - Correct ADR-019 news_item row: there is no per-item REST endpoint; verification batches via get_items(batch_size=-1) and intersects. - Modernize models/semantic.py typing to PEP 604 / lowercase generics per CLAUDE.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Chris CoutinhoandClaude Opus 4.7
|
aa4b9498a1 |
refactor(search): address PR #750 round 3 review feedback
- _verify_deck_cards: hoist int(board_id|stack_id|doc_id) out of the generic except Exception into an explicit try/except (TypeError, ValueError) before the network call, mirroring _verify_news_items. Malformed payloads now log a specific warning instead of "unexpected error". - _verify_news_items: add TODO(perf) above the get_items(batch_size=-1) call to mark the known fetch-all cost as a future profiling target. - SemanticSearchResult.id: revert from int|str back to int. The internal SearchResult.id stays int|str for forward-compat; the MCP response model narrows at the boundary. server/semantic.py casts r.id to int when constructing the response so future string-id types fail loudly here instead of silently widening the public API. - nc_semantic_search: replace the terse "extra for access filtering" comment with an ADR-019 NOTE block explaining the 2x over-fetch trade-off and the ghost-density under-delivery case (self-heals via lazy eviction). - tests/integration/test_verify_on_read.py: extend the module docstring to call out that only the note verifier is exercised against real Nextcloud, while file/deck_card/news_item are unit-only — documenting the suite split for future contributors. - ADR-019: rewrite "Module shape", "Verifier registry", example verifier, and "Deduplication" sections to match the shipped BatchVerifier interface (was per-id Verifier in the original draft). Add a "Why batch?" paragraph explaining the design choice. Update implementation checklist — every item is now [x] with corrected verifier names (plural) and the eviction module path (vector/eviction.py). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Chris CoutinhoandClaude Opus 4.7
|
21e5608a39 |
refactor(search): address PR #750 round 2 review feedback
Implements fire-and-forget eviction (ADR-019 §"Lazy eviction"): the search response no longer waits on Qdrant deletes, instead spawning evict() on a long-lived lifespan-owned task group. Falls back to inline eviction in modes without vector sync and in unit tests. Also: harden _verify_news_items against non-numeric ids (fail open instead of crashing the verifier); document the get_file_info None-on-404 contract; add INDEXED_DOC_TYPES single source of truth in vector/scanner.py referenced by the CI-guard test; write a Verify-on-Read Latency Budget section in docs/configuration.md covering the unbounded news.get_items fetch. Closes the two remaining ADR-019 implementation checklist items. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Chris CoutinhoandClaude Opus 4.7
|
7784ec02d7 |
refactor(search): address PR #750 review feedback
- Cap all_results to limit*2 after sort in the per-doc_types branch of nc_semantic_search to bound over-verification (was unbounded N-types). - Switch BatchVerifier from (client, doc_ids, user_id) to (client, results, semaphore). Verifiers now read file paths and deck board/stack ids from SearchResult.metadata instead of doing fresh Qdrant scrolls — eliminates one duplicate round-trip per file/deck-card verification. - Bound per-id verification concurrency with a shared anyio.Semaphore (default 20, matching server/semantic.py context-expansion convention). Prevents httpx pool exhaustion / rate limiting on large search pages. - Propagate stack_id from Qdrant payload to SearchResult.metadata in both bm25_hybrid.py and semantic.py (board_id was already propagated). - Drop now-unused _resolve_file_path / _resolve_deck_metadata helpers. - Drop redundant int(d) in requested predicate from _verify_news_items. - Rewrite eviction comment to be honest about inline (not background) execution and the resulting latency coupling. - ADR-019 status: Proposed -> Accepted. - Add news property to NextcloudClientProtocol. - Widen SearchResult.id and SemanticSearchResult.id to int | str to match BatchVerifier signature and document support for future string-id types. - Flip openWorldHint to True on nc_semantic_search_answer (it calls into Nextcloud via nc_semantic_search). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Chris CoutinhoandClaude Opus 4.7
|
d90e793d19 |
feat(search): verify-on-read for semantic search results (ADR-019)
The vector index lags Nextcloud (5-min webhook cron + scanner interval), producing ghost records for deleted/unshared documents until the next reconciliation. Verify each unique document against Nextcloud at query time, drop inaccessible results, and lazily evict the corresponding Qdrant points. Per-doc_type batch verifiers: notes/files/deck cards run concurrently per id; news items use a single fetch + intersect to avoid the per-item fetch-all amplification. Transient errors fail open (keep result, log warning) — only definitive 4xx drops. Multiple chunks of the same doc collapse to one verification call. Wired into nc_semantic_search before the limit trim and before context expansion. nc_semantic_search_answer's per-note re-fetch retained as a sub-second race guard since verification now happens upstream. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Chris CoutinhoandClaude Opus 4.7
|
367816e4e4 |
docs: round-4 reviewer nits
Address four small items from the latest PR #743 review: - login-flow-v2.md Compose example: add an inline comment + follow-up note pointing readers at Docker secrets for TOKEN_ENCRYPTION_KEY (the snippet is likely to be copy-pasted into production). - auth-flows.md: rename the third column in the Astrolabe → MCP Server diagram from "Nextcloud OIDC" to "OIDC Provider" so the diagram matches the multi-IdP framing in the surrounding prose. - login-flow-v2.md OAuth Endpoints section: rewrite the ambiguous "token issuance still comes from the IdP" line to make the cryptographic separation explicit — the MCP server exposes /token, but tokens are signed by the IdP's key and validated against its JWKS; the MCP server has no signing keys of its own. - README.md auth bullet: replace the jargony "OAuth-to-MCP supported, with app-password conversion to Nextcloud" with the reviewer's clearer wording: "MCP clients authenticate via OAuth, the server handles Nextcloud app passwords transparently". Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Chris CoutinhoandClaude Opus 4.7
|
0c6b766e7e |
docs: fix scope naming and round-3 reviewer feedback
The docs claimed scopes are mcp:-prefixed (mcp:notes.read, mcp:notes.write) and that the notes.* pair "covers all Nextcloud apps". Both are false. Per @require_scopes decorators across nextcloud_mcp_server/server/, scopes are unprefixed and per-app: notes.read/write, talk.read/write, files.read/write, calendar.read/write, contacts.read/write, deck.read/write, news.read, tables.read/write, cookbook.read/write, todo.read/write, collectives.read/write, sharing.write, semantic.read, plus standard OIDC scopes. Changes: - login-flow-v2.md: replace the false 2-row "covers all apps" scope table with the real per-app reference (links to scope_authorization.discover_all_scopes() as authoritative source); strip mcp: prefix from intro paragraph, sequence diagrams, @require_scopes example, WWW-Authenticate header example. Also fix sticky-session keying advice per reviewer: route on user identity (sub claim) rather than the raw bearer token, since tokens rotate on refresh. - auth-flows.md: clarify "Astrolabe (hosted UI) → MCP" matrix column header; strip mcp: from sequence diagram and key characteristics bullet; correct "issued by MCP server" to "issued by configured IdP" on the Login Flow v2 token. - authentication.md: strip mcp: from the high-level diagram and scope-enforcement prose; cross-link to the scope reference. - configuration.md: add NEXTCLOUD_OIDC_CLIENT_ID, NEXTCLOUD_OIDC_CLIENT_SECRET, and OIDC_DISCOVERY_URL to the Login Flow v2 vars table — these were undocumented in the table after the round-2 multi-IdP fix. - running.md: drop deprecated `version: '3.8'` from compose snippets (Compose v2 ignores it and emits warnings). - testing-oidc-consent.md: fix sample authorize URL and consent description to use real scope names instead of mcp:-prefixed ones (the manual test as written would have failed with invalid_scope). - CLAUDE.md: replace dead links to deleted oauth-architecture.md, oauth-setup.md, and audience-validation-setup.md with login-flow-v2.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Chris CoutinhoandClaude Opus 4.7
|
d21be6d5e1 |
docs: generalize OIDC framing to support multiple IdPs
Previous round narrowed the framing too far in the other direction —
made it sound like Nextcloud OIDC is *the* IdP. The MCP server
actually supports any OIDC-compliant provider (Nextcloud's built-in
OIDC, Keycloak, AWS Cognito, Auth0, etc.) selected via
`OIDC_DISCOVERY_URL`. `NEXTCLOUD_OIDC_CLIENT_ID/SECRET` are generic
OIDC client credentials despite the Nextcloud-flavored naming.
Code references:
- IdP discovery: app.py:607-668 (auto-detects integrated vs external
by comparing discovered issuer to NEXTCLOUD_HOST)
- JWKS: unified_verifier.py:71-73 (dynamically discovered, not
hard-coded to Nextcloud)
- IdP selection knob: OIDC_DISCOVERY_URL (config.py)
Changes:
- login-flow-v2.md: redraw "How It Works" diagram to show the IdP as
a separate component; replace "Nextcloud OIDC" with "configurable
IdP" framing throughout; add OIDC_DISCOVERY_URL to the env-var
reference; clarify NEXTCLOUD_OIDC_CLIENT_ID/SECRET are generic OIDC
creds; rename "OAuth Endpoints" subtitle to point at "the configured
IdP".
- running.md: rewrite the OAuth Mode intro and Quick Start note to
mention IdP configurability and OIDC_DISCOVERY_URL.
- configuration.md: update Best Practices "For Production" multi-user
bullet to reference the IdP selector and generic-creds caveat.
- auth-flows.md: generalize Astrolabe-flow and Login Flow v2
characteristics bullets — IdP and JWKS source are configurable.
- keycloak-multi-client-validation.md: REMOVE the "deprecated"
banner I added in
|
||
|
Chris CoutinhoandClaude Opus 4.7
|
319e82774e |
docs: correct OIDC architecture framing for Login Flow v2
The previous round of review feedback rested on a misunderstanding — that the MCP server is "the OAuth issuer" under Login Flow v2 and that NEXTCLOUD_OIDC_CLIENT_ID/SECRET are external-IdP-only. Code says otherwise (app.py:619/625/703-717, unified_verifier.py:72): - The MCP server is an OIDC relying party of Nextcloud OIDC. Tokens are signed by Nextcloud and validated against Nextcloud's JWKS in all modes — the server has no private signing keys. - Static NEXTCLOUD_OIDC_CLIENT_ID/SECRET are the preferred way to register the MCP server as that relying party; RFC 7591 DCR is a fallback when both are unset. - Login Flow v2 layers per-user app-password acquisition on top — it governs the MCP→Nextcloud data leg, not the relying-party setup. This commit reverts the inaccuracies introduced by |
||
|
Chris CoutinhoandClaude Opus 4.7
|
35c115ead6 |
docs: address remaining Login Flow v2 review feedback
Round-2 cleanup of PR #743 review comments not covered by
|
||
|
Chris CoutinhoandClaude Opus 4.7
|
d153e96520 |
docs: address Login Flow v2 review feedback
Fix issues raised by reviewer on PR #743: - troubleshooting.md: renumber "Getting Help" steps (4→3, 5→4) after earlier consolidation left a gap - installation.md: drop stale "OIDC app" prerequisite; admin access is now optional under Login Flow v2 (works on stock Nextcloud 16+) - semantic-search-architecture.md: rename VECTOR_SYNC_ENABLED to ENABLE_SEMANTIC_SEARCH in the Status callout (renamed in v0.58.0) - configuration.md: remove Quick Start references to deprecated oauth-multi-user / oauth-advanced templates and point to login-flow-v2.md; update "OAuth, Multi-User BasicAuth" label to "Login Flow v2, Multi-User BasicAuth" - auth-flows.md: fix background-sync diagram so Encrypt+persist step no longer crosses into the Nextcloud column Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |