The Astrolabe PHP→Vue settings refactor dropped three stable element ids
(#mcp-enable-background-button, #mcp-revoke-background-button,
#mcp-revoke-background-form) that the multi-user-basic integration suite
drives the background-sync enable/disable/revoke flows through. Their
absence timed out the 5s Playwright locators and failed four tests:
- test_astrolabe_multi_user_background_sync::test_multi_user_astrolabe_background_sync_enablement
- test_astrolabe_multi_user_background_sync::test_revoke_background_sync_access
- test_astrolabe_chunk_context::test_chunk_context_endpoint_uses_app_password
- test_astrolabe_plotly_visualization::test_astrolabe_plotly_visualization_with_basic_auth
(the latter two enable background sync via complete_astrolabe_authorization
before exercising the app-password / indexed-search paths).
Two-part fix:
1. Bump the astrolabe submodule to v0.20.1 (cbcoutinho/astrolabe#116),
which restores the three element ids on the refactored NcButtons.
2. Defense-in-depth in the test helpers: resolve the enable/revoke buttons
by their stable id first, falling back to the button's accessible name
so a future id rename degrades to a slower-but-working lookup instead of
a hard timeout. Avoids a combined `.or_()` locator, which would
strict-mode-violate (the id button also matches by text).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
🟡 The `worker` command never called initialize_document_processors(), so a
worker pod with ENABLE_UNSTRUCTURED/TESSERACT/CUSTOM configured silently ran
PyMuPDF-only (only the import-time-registered processor). The always-on API pod
registers them in its lifespan; the worker has its own startup path, so call
initialize_document_processors() there too (before run_worker_async).
🟢 Drop the unused get_database_url monkeypatch in the Postgres integration
fixture (build_app_for_url passes the URL explicitly; only the ssl lookup needs
pinning).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Round-4 review (non-blocking) items:
- get_procrastinate_conninfo: warn on an empty connect_timeout= value (it falls
back to the 10s default); preserve an explicit connect_timeout=0.
- Document the _doc_queueing_lock user_id invariant (NC rejects ':' in usernames).
- docs/configuration.md: note that `db downgrade` leaves procrastinate's tables
in place and how to drop them on a full teardown.
- reclaim_stalled_ingest_jobs: debug heartbeat log when nothing is stalled.
- Drop the redundant list() wrap in the integration stalled-jobs assertion.
Logging pattern: define a module-level `logger = logging.getLogger(__name__)`
and use it instead of function-local or inline getLogger(__name__) calls
(config.py, config_validators.py, tests/.../test_scope_authorization.py). The
test file's dev-only `scripts.*` import gets a ty: ignore since it resolves via
sys.path at runtime, not as an installed package.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Stop excluding tests/ from the ty-check pre-commit hook so touched test files
are type-checked. Fix the new ingest tests under the now-active check:
- cast duck-typed JobContext / App test doubles to their declared types;
- narrow the gated Postgres fixture's str | None URL (pytest.skip isn't modelled
as NoReturn by ty).
Pre-existing type issues in untouched test modules are unaffected (the hook
checks only changed files); they'll be cleaned as those files are next touched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Re-architect document ingest from the shared NATS-glued document-processor to a
per-tenant, in-process model owned by nextcloud-mcp-server (Deck #183). The MCP
server now owns both sides of ingest:
- Producer (api role): the scanner defers one job per changed document into the
app's Postgres via procrastinate (queueing_lock dedup; no execution lock, so a
crashed worker can't deadlock a doc — Qdrant upserts are idempotent).
- Consumer (worker role): `nextcloud-mcp-server worker` drains the queue and runs
the existing process_document pipeline; a periodic task reclaims jobs orphaned
in `doing` by a crash.
INGEST_QUEUE selects the transport (auto: postgres when DATABASE_URL is Postgres,
else the in-process anyio queue for SQLite/dev). procrastinate manages its own
tables (applied on a fresh DB at startup and by `db upgrade`). The vector-sync
status surface reads job counts from Postgres in postgres mode. procrastinate +
psycopg3 ship in the [postgres] extra; the app's own engine still uses asyncpg
(driver unification is a follow-up handled in the rendered Helm chart).
NATS JetStream, the Postgres-queue stub, the bus status subscriber, and nats-py
are removed.
BREAKING CHANGE: the external-NATS-ingest env vars are removed
(INGEST_MODE, STATUS_BACKEND, INGEST_BUS_URL, INGEST_BUS_NUM_REPLICAS,
FACT_EVENT_EMITTER). Use INGEST_QUEUE (memory|postgres) and the `worker`
command instead. TENANT_ID is retained (no longer NATS-subject-charset-validated).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Resolve the latest PR-review comment on the verify-on-read tag-gate work:
- tests: stringify note IDs in test_verify_on_read.py so SearchResult.id
matches production (scanner stringifies all IDs on write) — helper and the
keeps/deleted/mixed/dedupe assertions (blocking).
- tests: make the unshared-file negative control a PDF so the drop is
unambiguously "unshared", not a mime_type_filter mismatch.
- config: add Validator("VECTOR_SYNC_PDF_TAG", len_min=1) — an empty tag name
would make find_files_by_tag("") misbehave in the verifier and scanner.
- verification: correct the _verify_files comment — two batch fetches (tag
REPORT + EXCLUDED_TAGS lookup) are held under one semaphore slot; the
pure-Python intersection runs outside it.
- tests: de-duplicate the minimal-PDF constant into a shared PDF_BYTES in
tests/integration/conftest.py, imported by both integration modules.
Verified: ruff/format/ty/unit all green; the two integration modules
(10 tests) pass against a local Nextcloud (app-only, no MCP profile needed).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Verify-on-read only checked file *accessibility* (file_accessible_by_id),
never tag membership, so a file removed from the `vector-index` tag (but
still readable) kept surfacing in semantic search, and stale points only
got evicted when they happened to rank in a search's top-K.
Rework `_verify_files` to gate on current `vector-index` tag membership via
a single batch `find_files_by_tag(tag, mime_type_filter="application/pdf")`
REPORT per search (plus a one-shot EXCLUDED_TAGS lookup for exclusion-wins
parity) — exactly what the scanner indexes. A file is kept iff it is in that
set, so untagged / deleted / excluded files drop out immediately and the
existing eviction wiring reclaims their Qdrant points. The gate is strict
for all file results, own and shared. Mirrors the batch-fetch-and-intersect
shape of `_verify_news_items` (one semaphore slot, fail-open on fetch error,
malformed-id keep).
- Promote the tag name to a `vector_sync_pdf_tag` Settings field (dynaconf
env mapping VECTOR_SYNC_PDF_TAG) used by both scanner and verifier;
drop the scanner's direct os.getenv.
- Expose `find_files_by_tag` on NextcloudClientProtocol.
- Rewrite the file-verifier unit tests (tagged/untagged/deleted/excluded/
fail-open/non-numeric); update the ACL + verify-on-read integration tests
to seed tagged PDFs.
- Amend ADR-019 and the configuration.md verify-on-read latency budget.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1. Don't log unverified result titles: both search algorithms logged top-5
titles at DEBUG before verify-on-read; with owner-level share expansion the
unverified set can contain other users' docs. Algorithms now log a count
only; the verifying callers (server/semantic, viz_routes, api/visualization)
log verified titles after verify-on-read.
2. Cross-user FILE chunk context: get_chunk_with_context + the Qdrant chunk
helpers now take accessible_owners and use build_ownership_filter. For files
the expanded scope is honoured only after a per-file file_accessible_by_id
check (accessible_owners is owner-level, so the gate prevents a one-file
share recipient from reading any of the owner's cached chunks). note/deck/
news stay self-only (per-user APIs) — a documented gap. Both chunk endpoints
pass accessible_owners.
3. Algorithm usage: SemanticSearchAlgorithm is not dead (it backs the dense-only
option on the viz/API surfaces); added a clarifying comment in server/
semantic.py. Additionally wired accessible_owners + verify-on-read into the
/api/v1 search routes (unified_search, vector_search) so the astrolabe
surface is ACL-aware too — degrading gracefully to self-only/unverified for
non-provisioned callers instead of 401.
4. Overlapping conditions: build_ownership_filter no longer lists self in the
owner_id MatchAny branch (self is already covered by the user_id branch);
the owner_id branch carries only the OTHER owners.
Tests: build_ownership_filter dedup + chunk-bbox filter-shape updates; new
ACL-aware get_indexed_doc_types, cached-chunk lookup, and end-to-end cross-user
file chunk-context (recipient gets the chunk, non-recipient denied) tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- get_indexed_doc_types: add optional accessible_owners param and reuse
build_ownership_filter so cross-user doc-type discovery matches the real
search scope (was self-only / ACL-blind); docstring documents the self-only
default. Covered by test_get_indexed_doc_types_is_acl_aware.
- access_filter: build_ownership_filter now omits the owner_id branch entirely
for an empty owner set instead of relying on undocumented MatchAny(any=[])
semantics; updated the empty-list unit test accordingly.
- access_filter: make the uid_owner/owner share-owner extraction explicit
("absent, not empty") to avoid skipping on a falsy-but-present field.
- access_filter: add an operator note that pre-owner_id points need a re-index
to surface to share recipients (ACL search is a no-op for legacy data).
- verification/webdav: lock the file_accessible_by_id(scope="") contract with a
targeted multi-user test (owner + recipient True, non-recipient False).
- viz_routes: comment that verify-on-read eviction runs inline by design (no
lifespan task group available on the Starlette route).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- viz_routes: run verify_search_results before returning results. After the
accessible_owners expansion the viz can surface OTHER users' shared docs, so
it must drop ones the caller can no longer access (revoked share) — same as
the nc_semantic_search tool path. (Blocking review item.)
- access_filter: cache list_accessible_owners per user for 30s to keep the OCS
shares round-trip off the search hot path (failures aren't cached); document
the single-page OCS limitation; add a clear_accessible_owners_cache() test
helper. Comment the empty-accessible_owners MatchAny([]) edge case.
- verification: comment why cross-user eviction is a deliberate no-op (eviction
is scoped to the querying user's id, so a recipient's revoked access never
deletes the owner's points; the recipient self-heals via accessible_owners).
- algorithms: declare SearchResult.original_score (set by the viz route) so the
now-precisely-typed result list type-checks.
- tests: cross-user eviction-no-op safety test; autouse owners-cache reset in
the access_filter + shared-search tests; replace async-no-await qdrant fakes
with AsyncMock (clears SonarCloud S7503).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The remaining SonarCloud S5443 (publicly-writable directory) findings were the
hard-coded /tmp screenshot paths in revoke_background_sync_access, which became
"new code" once the surrounding function was edited. Replace every /tmp literal
in the file with tempfile.gettempdir() (which S5443 accepts), eliminating the
findings consistently rather than per-line.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
SonarCloud Automatic Analysis does not honour # NOSONAR, so the S6418
(hard-coded token) and S5443 (publicly writable /tmp) findings in the new
tests persisted. Fix them by construction instead:
- test_login_flow: use a trivial poll-token value ("tok") in the rewrite test
(it asserts the URLs, not the token) so it no longer looks like a secret.
- test_astrolabe bg-sync: build the debug screenshot path from
tempfile.gettempdir() rather than a hard-coded /tmp literal.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The new-code quality gate flagged test-only mock fixtures as security issues
(new_security_rating E):
- S2068 "hard-coded password" ×2: drop the unused "app_password" value from the
get_app_password_with_scopes mocks (the code under test only reads truthiness
+ "scopes").
- S6418 "hard-coded token": NOSONAR on the Login Flow v2 poll-token test fixture.
- S5443 "publicly writable directory": NOSONAR on the /tmp debug screenshot path
(matches this file's existing convention).
No behaviour change; all are test fixtures, not real credentials.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The OAuth provisioning tools (check_provisioning_status, revoke_nextcloud_
access) only consulted the refresh-token store + Astrolabe status, ignoring the
app_passwords store that Login Flow v2 (nc_auth_provision_access) and the
management API write to — the same store require_provisioning / get_client use
to grant tool access. Result: status reported "not provisioned" while tools
worked, and revoke said "nothing to revoke" while the credential persisted.
- _get_provisioning_status: also check storage.get_app_password_with_scopes,
reporting is_provisioned with credential_type=app_password,
flow_type=login_flow_v2.
- _revoke_nextcloud_access: when the credential is an app password, delete it
from storage + invalidate the scope cache (no IdP token to revoke);
refresh-token revocation via the Token Broker is unchanged.
- tests/unit/test_oauth_tools_app_password_provisioning.py: cover status +
revoke for the app-password path.
- bump astrolabe submodule (deprovision MCP on disable); fix a stale assertion
in the migrated bg-sync test (one-click flow has no separate app-password
generation step).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Astrolabe was refactored to mint session-derived JWTs (TokenGenerationRequest
Event) and a one-click background-indexing opt-in, dropping the OAuth
authorize/callback/refresh surface. Bump the submodule and bring the test
suite in line:
- New test_astrolabe_session_jwt_search.py: a logged-in user searches via the
minted JWT with no provisioning (replaces the obsolete login_flow_provisioning
OAuth-authorize test; token_refresh test deleted — refresh flow is gone).
- settings_buttons: assert the new revoke endpoint + that oauth/disconnect is
gone (404).
- multi_user_background_sync / plotly / chunk_context: drop the OAuth authorize
step; provision via the one-click "Enable background indexing" button
(#mcp-enable-background-button -> #mcp-revoke-background-button) instead of
generating + pasting an app password.
- docker-compose.yml: mount the astrolabe submodule into the app container.
- third_party/astrolabe: bump to the one-click opt-in commit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The ACL-aware vector filter (PR #813) expands a user's search to documents
whose owner shared them, but verify-on-read still re-checked each file by
PATH under the *searching* user's WebDAV root. Nextcloud mounts received
shares at the recipient's root by basename, so a nested shared file (e.g.
owner's /docs/report.pdf) 404s for the recipient and was silently dropped —
defeating the filter for everything but root-level files.
Verify files by their global Nextcloud file id instead (the file doc_id IS
that id): WebDAVClient.get_file_info_by_id was insufficient (the dav/meta
endpoint only resolves the user's own storage, not shares), so add
WebDAVClient.file_accessible_by_id which runs a WebDAV SEARCH over the user's
whole tree (incl. mounted shares) filtered on oc:fileid. Empirically this
resolves owned, directly-shared, and folder-shared files; an empty result is
a definitive drop, transport errors are kept as transient.
- search/verification.py: _verify_files now checks file_accessible_by_id.
- client/webdav.py: add file_accessible_by_id (SEARCH by fileid).
- tests/integration/test_acl_owner_filter.py: filter matrix vs real Qdrant.
- tests/integration/test_acl_shared_search.py: real-Nextcloud share -> search.
- tests/integration/test_verify_on_read.py: nested shared file kept for the
recipient; unshared file dropped.
- tests/unit/search/test_verification.py: id-based verifier semantics.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Addresses all 8 items in the round-4 bot review plus 4 remaining
SonarQube OPEN issues that were silently broken by round 3's
malformed NOSONAR markers.
NOSONAR syntax fix (clears the remaining 4 OPEN SQ issues)
----------------------------------------------------------
Round 3 used ``# NOSONAR S<rule_key>`` form. SonarQube Python doesn't
recognize the rule-key suffix — it treats the whole thing as a
malformed suppression directive (S7632) AND lets the underlying rule
keep firing (S7503 on ``_Cursor.__aenter__/__aexit__``).
Switch every marker to bare ``# NOSONAR``, with the rationale moved
into a preceding comment block. Affected sites:
- storage.py: ``_Cursor.__aenter__``, ``_Cursor.__aexit__``
- config.py: ``get_database_ssl()`` ``return False`` + ``ssl.create_default_context()``
- test_storage_logging.py: ``SENTINEL_PASSWORD_FRAGMENT`` constant
- test_storage_postgres.py: three ``bob_pw_v1`` / ``bob_pw_v2`` / ``carol_pw`` literals
Bot 🔴#1 — defensive NOSONAR on get_database_ssl `return False`
--------------------------------------------------------------
Bot predicted S4830 fires on the operator-opt-out path. SQ output
shows it doesn't currently fire, but bare NOSONAR added defensively
with rationale comment.
Bot 🔴#2 — defensive NOSONAR on f-string SQL
--------------------------------------------
``update_oauth_session`` builds its SET clause via ``f"{', '.join(update_fields)}"``;
``get_audit_logs`` builds its WHERE clause via string concatenation.
Both are safe (the fragments only come from this function's own
branches, no user input), but the patterns trip taint analysers.
Annotated both with bare NOSONAR + safety comment explaining the
hardcoded-fragments invariant. Note: S2077 doesn't currently fire
on these; defensive.
Bot 🟡#3 — pg_advisory_lock for concurrent migrations
-----------------------------------------------------
Without coordination, two pods rolling-updating simultaneously can
both observe ``has_alembic=False`` and both try to apply migrations
from scratch — the second crashes with "relation already exists".
New ``_migration_lock()`` async context manager:
- On Postgres: ``SELECT pg_advisory_lock(:lock_id)`` on a fresh
connection (separate from the engine pool so it survives the
``to_thread.run_sync`` worker), held across BOTH the schema-inspect
AND the migration call. Without that span, two pods could each
observe "no alembic_version" before either started migrating,
defeating the lock.
- On SQLite: yields immediately (file-level locking serializes
writes natively).
Lock ID derived from
``sha256(b"nextcloud-mcp-server:migrations")[:8]`` as a stable signed
int64 so we can't collide with other apps sharing the same Postgres.
Bot 🟡#4 — RefreshTokenStorage.close() + lifespan wiring
--------------------------------------------------------
New idempotent ``close()`` method calls ``await engine.dispose()``,
nulls the engine, resets ``_initialized``. Wired into both
``app_lifespan_basic`` (BasicAuth) and the OAuth lifespan teardown,
each wrapped in ``try/except Exception`` with ``logger.warning`` so a
buggy dispose can't block SIGTERM. Without this, pooled asyncpg
connections leak server-side slots until
``idle_in_transaction_session_timeout`` reaps them — with small pool
defaults and frequent k8s rolling restarts this can starve
``max_connections``.
Bot 🟢#5 — is_sqlite_url docstring on :memory:
----------------------------------------------
Updated docstring to note both file-backed and in-memory forms are
recognized; caller is responsible for ``:memory:`` magic.
Bot 🟢#6 — db_path via make_url(...).database
---------------------------------------------
Replaced ``database_url.split("///", 1)[1]`` hack with SQLAlchemy's
own URL parsing. Naturally handles in-memory (``.database is None``
→ falls back to ``""``). Same lazy-import pattern as the existing
``mask_db_password`` to avoid module-import-time cost.
Bot 🟢#7 — _to_sync_url unrecognized-driver guard
-------------------------------------------------
Pulled ``_KNOWN_ASYNC_DRIVERS = ("aiosqlite", "asyncpg")`` into a
module constant. When an unrecognized ``+<driver>`` token survives
the strip, emits ``logger.warning`` with the known-supported list.
Behavior unchanged for valid URLs.
Bot 🟢#8 — get_audit_logs SELECT * → explicit columns
-----------------------------------------------------
Replaced ``SELECT *`` with explicit column list. Future schema
additions stay out of the dict return.
New tests
---------
- ``test_close_disposes_engine``: pins the public contract — engine
nulled, state reset, second call is a no-op.
- ``test_concurrent_initialize_serialized_by_advisory_lock``: spawns
3 concurrent inits against a fresh schema; asserts no "relation
already exists" and exactly one ``alembic_version`` row at the end.
Without the lock, this reliably fails on the second concurrent
task.
Docs
----
- ADR-026: new "Concurrent migrations across pods" subsection
documents the advisory-lock approach + lock-ID derivation.
Verification
------------
- ``uv run pytest tests/unit/`` — 1025 passed.
- ``TEST_DATABASE_URL=… uv run pytest tests/integration/test_storage_postgres.py -m postgres`` — 9 passed (was 7).
- ``ruff check && ruff format --check && ty check`` — clean.
Expected post-push: SQ scan reports 0 OPEN issues (was 4).
Tracked on Astrolabe Cloud POC board, card #99.
---
_This PR was generated with the help of AI, and reviewed by a Human_
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Round-3 fixes. Two threads:
- 9 OPEN SonarQube issues caused the "E Security Rating on New Code"
gate failure. The bot's diagnosis (sa.text(text_sql) → SQL injection)
was a wrong guess; the actual SQ rules firing were different.
- Bot's substantive concerns: pool defaults too aggressive,
delete_browser_session RETURNING path untested on Postgres,
schema_version legacy table created on Postgres, stale module
docstring.
- User's underlying question on the pool: "isn't 1 connection enough?"
Right-sized to 2+5 and documented the concurrency model in ADR-026
so the rationale is durable.
SonarQube quality-gate fixes (clears all 9 OPEN issues)
-------------------------------------------------------
- BLOCKER S6418: rename `SECRET` constant in test_storage_logging.py
to `SENTINEL_PASSWORD_FRAGMENT` + NOSONAR with rationale.
- CRITICAL S3776: extract `_build_postgres_engine()` from
`initialize()` (was complexity 26 > 15); incidentally creates a
clean unit-test seam for engine args.
- CRITICAL S4423: `ssl.create_default_context(cafile=...)` is flagged
as "weak protocol" — Python 3.10+ already negotiates the strongest
available protocol. Explicitly pass `purpose=ssl.Purpose.SERVER_AUTH`
and NOSONAR with the Python-version rationale.
- MAJOR S3358: split the TLS-mode nested ternary in the engine
factory into a `_describe_ssl_arg()` helper.
- MAJOR S2068 ×3: bind test app-password literals to local vars and
put `# NOSONAR S2068` on the same line as the literal (anchoring
requirement) instead of on the closing paren.
- MINOR S7503 ×2: `# NOSONAR S7503` on `_Cursor.__aenter__/__aexit__`
— they MUST be `async` per the context-manager protocol.
Pool sizing right-sized (answers "why so many connections?")
------------------------------------------------------------
- `DATABASE_POOL_SIZE` default 10 → **2**.
- `DATABASE_MAX_OVERFLOW` default 20 → **5**.
- Per-pod max drops from 30 to 7. With 3 replicas, total = 21
connections (was 90) — well under managed-Postgres
`max_connections=100`.
- New INFO log at startup: `Postgres engine ready: pool_size=N
max_overflow=M (per-pod max K connections)`. Surfaces the active
sizing without grepping config.
- New ADR-026 § "Concurrency model and pool sizing" explains
asyncpg's single-flight connection semantics, the MCP workload
shape (read-mostly point lookups), why-not-1 (multi-user
serialization), and the tune-up/tune-down recipe.
- `docs/configuration.md` table updated with new defaults +
homelab-vs-prod tuning guidance, linking the ADR.
RETURNING path covered on Postgres
----------------------------------
- New `test_browser_session_delete_returning` exercises the
`DELETE … RETURNING user_id` path — the only RETURNING clause in
the storage layer and the most dialect-sensitive SQL in this PR.
Asserts both present-row (returns True, row gone) and absent-row
(returns False) branches.
Schema portability polish
-------------------------
- `alembic 001`: gate `schema_version` table creation on
`op.get_bind().dialect.name == "sqlite"`. The table exists purely
to match the fingerprint of pre-Alembic SQLite databases; fresh
Postgres installs no longer carry the dead legacy table.
Misc polish
-----------
- Module docstring: "SQLite-based" → "SQL-backed", with a sentence
on the DATABASE_URL opt-in and an ADR-026 link.
- Comment on `_wrap_row` noting `row._mapping` is the documented
RowMapping accessor in SQLAlchemy 2.x despite the underscore.
Skipped (rationale in PR reply)
-------------------------------
- `_qmark_to_named` SQL-comment handling: docstring already notes
the limitation; no `?` in storage SQL comments today.
- Module-level `anyio.Lock()`: established precedent confirmed by
the bot itself.
- `get_audit_logs` `SELECT *`: pre-existing pattern, out of scope.
Verification
------------
- `uv run pytest tests/unit/` — 1025 passed.
- `TEST_DATABASE_URL=… uv run pytest tests/integration/test_storage_postgres.py -m postgres` — 7 passed.
- `ruff check && ruff format --check && ty check` — clean.
- Confirmed `schema_version` absent on fresh Postgres, still present
on fresh SQLite.
Tracked on Astrolabe Cloud POC board, card #99.
---
_This PR was generated with the help of AI, and reviewed by a Human_
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Round-2 fixes after the bot review on PR #798 plus two user follow-ups
(self-signed Postgres support; asyncpg should be a PyPI extra). Folded
into the same PR rather than a follow-up since the work is still
unmerged.
Security
--------
- Mask database credentials in all 5 log call sites (storage.py × 4,
migrations.py × 1) via a new `mask_db_password()` helper in config.py.
Uses SQLAlchemy's `make_url(...).render_as_string(hide_password=True)`
with a regex fallback so the masking path never raises.
- New `tests/unit/test_storage_logging.py` asserts a sentinel password
never appears in `caplog` during `RefreshTokenStorage.initialize()`.
Distribution
------------
- `asyncpg` moved to `[project.optional-dependencies] postgres` so a
vanilla `pip install nextcloud-mcp-server` no longer pulls in the
~5 MB C extension. The Docker image runs `uv sync --extra postgres`,
so containerized deployments are unchanged.
- When `DATABASE_URL=postgresql+asyncpg://...` is set on a venv missing
the extra, `RefreshTokenStorage.initialize()` raises a friendly
RuntimeError pointing at `[postgres]` rather than the generic
ModuleNotFoundError.
TLS for the Postgres backend
----------------------------
- New `DATABASE_VERIFY_SSL` + `DATABASE_CA_BUNDLE` env vars mirror the
existing `NEXTCLOUD_VERIFY_SSL` / `NEXTCLOUD_CA_BUNDLE` pattern
(validators in Settings.__post_init__, `get_database_ssl()` helper
alongside `get_nextcloud_ssl_verify()`). `DATABASE_VERIFY_SSL=false`
wins over `DATABASE_CA_BUNDLE` for incident-response convenience.
- Default is **None** rather than True — keeps PR #798's behavior
intact for cluster-internal Postgres that runs without TLS. Operators
opt into verify-full or supply a private CA. ADR-026 records the
reasoning vs the Nextcloud HTTPS default.
- Engine factory in `storage.py` passes `ssl` via `connect_args` only
when `get_database_ssl()` returns non-None; otherwise asyncpg's
default (`prefer`) applies.
- Storage logs which TLS mode is active at INFO (no secret material).
Configurable connection pool
----------------------------
- `DATABASE_POOL_SIZE` (default 10) and `DATABASE_MAX_OVERFLOW`
(default 20) replace the hardcoded engine values. With many replicas
this can blow past managed-Postgres `max_connections=100`; tune down
for large fleets.
- gte-1 / gte-0 validators in __post_init__ reject 0/negative pool
sizes at startup with the offending value in the error.
Consistency polish
------------------
- Migration 006: convert raw `op.execute("ALTER TABLE ... ADD COLUMN")`
to `op.batch_alter_table(...).add_column(sa.Column("nonce", sa.Text))`
for stylistic consistency with the rewritten 001-005. Downgrade now
drops the column instead of being a no-op.
- `registered_webhooks.created_at` standardized from `sa.Float` to
`sa.BigInteger` (all other `*_at` columns); `store_webhook()` casts
`time.time()` → `int`.
- `is_sqlite_url()` made case-insensitive.
Testing
-------
- New `tests/integration/test_storage_postgres.py::test_cleanup_expired_roundtrip`
exercises `cleanup_expired_tokens`, `cleanup_expired_sessions`, and
`cleanup_expired_browser_sessions` — relies on DELETE rowcount,
historically dialect-tricky.
- `tests/unit/test_ssl_config.py` extended with `TestDatabaseSSLSettings`
+ `TestGetDatabaseSSL` classes (9 new tests) mirroring the existing
Nextcloud SSL tests one-for-one.
Docs
----
- `docs/configuration.md` Centralized-Storage section grew the four new
env vars + a homelab example with a private CA.
- `docs/ADR-026` grew Distribution, TLS, and `alembic/env.py` async-pattern
subsections explaining the non-obvious design choices.
Helm chart counterpart in cbcoutinho/helm-charts PR #34 (separate
commit on `feat/nextcloud-mcp-server-database-url`).
Verification
------------
- `uv run pytest tests/unit/` — 1025 passed.
- `TEST_DATABASE_URL=... uv run pytest tests/integration/test_storage_postgres.py -m postgres` — 6 passed (including new cleanup test).
- `uv run ruff check && uv run ruff format --check && uv run ty check -- nextcloud_mcp_server` — clean.
Tracked on Astrolabe Cloud POC board, card #99.
---
_This PR was generated with the help of AI, and reviewed by a Human_
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a `DATABASE_URL` setting that lets `RefreshTokenStorage` run against
any SQLAlchemy async backend, primarily `postgresql+asyncpg://...` for
HA k8s deployments. Default behavior is unchanged: when `DATABASE_URL` is
unset the server falls back to the existing `TOKEN_STORAGE_DB` path /
ephemeral SQLite tempfile.
Why
---
Today every MCP pod needs its own PVC to hold the SQLite file, which
pins the Deployment to one replica and blocks horizontal scaling. With
this change, operators can point all replicas at a shared Postgres
(CNPG, RDS, etc.) and the pods become stateless. Encryption stays in
Python (Fernet); the database only sees ciphertext.
What changed
------------
- `config.get_database_url()` resolves DATABASE_URL → TOKEN_STORAGE_DB →
ephemeral tempfile in that priority order.
- `RefreshTokenStorage` builds a process-shared `AsyncEngine` in
`initialize()`. SQLite gets NullPool; Postgres gets pool_size=10,
max_overflow=20, pool_pre_ping=True. 30 aiosqlite call sites adapted
via a thin `_DBConn` / `_Cursor` / `_Row` / `_ExecuteCtx` shim so
existing method bodies need no churn beyond the connection
context-manager swap.
- 7 `INSERT OR REPLACE` statements rewritten as portable
`INSERT ... ON CONFLICT (...) DO UPDATE` (SQLite ≥ 3.24, Postgres ≥ 9.5).
- `sqlite_master` legacy-detection lookup replaced with SQLAlchemy
inspector so the path works against either backend.
- File-permission hardening + parent-dir creation gated on
`is_sqlite_url(...)` — centralized backends manage their own filesystem.
- Alembic migrations 001/002/003/005 converted from raw `op.execute(SQL)`
to portable `op.create_table()` / `op.create_index()` with SQLAlchemy
types. All timestamp columns are `sa.BigInteger` so Postgres allocates
BIGINT (unix epochs don't fit in INT4). SQLite treats BIGINT as
INTEGER, so existing deployments at revision 006 see no schema drift.
- `migrations.py` + CLI take URLs; `db {upgrade,downgrade,current,history}`
gain `--database-url / -u` alongside the legacy `--database-path / -d`.
`get_current_revision()` uses SQLAlchemy inspector instead of raw
sqlite3, so the CLI works against Postgres too.
- `docker-compose.yml` adds a `postgres-test` service under the
`postgres` profile (pinned `postgres:16-alpine` digest) for
integration testing.
- Unit storage tests parametrized over backends via shared
`tests/fixtures/storage_backend.py` — every test in
`test_app_password_storage.py` and `test_webhook_storage.py` runs
once per backend that is available. Postgres is opted in by
`TEST_DATABASE_URL`.
- New `tests/integration/test_storage_postgres.py` (5 tests, marked
`postgres` + `integration`) covers refresh-token, app-password,
OAuth-session, webhook, and audit-log paths end-to-end on Postgres.
- New `docs/ADR-026-pluggable-database-backend.md` records the decision;
`docs/configuration.md` documents `DATABASE_URL` with examples.
Out of scope
------------
- No SQLite → Postgres data migration tool (clean cutover; tokens reissue
on next login, webhooks re-register on next sync tick).
- This repo does not provision Postgres. The matching helm chart change
lives in cbcoutinho/helm-charts (database.url / existingSecret values).
Verification
------------
- `uv run pytest tests/unit/` — 1012 passed, SQLite path unchanged.
- `docker compose --profile postgres up -d postgres-test`
- `TEST_DATABASE_URL=... uv run pytest tests/integration/test_storage_postgres.py -m postgres -v`
— 5 passed.
- `TEST_DATABASE_URL=... uv run pytest tests/unit/test_app_password_storage.py
tests/unit/test_webhook_storage.py` — 50 passed (25 per backend).
- `uv run ruff check && uv run ruff format --check && uv run ty check -- nextcloud_mcp_server` — clean.
Tracked on Astrolabe Cloud POC board, card #99.
---
_This PR was generated with the help of AI, and reviewed by a Human_
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Follow-up to #787/#789 (ADR-022 cleanup). After
`oauth_enabled ↔ enable_login_flow` became an invariant, the
`use_basic_auth=False` branch in `vector/oauth_sync.py` — and the
parameter wiring that fed it — was no longer reachable from any
supported deployment mode. This commit removes the dead code.
- nextcloud_mcp_server/vector/oauth_sync.py:
- Deleted `get_user_client_oauth` (the OAuth-token refresh helper) and
its `VECTOR_SYNC_SCOPES` constant.
- Deleted the `get_user_client` dispatcher. Internal callers now call
`get_user_client_basic_auth` directly.
- Dropped the `use_basic_auth: bool` parameter from `user_scanner_task`,
`multi_user_processor_task`, `_run_user_scanner_with_scope`, and
`user_manager_task`.
- Dropped the `token_broker` parameter from the same four functions —
they no longer need it now that the OAuth-refresh path is gone. The
`TokenBrokerService` constructed in `app.py` is still used by the
management API revoke endpoint, just not by background sync.
- Simplified the user-list query in `user_manager_task` to always read
from the `app_passwords` table.
- Replaced all `mode_label = "BasicAuth" if use_basic_auth else "OAuth"`
with a literal `[BasicAuth]` log prefix (keeps existing log filters
working).
- Updated the module docstring to describe the post-cleanup shape.
- Dropped the now-unused `TYPE_CHECKING` import of `TokenBrokerService`.
- nextcloud_mcp_server/app.py: dropped the `use_basic_auth = True` block
and the now-stale `token_broker if not use_basic_auth else None` /
`use_basic_auth` positional args from the two `tg.start(...)` calls in
the multi-user vector-sync lifespan. Token broker construction stays —
still consumed by the management API revoke endpoint via
`app.state.oauth_context["token_broker"]`.
- tests/integration/test_app_password_provisioning.py: deleted four tests
that exercised the now-removed OAuth-refresh path
(`test_oauth_mode_uses_refresh_token_only`,
`test_oauth_mode_raises_error_without_token`,
`test_get_user_client_oauth_function`,
`test_oauth_mode_requires_token_broker`) plus the
`test_get_user_client_dispatches_to_basic_auth` test for the deleted
dispatcher. Updated the module docstring + imports accordingly. The
BasicAuth-mode tests (`test_basic_auth_mode_uses_local_storage`,
`test_multiple_users_basic_auth_mode`, etc.) all remain.
No runtime-behaviour change in any supported deployment mode — the deleted
branches were already unreachable post-PR #787. 3 files changed,
+59 / -301; 1010 unit tests pass; integration jobs for
`mcp-login-flow` and `mcp-multi-user-basic` are the critical regression
gates before merge.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two unrelated CI failures on this branch, one fix each:
- tests/integration/test_deck_vector_search.py: pass str(card.id) to
get_chunk_with_context. The function's contract is doc_id: str
(keyword-indexed in Qdrant), and real callers (viz_routes.py URL
path, server/semantic.py via str(result.id)) all stringify. The
test was the only int caller, hitting the .isdigit() guard added
earlier on this branch.
- tests/server/login_flow/test_login_flow_integration.py:
test_check_status_provisioned now accepts scopes=None as valid.
Per ProvisionStatusResponse in models/auth.py, None is the
documented sentinel for "all scopes granted" — and the web
provisioning path (provision_routes.py, used by Astrolabe's
"Enable Semantic Search" flow exercised by the new regression test
added on this branch) stores exactly that. The previous
is-not-None assertion hid behind test order until that flow ran.
- Replace anyio.sleep(0) with anyio.lowlevel.checkpoint()
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add the missing end-to-end coverage for the seam that PR #773's
`ALLOWED_MGMT_CLIENT` ↔ `astrolabeMcpClientOAuth00000000000` drift
bug exposed: browser → Astrolabe (NC PHP app) → MCP server's
management API on the `mcp-login-flow` profile. Every existing
`tests/integration/test_astrolabe_*.py` is marked `multi_user_basic`
and exercises the BasicAuth flow, not Login Flow v2 / OAuth.
The new test mirrors the production-shaped flow exactly:
1. Log in as admin via Playwright.
2. Navigate to `/settings/user/astrolabe`.
3. Click the "Enable Semantic Search" OAuth link rendered by
`oauth-required.php`. (Same selector Astrolabe's own e2e helper
uses — `third_party/astrolabe/tests/e2e/helpers/authorize.ts`.)
4. Click "Allow" on the Nextcloud OIDC consent screen.
5. Wait for the redirect back to the Astrolabe settings page.
6. Assert the "Enable Semantic Search" link is no longer visible.
Step 6 is the canary for the drift class: if Astrolabe's management
API call to `/api/v1/users/{id}/session` is rejected (HTTP 401, the
original bug), the session lookup falls back to "no token" and the
same `oauth-required.php` template re-renders with the link still
present — so the test fails loudly with a message naming the
likely cause.
Reuses `login_to_nextcloud` and `navigate_to_astrolabe_settings`
helpers from `tests/integration/test_astrolabe_multi_user_background_sync.py`
(already pattern-imported by the Plotly viz test). No fixture-level
OIDC client creation: `app-hooks/before-starting/26-configure-astrolabe-oauth.sh`
already provisions `astrolabeMcpClientOAuth00000000000` with the
correct redirect URI and scopes when `MCP_SERVER_URL` is set in the
shell that runs `docker compose --profile login-flow up`.
The test skips cleanly when admin is already authorized (typical
state on a re-run against a long-lived dev stack), so it's safe to
run repeatedly without manual reset.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
NextcloudClient.find_files_by_tag now mirrors the directory semantics
already used by the exclusion path (issue #710): when a tagged item is
a folder, walk its descendants via WebDAV SEARCH (Depth: infinity) and
include any files matching the MIME filter. Without this, tagging the
root of a corpus with `vector-index` indexed nothing because the tag
applies to the directory only, not to its children.
The vector scanner additionally consults EXCLUDED_TAGS now, so a folder
marked off-limits is skipped even if it (or an ancestor) carries the
include tag — defense-in-depth, matching the "exclusion wins" contract
already enforced by the MCP file tools.
Also addressed a recurring memory-style nit: pre-existing f-string log
lines in find_files_by_tag were converted to lazy %-style.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Addresses two points from the latest PR #764 review:
1. The anyio.Lock in get_excluded_file_paths bought nothing under
anyio's cooperative multitasking model (single-threaded between
awaits, raw set mutations are already safe). _resolve_one_tag now
builds a local set of paths and appends it to a shared list — list
append between awaits is safe without a lock — and the caller
merges via set().union(*results) after the task group completes.
This removes the cognitive overhead the reviewer flagged without
changing the public API.
2. Adds tests/integration/test_tag_exclusion.py exercising the
resolution pipeline end-to-end against a real Nextcloud instance:
creates a system tag, tags a real file and a real directory,
verifies get_excluded_file_paths resolves both via real PROPFIND +
REPORT calls, and verifies is_path_excluded correctly classifies
exact matches, descendants of tagged directories, and unrelated
paths. Includes the disabled-feature short-circuit case.
Cleanup runs in reverse order (untag, delete files); per-run uuid
suffix avoids cross-run interference.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Round 7 raised 5 issues; this round addresses all of them and fixes
the underlying causes (not just the comments) where applicable so
they don't get re-flagged in future passes.
Critical:
- verified_count description in SemanticSearchResponse said "unique
documents" but the value is len(verified_results), a chunk count.
Description rewritten to accurately document chunk-level granularity
AND explicitly call out the asymmetry with dropped_count (which
counts unique (doc_id, doc_type) pairs).
- _verify_files false-eviction risk: the round-6 doc-only fix was
re-flagged. Address at the source — widen WebDAVClient.get_file_info
to raise HTTPStatusError on 404 (matching the rest of the client
convention) and reserve None for the genuinely ambiguous
malformed-PROPFIND case. _verify_files now keeps the result on None
(cannot tell whether the file exists) and evicts only on a
definitive HTTPStatusError 404. Tests updated; new test added for
the malformed-XML keep-result path.
Non-critical:
- News verifier semaphore lifetime now explicitly documented: one
slot held for one deduplicated fetch per search is the correct
backpressure behaviour.
- Cross-reference comments in _verify_notes / _verify_deck_cards no
longer claim "Mirrors X" pointing at functions defined later in
the file; now use direction-neutral "parallel implementation in".
- accessible_by_type is mutated by concurrent run_verifier tasks; a
comment explains why this is race-free under anyio's cooperative
multitasking (distinct keys per task, no await between read and
write) so a future reader doesn't add a redundant lock.
- Knock-on: tests/integration/test_rag.py wraps get_file_info in a
try/except for the new contract.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes out the remaining nits flagged in the round-6 review.
Critical:
- _verify_files contract comment now enumerates all None-return cases
(404 + malformed PROPFIND XML) and documents the false-eviction
trade-off; self-healing via re-indexing recovers
- int(r.id) cast at the SemanticSearchResult boundary now raises a
TypeError with explicit doc_type/value context instead of bubbling
up as an opaque "Search failed: ..." McpError
Design observations:
- nc_semantic_search_answer docstring documents the per-note
round-trip cost from the post-verification race guard
- News verification latency hint added to configuration.md
- SemanticSearchResponse exposes verified_count + dropped_count so
short result pages on high-ghost-density indexes are
distinguishable from genuine scarcity. verify_search_results now
returns (kept, dropped_count); production caller and tests updated
Minor:
- Comment clarifies the .get() fallback in verify_search_results is
defensive only (run_verifier always populates the entry)
- Eviction task-group guard narrowed from except Exception to
except RuntimeError (the only documented failure mode of
TaskGroup.start_soon on a closed group)
- Indexer logs a warning when a deck_card task is missing
board_id/stack_id, surfacing data-quality issues at index time
rather than at verification time
- New unit test covers the news verifier's non-numeric-id fail-open
path (one bad doc_id keeps the entire batch)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- _verify_deck_cards: hoist int(board_id|stack_id|doc_id) out of the generic
except Exception into an explicit try/except (TypeError, ValueError) before
the network call, mirroring _verify_news_items. Malformed payloads now log
a specific warning instead of "unexpected error".
- _verify_news_items: add TODO(perf) above the get_items(batch_size=-1) call
to mark the known fetch-all cost as a future profiling target.
- SemanticSearchResult.id: revert from int|str back to int. The internal
SearchResult.id stays int|str for forward-compat; the MCP response model
narrows at the boundary. server/semantic.py casts r.id to int when
constructing the response so future string-id types fail loudly here
instead of silently widening the public API.
- nc_semantic_search: replace the terse "extra for access filtering" comment
with an ADR-019 NOTE block explaining the 2x over-fetch trade-off and the
ghost-density under-delivery case (self-heals via lazy eviction).
- tests/integration/test_verify_on_read.py: extend the module docstring to
call out that only the note verifier is exercised against real Nextcloud,
while file/deck_card/news_item are unit-only — documenting the suite split
for future contributors.
- ADR-019: rewrite "Module shape", "Verifier registry", example verifier,
and "Deduplication" sections to match the shipped BatchVerifier interface
(was per-id Verifier in the original draft). Add a "Why batch?" paragraph
explaining the design choice. Update implementation checklist — every
item is now [x] with corrected verifier names (plural) and the eviction
module path (vector/eviction.py).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The vector index lags Nextcloud (5-min webhook cron + scanner interval),
producing ghost records for deleted/unshared documents until the next
reconciliation. Verify each unique document against Nextcloud at query
time, drop inaccessible results, and lazily evict the corresponding
Qdrant points.
Per-doc_type batch verifiers: notes/files/deck cards run concurrently
per id; news items use a single fetch + intersect to avoid the per-item
fetch-all amplification. Transient errors fail open (keep result, log
warning) — only definitive 4xx drops. Multiple chunks of the same doc
collapse to one verification call.
Wired into nc_semantic_search before the limit trim and before context
expansion. nc_semantic_search_answer's per-note re-fetch retained as a
sub-second race guard since verification now happens upstream.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous run on nc32 failed at the search-result assertion because
`wait_for_vector_sync` returned on the first indexed-count bump (deck
seed cards) before this specific note hit Qdrant. Replace the single
search call with a poll that retries every 2s until the unique term
returns our note, or times out after 60s with a loud diagnostic. The
previously-observed flake would now wait past the deck-card indexing
window rather than racing it.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per review:
- Hoist `import httpx` out of the two test function bodies and into
the module imports at the top of
test_astrolabe_chunk_context.py.
- Simplify the regression guard in
test_management_chunk_context_endpoint.py to use
`mock.assert_awaited_once_with(...)` instead of manually unpacking
call_args. This is stricter — it fails loudly on signature change —
and matches the canonical pattern for asserting mock calls.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Rename test_chunk_context_endpoint_handles_missing_app_password to
test_chunk_context_endpoint_rejects_invalid_bearer so it reflects
what is actually exercised: an invalid bearer is rejected upfront at
validate_token_and_get_user, not at the NotProvisionedError branch.
The NotProvisionedError path is covered by the corresponding unit
test in test_management_chunk_context_endpoint.py.
- Hoist `import base64` to module level per PEP 8.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Astrolabe's ApiController endpoints (search, chunk-context) require a
CSRF `requesttoken` header — axios picks it up from OC.requestToken
automatically in the SPA, but page.request.get() does not.
The first CI run failed on the search step with 412 CSRF check failed
before reaching the chunk-context assertion that was supposed to
surface the handler bug. Load the Astrolabe page, read OC.requestToken,
and pass it on both calls.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The /api/v1/chunk-context and /api/v1/pdf-preview handlers in
api/visualization.py forwarded the incoming OAuth bearer directly to
Nextcloud via NextcloudClient.from_token. In multi-user BasicAuth mode
Nextcloud has no validator for those bearers on Notes/WebDAV, so it
treats the request as anonymous and returns 401 — surfaced to the user
as a 500 from /apps/astrolabe/api/chunk-context. Search worked because
it only hits Qdrant.
Architecturally, OAuth is only for Astrolabe→MCP server; MCP server→
Nextcloud always uses the per-user app password stored during provision
(background sync already does this via vector.oauth_sync).
- Resolve the Nextcloud client through get_user_client_basic_auth in
both get_chunk_context and get_pdf_preview, surfacing
NotProvisionedError as a clean 401 instead of opaque 500.
- Apply the same fix to the session-cookie variant in
auth/viz_routes.chunk_context_endpoint for the internal viz UI.
Tests:
- New unit file test_management_chunk_context_endpoint.py, including a
regression guard that asserts get_user_client_basic_auth is awaited
(so reverting to from_token fails without needing a live Nextcloud).
- Updated test_management_pdf_preview_endpoint.py to mock the new auth
path (drops extract_bearer_token / NextcloudClient.from_token patches).
- New integration test test_astrolabe_chunk_context.py drives the full
chain (browser → Astrolabe → MCP → Nextcloud) in multi-user BasicAuth
mode, plus bare-bones 401 checks on the MCP endpoint.
Full unit suite: 546 passed.
Companion PR on astrolabe (cbcoutinho/astrolabe#66) sends the Nextcloud
UID as loginName in the app-password POST body so the stored record is
complete. Submodule bump to that branch will follow once CI reproduces
the failure on the old submodule.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The multi-user-basic integration job was consistently failing with
`CancelledError: Cancelled via cancel scope ... by <async_generator_athrow>`
followed by a cascade of `anyio.ClosedResourceError` in every subsequent
test. Root cause: `create_mcp_client_session` was declared as an async
generator driven by `async for session in ...:`, so Python's generator
finalizer (`aclose`) ran under pytest-asyncio's cleanup task instead of
the task that owned the nested `streamablehttp_client` cancel scope.
anyio then raised when the inner task group saw its scope being exited
from a foreign task, leaving the memory object streams half-closed and
poisoning the rest of the session.
Switching to `@asynccontextmanager` + `async with ... as session:` makes
`__aenter__`/`__aexit__` run in the frame that owns the context manager,
satisfying anyio's structured concurrency requirements.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Remove all RFC 8693 token exchange tests (integration, manual, keycloak)
since Nextcloud doesn't support bearer tokens without upstream patches
- Remove manual impersonation/ADR-004 scripts and their docs
- Clean up token_exchange singleton from integration conftest
- Improve logging in _complete_login_flow_v2_as_user with step-by-step
[username] prefixed messages matching _complete_login_flow_v2 style
- Remove unnecessary time staggering from all_login_flow_user_tokens;
concurrent token acquisition works without artificial delays
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Extract reusable wait_for_vector_sync() helper with 90s max_wait (up from
30s) to handle slow single-worker processing in CI. Increase processor
workers to 2 for the mcp service to parallelize note indexing.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Consolidate three independent RefreshTokenStorage lazy singletons into a
single lock-protected get_shared_storage() function, eliminating race
conditions on concurrent first-access. Remove blanket try/except in
_get_stored_scopes so storage errors propagate as proper MCP errors
instead of silently triggering "please provision" messages. Handle
declined/cancelled elicitation results in Login Flow tools by cleaning up
sessions and returning clear status. Add update_app_password_scopes() to
avoid unnecessary decrypt/re-encrypt when only scopes change. Add
unprovisioned-user early exit and no-op detection to nc_auth_update_scopes.
Remove four dead config fields and misleading NEXTCLOUD_PASSWORD deprecation
warning. Add periodic login flow session cleanup task. Generate separate
Fernet keys per service. Add board cleanup in deck integration test. Gate
CI unit tests on linting and skip Astrolabe build for single-user profile.
Fix test markers from oauth to multi_user_basic for astrolabe integration
tests. Update login_flow.py docstrings to document outbound HTTP calls.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
PHP setup was gated behind needs-playwright but Astrolabe build needs
composer unconditionally. Add multi-user-basic CI matrix entry with
proper marker filtering. Upload Playwright screenshots and service logs
as artifacts on failure for easier debugging.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Move httpx import to top-level and use anyio task group for concurrent
validation in cleanup_invalid_app_passwords (storage.py)
- Respect Retry-After header for 429 responses, capped at 300s (oauth_sync.py)
- Soften pre-validation exceptions so transient failures don't crash the
background sync task (oauth_sync.py)
- Replace f-string SQL with blanket DELETE and add returncode checks (conftest.py)
- Extract clear_stale_test_state() helper to deduplicate cleanup logic
in astrolabe background sync tests
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The revoke test failed because it only completed Step 2 (app password) but
not Step 1 (OAuth authorization). In hybrid mode, Astrolabe requires both
steps for $isFullyConfigured=true, which gates the "Revoke Access" button.
Changes:
- Use complete_astrolabe_authorization() in revoke test for full two-step flow
- Add stale state cleanup (app passwords, bruteforce entries, Astrolabe prefs)
to both enablement and revoke tests
- Add startup cleanup of invalid app passwords in BasicAuth mode
- Pre-validate credentials before entering scanner loop to fail fast
- Handle 401/403/429 in scanner with proper backoff and circuit breaking
- Clean up app passwords in test_users_setup fixture teardown
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add cross-system interface test annotations to the 5 astrolabe test files,
clarifying they test the MCP server's integration with the Astrolabe
Nextcloud app (installed from the app store, source now in a separate repo).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Replace Close button click with Escape key in app password dialog
(h2 element was intercepting pointer events)
- Make test_users_setup fixture idempotent by checking user existence
before creation and only tracking created users for cleanup
- Fix search results detection by removing wait for .app-content-wrapper
CSS class that doesn't exist in Astrolabe's Vue app
- Add progress logging during results polling
- Increase polling timeout to 30 seconds for search results
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The reorder_card method was using the API route
/api/v1.0/boards/{boardId}/stacks/{stackId}/cards/{cardId}/reorder
which has a parameter conflict: the URL's {stackId} (current stack)
overrides the body's stackId (target stack) in Nextcloud's routing.
This caused cards to stay in their original stack even when the API
reported success.
Switched to the non-API route /cards/{cardId}/reorder which correctly
reads stackId from the request body, matching the behavior of the
working curl command reported in the issue.
Also added the required OCS-APIRequest headers that were missing.
Fixes#469
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
