Extend the documents-vs-chunks split to the remaining status surfaces so all
three report consistently (Deck #195):
- nc_get_vector_sync_status MCP tool + VectorSyncStatusResponse: add
indexed_documents (distinct) and indexed_chunks; keep indexed_count as a
deprecated alias of indexed_chunks. Reuses count_indexed.
- userinfo HTML page (/app/vector-sync/status): show Indexed Documents AND
Indexed Chunks rows; switch its count to count_indexed (which also excludes
placeholder points — the old raw count included them).
- /api/v1/vector-sync/status: restore indexed_count as a deprecated alias of
indexed_chunks so existing consumers (integration tests, pre-#115 UI) keep
working; the change is now purely additive for indexed_count.
Tests: VectorSyncStatusResponse documents/chunks/alias + zeroed defaults.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Note at both metrics-task call sites that receive_stream is None in postgres
mode (get_ingest_pending falls back to procrastinate counts).
- Add test_default_is_exact_true covering the status-endpoint count path.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Stop mounting the vendored astrolabe submodule into the Nextcloud `app`
container by default (comment out the /opt/apps/astrolabe bind mount). With
the mount absent, the post-installation hook (20-install-astrolabe-app.sh)
falls through to `occ app:install astrolabe` + `app:enable`, so the dev/CI
stack now exercises the published app-store package rather than a locally
built dev copy. This catches packaging issues (e.g. missing built assets in
the released app) that a source build would mask.
Bump the third_party/astrolabe submodule to v0.16.6, which includes the
background-indexing re-login fix (astrolabe#93).
The dev mount and the CI "Build Astrolabe app" step are retained (commented
mount can be re-enabled locally) so developers can still iterate against the
vendored source on demand.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PR #89 (session-derived JWT auth) merged to astrolabe main and released as
v0.16.1; track the released tag instead of the feature-branch commit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fixes the multi-user-basic regression introduced by the previous bump: the
astrolabe app's hard https-refusal blocked sending the app password to the
in-cluster http MCP endpoint (http://mcp-multi-user-basic:8000), so background
indexing never ran. Now warns instead of blocking; also clears the SonarCloud
S5332 hotspot on the dropped test's http literal.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- vector/qdrant_client.py: add owner_id to _PAYLOAD_INDEX_FIELDS (BLOCKING).
Every search applies MatchAny(key="owner_id", ...); without a keyword index
Qdrant full-scans the collection and may 400 on Qdrant Cloud strict mode.
_ensure_payload_indexes is idempotent so existing collections migrate at
startup.
- search/access_filter.py: bound the process-global _owners_cache with an LRU
cap (was one unbounded entry per active user, never evicted); document the
owner-level over-fetch limitation (a prolific sharer floods the recall
buffer with ghost candidates that verify-on-read drops, with no second
Qdrant pass) as a TODO toward per-file filtering.
- search/algorithms.py + semantic.py + bm25_hybrid.py: promote
accessible_owners from **kwargs to an explicit keyword-only parameter on the
SearchAlgorithm ABC and both implementations, so a misspelled keyword is a
type error rather than a silent fall back to self-only scope.
- search/verification.py: document that _verify_files now verifies by global
file id (WebDAV SEARCH), not by path.
- tests/unit/search/test_access_filter.py: add cache-hit, TTL-expiry,
failure-not-cached, and LRU-bound tests.
Bumps the astrolabe submodule with the matching #89 review fixes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Tracks astrolabe feat/session-derived-jwt-auth tip (86616d9). Test-only
change (rename a unit-test constant to clear SonarCloud S2068); no functional
or runtime difference from the previously-verified submodule commit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bumps the astrolabe submodule to validate the one-click background-sync app
password internally (IProvider::getToken) instead of via an HTTP loopback to
overwrite.cli.url, which is unreachable from inside the app container and broke
the multi-user-basic integration legs (enable_background_sync never succeeded).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Aligns PR #813's submodule pointer with astrolabe PR #89 tip (05176f9).
No MCP-server runtime change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Aligns PR #813's submodule pointer with astrolabe PR #89 tip (091c68b),
which makes `composer run psalm` clean. No MCP-server runtime change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
No runtime change — keeps the PR #813 submodule pointer aligned with the
astrolabe PR #89 tip (587caa6).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The OAuth provisioning tools (check_provisioning_status, revoke_nextcloud_
access) only consulted the refresh-token store + Astrolabe status, ignoring the
app_passwords store that Login Flow v2 (nc_auth_provision_access) and the
management API write to — the same store require_provisioning / get_client use
to grant tool access. Result: status reported "not provisioned" while tools
worked, and revoke said "nothing to revoke" while the credential persisted.
- _get_provisioning_status: also check storage.get_app_password_with_scopes,
reporting is_provisioned with credential_type=app_password,
flow_type=login_flow_v2.
- _revoke_nextcloud_access: when the credential is an app password, delete it
from storage + invalidate the scope cache (no IdP token to revoke);
refresh-token revocation via the Token Broker is unchanged.
- tests/unit/test_oauth_tools_app_password_provisioning.py: cover status +
revoke for the app-password path.
- bump astrolabe submodule (deprovision MCP on disable); fix a stale assertion
in the migrated bg-sync test (one-click flow has no separate app-password
generation step).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Astrolabe was refactored to mint session-derived JWTs (TokenGenerationRequest
Event) and a one-click background-indexing opt-in, dropping the OAuth
authorize/callback/refresh surface. Bump the submodule and bring the test
suite in line:
- New test_astrolabe_session_jwt_search.py: a logged-in user searches via the
minted JWT with no provisioning (replaces the obsolete login_flow_provisioning
OAuth-authorize test; token_refresh test deleted — refresh flow is gone).
- settings_buttons: assert the new revoke endpoint + that oauth/disconnect is
gone (404).
- multi_user_background_sync / plotly / chunk_context: drop the OAuth authorize
step; provision via the one-click "Enable background indexing" button
(#mcp-enable-background-button -> #mcp-revoke-background-button) instead of
generating + pasting an app password.
- docker-compose.yml: mount the astrolabe submodule into the app container.
- third_party/astrolabe: bump to the one-click opt-in commit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The `mcp-login-flow` profile's `ALLOWED_MGMT_CLIENT` was set to the
test-fixture id `nextcloudMcpServerUIPublicClient` only, but the actual
Astrolabe app provisions its OIDC client as
`astrolabeMcpClientOAuth00000000000` (see
`app-hooks/before-starting/26-configure-astrolabe-oauth.sh:39`). All
tokens issued through the "Enable Semantic Search" flow were rejected
with HTTP 401 by `unified_verifier.py:222-227`'s allowlist check, and
the Astrolabe UI's retry loop subsequently exhausted the
`api/passwords.py` 5/hr rate limit (HTTP 429).
Switch the `mcp-login-flow` allowlist to Astrolabe's client id so
production-shaped traffic actually validates. The `mcp-multi-user-basic`
profile keeps `nextcloudMcpServerUIPublicClient` for the
`configure_astrolabe_for_mcp_server` test fixture.
Also bump `third_party/astrolabe` 0.13.12 → 0.14.0 to pull in the
chunk-context indexed-lookup fix (#75) and the PDF bbox highlight
overlay (#76) that match the master-side changes already merged on
this branch.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Picks up astrolabe/astrolabe#61 which fixes app password provisioning
failure caused by loginName mismatch in ITokenProvider::generateToken().
This was the root cause of vector sync never indexing in multi-user
BasicAuth mode, which caused the plotly visualization test to fail.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Drop OIDC fork: comment out third_party/oidc mount, use upstream
v1.16.3 from app store (fixes consent redirect race, PR #631)
- Support client_secret_basic auth: add _extract_basic_auth() helper
so TS MCP SDK can authenticate at token endpoint (RFC 6749 §2.3.1)
- Multi-issuer JWT validation: accept tokens with internal Docker
issuer (http://app:80) or public URL (NEXTCLOUD_PUBLIC_ISSUER_URL)
since AS proxy obtains tokens server-to-server
- Introspection fallback: try token introspection when JWT verification
fails, supporting both JWT and opaque token types
- Register all tool scopes in DCR: add semantic:read, collectives:read,
collectives:write to OIDC client allowed_scopes so tokens include
them and semantic search tools are visible to authenticated clients
- Auto-create Astrolabe OAuth client: new app-hook creates OIDC client
and stores credentials in config.php so the "Authorize via OAuth"
button works without manual setup
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Extract reusable wait_for_vector_sync() helper with 90s max_wait (up from
30s) to handle slow single-worker processing in CI. Increase processor
workers to 2 for the mcp service to parallelize note indexing.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The test_semantic_search_answer_successful_sampling test creates a note
and waits 30s for indexing, but the scanner only ran every 60s. Aligning
with the CI overlay's 5s interval ensures new notes are indexed in time.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Make /api/v1/status and /api/v1/vector-sync/status available in all
non-Smithery deployment modes so Astrolabe can show server status even
in single-user BasicAuth mode. Previously these were only mounted when
OAuth or multi-user BasicAuth with offline access was enabled.
- Split management API routes into public (Tier 1) and authenticated (Tier 2+)
- Enable semantic search with in-memory Qdrant for single-user docker service
- Update astrolabe submodule with admin settings fix
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause: ConsentController::grant() only passed client_id and scope
in the post-consent redirect, relying on PHP session fallback for state,
response_type, redirect_uri etc. On NC 32 (PHP 8.4), session values
were intermittently lost between session->close() and the subsequent GET
request, causing 500 errors from trim(null) / matchRedirectUri(null).
OIDC app fixes:
- Pass all OAuth params in consent redirect URL (eliminates session race)
- Add null safety guard in authorize endpoint (400 instead of 500)
Test infra fixes:
- Wait for OIDC redirect chain to settle before handling consent screen
(fixes "Execution context was destroyed" Playwright errors)
- Capture nextcloud.log in CI failure artifacts for PHP error debugging
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The OIDC app 1.16.2 broke the consent flow by only falling back to
session params when client_id is missing. After consent, the redirect
includes client_id and scope but loses state, response_type, and
redirect_uri — causing a 500. The submodule fix restores per-param
session fallback when ANY critical param is missing.
Also adds a CI build step for the OIDC app (composer + npm) so the
JS assets (oidc-consent.js, oidc-redirect.js) are available in OAuth
test profiles.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
MCP clients like Claude Code were unable to use tools because tokens
obtained directly from Nextcloud had the wrong audience claim. The MCP
server now acts as its own OAuth Authorization Server, proxying auth
to Nextcloud with its own client_id so tokens have the correct audience.
New endpoints: /.well-known/oauth-authorization-server, /oauth/token,
/oauth/register. Modified /oauth/authorize from pass-through to
intermediary pattern. PRM now points authorization_servers to the MCP
server instead of Nextcloud.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add cross-product matrix (3 versions x 4 auth modes = 12 CI jobs)
- Parameterize Nextcloud image in docker-compose.yml via NEXTCLOUD_IMAGE env var
- Pin NC 31.0.8, 32.0.6, 33.0.0 with SHA digests in workflow
- Add Renovate customManagers to auto-update NC images in workflow
- Fix Astrolabe install hook to prefer volume mount over app store
- Bump Astrolabe submodule to support NC 33 (max-version 31→33)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Astrolabe has been extracted to its own repository at
github.com/cbcoutinho/astrolabe for independent releases.
Changes:
- Replace third_party/astrolabe/ directory with git submodule
- Remove astrolabe-ci.yml and appstore-build-publish.yml workflows
- Remove scripts/bump-astrolabe.sh
- Remove Astrolabe sections from bump-version.yml workflow
- Remove Astrolabe build steps from test.yml CI workflow
- Remove astrolabe volume mount from docker-compose.yml
- Simplify astrolabe install hook to always use app store
- Update CONTRIBUTING.md to reflect two-component monorepo
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add pagination to getAllUsersWithTokens() with limit/offset params
- Update RefreshUserTokens to process users in batches of 100
- Add lock TTL documentation to withTokenLock() docstring
- Fix psalm type errors in getAccessToken() method
- Add unit tests for pagination and batched processing
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Adds distributed locking using Nextcloud's ILockingProvider to prevent
race conditions between background job and on-demand token refresh.
Uses double-check locking pattern:
1. Quick check without lock - return immediately if token is valid
2. Acquire exclusive lock if token needs refresh
3. Re-check after lock - another process may have refreshed
4. Refresh only if still needed
5. Graceful degradation on LockedException
Changes:
- McpTokenStorage: add ILockingProvider, withTokenLock() method
- McpTokenStorage: update getAccessToken() with locking pattern
- RefreshUserTokens: wrap refresh in withTokenLock(), catch LockedException
- Add comprehensive unit tests for locking behavior
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Fixes missing issued_at parameter when storing tokens refreshed via
getAccessToken() callback, ensuring accurate token lifetime calculation
for the background refresh job.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Prevents users from having to re-authorize Astrolabe after periods of
inactivity by proactively refreshing OAuth tokens before they expire.
Changes:
- Add RefreshUserTokens background job that runs every 15 minutes
- Add on-demand token refresh in SemanticSearchProvider (Unified Search)
- Store issued_at timestamp for accurate token lifetime calculation
- Add getAllUsersWithTokens() to query users needing refresh
The job dynamically calculates refresh threshold based on actual token
lifetime (50% remaining), working with any IdP (Nextcloud OIDC, Keycloak,
etc.) rather than relying on IdP-specific configuration.
Closes#510
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
