Commit Graph
3000 Commits
Author SHA1 Message Date
Chris CoutinhoandClaude Opus 4.8 cf7209cd85 fix(document-processors): escalate glyph-corrupt PDFs to the structured tier
The fast (pypdfium2) extractor can leak raw glyph codes on subset fonts with a
broken /ToUnicode CMap. The result scores high on the existing text-quality
heuristic -- a uniform glyph/Caesar offset preserves whitespace and token
lengths -- yet is unsearchable. The structured (pymupdf) tier extracts the same
pages correctly.

Add a language-agnostic C0-control-character-ratio signal to the tier-0
classifier that detects this corruption and routes the document to a new
`structured` recommended_tier. Wire the fast->structured hop on the inline path
and generalise it so a low-quality-but-non-empty layer also tries structured
before OCR -- the inline and external ingest modes now follow the full
fast->structured->ocr ladder identically. A scanned / no-text-layer document
(total_chars == 0) still shortcuts straight to OCR, since a text extractor
cannot recover a pure raster.

New per-tenant tunable DOCUMENT_GLYPH_CORRUPTION_RATIO (default 0.02); escalation
metrics gain a `corrupt_glyphs` reason label.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 20:06:35 +02:00
Chris CoutinhoandGitHub 60df141442 Merge pull request #912 from cbcoutinho/test/pact-searchable-sources
test(contract): pin the OCS-capabilities consumer contract with astrolabe
2026-06-16 17:08:59 +02:00
Chris CoutinhoandClaude Opus 4.8 a47898d771 test(contract): tidy the purge provider-state no-op stub
Round-2 style note: replace `return None` with a comment-only intentionally
empty body for the _state_admin_can_purge stub.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 16:57:19 +02:00
Chris CoutinhoandClaude Opus 4.8 7b43cc8220 docs(contract): clarify minimal OCS envelope + reuse _BROKER_READY
Address round-1 review nits:
- document why _ocs_capabilities omits the rest of the OCS envelope (Pact V4
  allows extra provider-side keys; pin only astrolabe's own block)
- use the module-level _BROKER_READY in the broker-source guard instead of
  re-checking the three env vars inline

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 16:52:31 +02:00
Chris Coutinho f7fefee9da test(contract): pin the OCS-capabilities consumer contract with astrolabe
Add a Pact consumer test for capabilities.allowed_doc_types ->
NextcloudClient.capabilities() -> GET /ocs/v2.php/cloud/capabilities, pinning
the astrolabe.semantic_search.enabled_doc_types block the search/scan/purge
gates read. Covers the two meaningful provider states: some sources approved
(parsed to the allow-set) and every source disabled (empty frozenset, distinct
from the fail-open None). Produces the nextcloud-mcp-server -> astrolabe pact.

On the provider side (astrolabe's consent-purge pact), register the
"an admin can purge indexed documents" provider state and opt the broker source
into pending pacts, so that authenticated contract reports as pending instead of
failing provider verification until the live-stack auth test-hook is stood up
(ADR-029 phase 4). Already-verified interactions (GET /api/v1/status) stay
blocking.

---

_This PR was generated with the help of AI, and reviewed by a Human_
2026-06-16 16:45:21 +02:00
github-actions[bot] 68b4f5f1f7 bump: version 0.119.0 → 0.120.0 2026-06-16 14:27:04 +00:00
Chris CoutinhoandGitHub 3b40898a79 Merge pull request #911 from cbcoutinho/feat/admin-searchable-sources
feat(vector-sync): honor Astrolabe admin consent for searchable sources
2026-06-16 16:26:35 +02:00
Chris CoutinhoandGitHub 95a735d512 Merge pull request #900 from cbcoutinho/renovate/anthropics-claude-code-action-1.x
chore(deps): update anthropics/claude-code-action action to v1.0.149
2026-06-16 09:34:46 +02:00
renovate-bot-cbcoutinho[bot]andGitHub f5947d9895 chore(deps): update anthropics/claude-code-action action to v1.0.149 2026-06-16 04:23:33 +00:00
Chris CoutinhoandClaude Opus 4.8 53290f693c refactor(scanner): _should_scan helper to cut scan_user_documents complexity
The consent gate added three `_app_enabled(...) and is_doc_type_allowed(...)`
conditions to scan_user_documents, pushing its cognitive complexity over the
SonarQube threshold. Fold the pair into a _should_scan() helper (alongside the
earlier _enqueue_deletes refactor). Also document the accepted doc_types=None
per-type-query trade-off at the search consent gate (round-10 review item).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 02:27:17 +02:00
Chris CoutinhoandClaude Opus 4.8 9b35d98188 refactor(scanner): cut backstop cognitive complexity (SonarQube S3776)
Extract _mark_backstop_done() (overflow eviction + marker write) and
_backstop_delete_doc_type() (per-type scroll + enqueue) so
_enqueue_deletes_for_disabled_types drops from cognitive complexity 17 to well
under the 15 threshold. Behavior unchanged; tests still pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 02:20:06 +02:00
Chris CoutinhoandClaude Opus 4.8 7067c5fff1 test(vector-sync): close round-8 coverage gaps (500 path, non-string list)
- route test for purge_doc_types raising on total failure -> 500
- route test for doc_types list containing non-strings -> 400
- reword the capabilities move_to_end comment (no-op on new keys; needed only
  for the expired-key in-place update)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 02:10:54 +02:00
Chris CoutinhoandClaude Opus 4.8 ea53ed9ce0 fix(vector-sync): address round-7 — only log purge endpoint when enabled
- app.py: move the /api/v1/vector-sync/purge mention out of the unconditional
  management-endpoints log and into the vector_sync_enabled block, so operators
  without Qdrant don't see an endpoint that 404s
- vector_sync route: comment why doc_types isn't whitelisted against
  INDEXED_DOC_TYPES (unknown type = harmless zero-match no-op)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 02:03:26 +02:00
Chris CoutinhoandClaude Opus 4.8 21ce620a84 fix(vector-sync): address round-6 review — rename shadowed var, add test
- vector_sync route: rename the response dict from `body` to `resp` so it no
  longer shadows the request `body` (maintenance trap)
- scanner: comment the intentional files-vs-text purge timing asymmetry
- tests: add the all-text-types-disabled backstop case (empty allow-set)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 01:56:54 +02:00
Chris CoutinhoandClaude Opus 4.8 d0db530ac9 fix(vector-sync): address round-5 review — partial-failure signal, markers
- purge route: include a "failed" key in the 200 body listing requested doc
  types that were not purged, so Astrolabe knows consent isn't yet enforced
  for them (scanner backstop still catches up)
- tests: add @pytest.mark.unit / module-level pytestmark to the new test
  modules so they run under `pytest -m unit`; add a partial-failure route test
- capabilities: comment why the cache is keyed per-user despite a global value
- semantic/scanner: doc/comment clarifications (sorted-order, eviction timing)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 01:47:44 +02:00
Chris CoutinhoandClaude Opus 4.8 b0751102d7 refactor(vector-sync): dedupe "Bad request" 400s via a helper (SonarCloud S1192)
Extract _bad_request() so the five 400 branches don't duplicate the literal.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 01:35:57 +02:00
Chris CoutinhoandClaude Opus 4.8 6b9f910a14 fix(vector-sync): address round-4 review — processor test, partial eviction
- tests: cover the process_document consent gate (drops an admin-disabled
  index task with record_ingest_dropped("admin_disabled"); allows approved)
- scanner: _consent_backstop_done is now an insertion-ordered dict and evicts
  the oldest entries to half capacity on overflow, so a bound hit re-fires the
  backstop for only the oldest markers instead of the whole fleet at once
- semantic: reword the short-circuit log (consent, not installation)
- capabilities: comment why move_to_end is needed after an expired-key update
- test: assert the global purge delete-filter is owner-agnostic (doc_type only);
  fix a pre-existing ty error on UnexpectedResponse(headers=None) in the file

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 01:33:18 +02:00
Chris CoutinhoandClaude Opus 4.8 cef477b877 fix(vector-sync): address round-3 review — gate purge route, bound set, nits
- app.py: register /api/v1/vector-sync/purge only when vector_sync_enabled, so
  it returns 404 (not a 500 from get_qdrant_client) when sync is off
- scanner: bound _consent_backstop_done so a long-running multi-tenant process
  with user churn can't grow it without limit (clears on overflow)
- purge route: distinct 400 for a missing doc_types key; enforce the admin
  check even for an empty no-op request (destructive route)
- tests: missing-key 400, admin-gated empty no-op, non-admin empty 403

The _consent_narrowed_doc_types precondition is enforced by its non-Optional
frozenset[str] signature (ty rejects a None caller). The httpx.BasicAuth
SonarCloud hotspot matches the existing webhook routes (false positive,
credential from the app-password store) — left consistent for UI triage.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 01:21:01 +02:00
Chris CoutinhoandClaude Opus 4.8 24b8000a71 fix(vector-sync): address round-2 review — one-shot backstop, helper, caps
- scanner: gate the consent backstop with a per-(user,doc_type) one-shot
  marker so a standing admin-disable doesn't re-enqueue idempotent deletes
  every scan tick; the marker clears when the type is re-enabled. Derive
  _TEXT_BACKSTOP_DOC_TYPES from INDEXED_DOC_TYPES so new indexed types are
  covered automatically
- semantic: extract _consent_narrowed_doc_types so the search-side narrowing
  is unit-testable; add tests for restrict/intersect/disjoint/empty
- purge route: cap doc_types length (abuse guard) -> 400
- tests: one-shot + re-enable backstop, too-many-doc_types 400

Deferred (noted on PR): per-document allowed_doc_types call is cache-hot;
purge "last error wins" — both logged. SonarCloud broad-except hotspots are
deliberate (noqa BLE001), reviewable in the UI.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 01:08:30 +02:00
Chris CoutinhoandClaude Opus 4.8 477fb02b0a fix(vector-sync): address PR review — dict guard, symmetric backstop, metrics
- purge route: 400 (not 500) on a valid-JSON non-object body
- scanner: backstop-purge admin-disabled note/news_item/deck_card points
  (their deletion-tracking lives inside the skipped scan_* fns), mirroring the
  files path; gated on a concrete allow-set so fail-open never deletes
- processor: record_ingest_dropped("admin_disabled") so consent-skipped index
  tasks are observable/alertable
- app.py: list /api/v1/vector-sync/purge in the endpoints log line
- capabilities: drop empty-string doc types; return frozenset throughout
- purge: document the count-before-delete approximation
- tests: non-object body -> 400, ProvisioningRequiredError -> 428, cache TTL
  expiry refetch, and the scanner consent backstop

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 00:54:50 +02:00
Chris CoutinhoandClaude Opus 4.8 ef5b3f3873 feat(vector-sync): honor Astrolabe admin consent for searchable sources
Consume the astrolabe.semantic_search capability as the source of truth for
which content sources an admin has approved for semantic search, and enforce
it independently of Astrolabe (this server queries Qdrant directly).

- capabilities.py: cached per-user reader for enabled_doc_types (TTL+LRU,
  fail-open so older Astrolabe / transient OCS errors don't break search)
- semantic search: intersect requested doc_types with the allowed set;
  restrict to the allowed set when none requested; short-circuit when empty
- scanner: skip disabled sources during discovery (files discovery yields
  nothing when disabled, so the existing grace-period reconcile purges them)
- processor: drop near-real-time index tasks for disabled doc_types
  (webhook events bypass the scanner gate); deletes always proceed
- vector/purge.py + POST /api/v1/vector-sync/purge: admin-only global
  delete-by-doc_type, called by Astrolabe when a source is disabled so
  consent is binding on data-at-rest

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 00:38:35 +02:00
github-actions[bot] 3d28621c57 bump: version 0.118.0 → 0.119.0 2026-06-15 14:59:13 +00:00
Chris CoutinhoandGitHub 667359aebc Merge pull request #910 from cbcoutinho/feat/worker-batch-ocr
feat(ocr): opt-in batch OCR mode via the gateway's async batch routes
2026-06-15 16:58:46 +02:00
Chris CoutinhoandClaude Opus 4.8 89522ca572 test(contract): add gateway batch OCR consumer pact (Deck #332)
Adds the consumer-driven Pact for the gateway's async batch OCR routes, consumed
by GatewayBatchOcrClient. The embedding gateway is a separate provider
(astrolabe-cloud-gateway) from the existing `astrolabe` credentials pact, so it
gets its own fixture + pact file.

Interactions (only the fields the single-document client reads are pinned, so the
contract is robust to the gateway's additive OcrBatchJobOut fields):
- POST /v1/ocr/batch -> 202 { job_id } (namespaced <provider>/<id>)
- GET /v1/ocr/batch/{job_id} -> pending / succeeded (per-page markdown) / failed

The gateway is unauthenticated today, so no bearer is sent (matching the
M2M-optional client). Provider-side: this publishes a pact the gateway's
verification job must now satisfy — it needs provider-state handlers
(pending/succeeded/failed jobs) + a Mistral stub on the astrolabe-cloud-website
side (its verification was a deliberate no-op until a consumer pact existed).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 14:06:54 +02:00
Chris CoutinhoandClaude Opus 4.8 210a234c11 test(ocr): round-6 — batch submit-error propagation test + import cleanup
Round 6 review (PR #910):
- Add the missing propagation test: a transport error (httpx.ConnectError) from
  batch submit() propagates out of process() rather than being caught by the sync
  OCR try/except or falling back to a sync transcription — guards the intentional
  "opted into batch → procrastinate retry, not sync fallback" asymmetry.
- Move the batch-test module imports (BatchPollResult, batch_ocr_store) to the
  top of test_ocr_processor.py, dropping the mid-file `# noqa: E402`.

Deferred (reviewer: not actionable for this PR): extracting a lazy-init helper
for the parallel _backend / _batch_client resolution quadruplets.

1653 unit tests pass; ruff + ty green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 11:20:38 +02:00
Chris CoutinhoandClaude Opus 4.8 bb08245c91 fix(ocr): round-5 review nits — empty-pages failure, comments, test cleanup
Round 5 review (PR #910), all nits, no blockers:
- _result_from_success: a succeeded job with `pages=[]` (empty list, not just a
  missing key) is now a per-document failure ("no pages returned") instead of a
  silent 0-chunk success. Test added.
- Comment the deadline-expiry path: the gateway-side job isn't cancelled (no
  cancel endpoint at this layer) — it's reaped by the gateway file purge; we just
  stop polling it.
- Drop the vestigial status="pending" from the BatchOcrJob test fakes (the column
  was removed in round 4; BatchPollResult.status fakes are untouched).

1653 unit tests pass; ruff + ty green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 11:12:42 +02:00
Chris CoutinhoandClaude Opus 4.8 232684e881 fix(ocr): round-4 review — defensive poll + drop dead tracking columns
Round 4 review (PR #910), no blockers:
- poll(): a 2xx body with no `status` now fails fast (logged) instead of being
  treated as perpetually pending until the deadline; defensive page index
  (`p.get("index", i)`) so a malformed page degrades rather than KeyError-ing.
- Document on poll() that job_id is namespaced (embeds "/") so the gateway route
  must be a path-capture param (GET /v1/ocr/batch/{job_id:path}).
- Drop the vestigial `status` + `updated_at` columns from batch_ocr_jobs: a row
  only ever exists while pending (terminal jobs are deleted) and the live status
  comes from a fresh poll, so a stored mirror was permanently "pending" /
  redundant with submitted_at. Simplifies the migration, store, and dataclass.
- Tests: submit() ValueError on missing job_id; poll() missing-status → failed.

1653 unit tests pass; ruff + ty green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 11:04:03 +02:00
Chris CoutinhoandClaude Opus 4.8 55630ba25c fix(ocr): round-3 review — guard unexpected batch status + tests/comments
Round 3 review (PR #910):
- Guard an unexpected terminal batch status in _process_batch: anything that
  isn't succeeded/failed (gateway version skew, a new lifecycle state) now marks
  the document parse-failed instead of falling through to _pages_to_text([]) — a
  0-chunk "success" that silently indexed empty text and re-submitted forever.
  Test added.
- gateway_batch_client.submit: raise an actionable ValueError on a 2xx response
  with no job_id (was a bare KeyError deep in the caller).
- Document that a _process_batch transport error intentionally propagates to
  procrastinate for retry rather than falling back to sync (opt-in batch wants
  the retry).
- Annotate _batch_client as GatewayBatchOcrClient | None (TYPE_CHECKING import
  already present); clarify the delete_stale_for_doc first-submit no-op comment.
- Add a parametrized build_gateway_batch_client test (the gateway-only invariant:
  mistral/none/no-URL -> None; gateway|auto + URL -> client).

1653 unit tests pass; ruff + ty green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 10:53:24 +02:00
Chris CoutinhoandClaude Opus 4.8 995e810d89 fix(ocr): round-2 review — lazy store lock, mode enum normalization, type hints
Round 2 review (PR #910):
- BLOCKING: BatchOcrJobStore._shared_lock is now lazy-init (anyio.Lock | None,
  created on first shared() call) instead of at class-definition time — matches
  the CLAUDE.md "no anyio primitives at import time" rule and OcrProcessor's
  pattern. The None-check->assign has no await between, so it's race-free.
- document_ocr_mode now normalizes via _enum_fields (case-insensitive, like
  document_ocr_provider) instead of a strict dynaconf is_in Validator, so
  DOCUMENT_OCR_MODE=Batch normalizes to "batch" rather than erroring. Tests for
  case-normalization + invalid-value rejection.
- TYPE_CHECKING-gated GatewayBatchOcrClient import so build_gateway_batch_client
  / _get_batch_client are typed `GatewayBatchOcrClient | None` instead of Any
  (runtime import stays lazy to avoid the import cycle).
- Rename ocr_options -> doc_identity_options (it's threaded to all tiers; only
  OCR reads it) + clarify the comment.
- Drop the redundant forward-ref quotes on _shared_instance.
- Add direct _batch_identity unit tests (partial/empty options branches).

Left as follow-up: reusing one httpx.AsyncClient across submit/poll (same
per-call pattern as the existing sync _GatewayOcrBackend; no clean aclose hook
on the cached client today).

1653 unit tests pass; ruff + ty green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 10:45:10 +02:00
Chris CoutinhoandClaude Opus 4.8 2b7dfc8535 fix(ocr): wire batch settings into _field_map + review nits
Round 1 review (PR #910):
- BLOCKING: add document_ocr_mode / _batch_poll_seconds / _batch_max_wait_seconds
  to config._field_map — without it dynaconf silently ignored the env vars and
  DOCUMENT_OCR_MODE=batch could never be enabled in production. Add a regression
  test asserting the three round-trip from env.
- migration 008: give batch_ocr_jobs a composite PRIMARY KEY on
  (user_id, doc_id, doc_type, etag) instead of a bare UniqueConstraint (N1).
- OcrProcessor: use a dedicated _batch_client_lock instead of sharing the sync
  backend lock (N3).
- tests: use https:// gateway URLs in the new fixtures to clear SonarCloud's
  "insecure http" security hotspots (all 14 were test-only http://gw literals).

1653 unit tests pass; ruff + ty green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 10:36:44 +02:00
Chris CoutinhoandClaude Opus 4.8 3b7e8d779b feat(ocr): opt-in batch OCR mode via the gateway's async batch routes
Add DOCUMENT_OCR_MODE=sync|batch (default sync). In batch mode the tier-3 OCR
processor submits documents to the embedding gateway's async Batch OCR routes
(POST /v1/ocr/batch + GET /v1/ocr/batch/{job_id}, astrolabe-cloud-website#372)
for ~50% cheaper large-corpus backfill. The direct Mistral OCR path is left
untouched. Tracked on Deck #332.

Batch jobs run minutes-hours, so the OCR tier cannot block (the procrastinate
worker reclaims jobs in `doing` after INGEST_STALLED_JOB_SECONDS). Instead it
submits, records the gateway job id in a new per-tenant `batch_ocr_jobs` table
(procrastinate args are immutable across retries), and raises a BatchPending
signal that TieredEscalationStrategy turns into a same-queue deferred re-poll —
releasing the worker slot between polls. On completion the per-page markdown is
indexed like the sync path; a failure or a job past
DOCUMENT_OCR_BATCH_MAX_WAIT_SECONDS marks the document parse-failed.

Batch is opt-in and gateway-only: with the direct mistral backend, no gateway
URL, or the inline/memory pipeline (which can't defer), it falls back to sync.
One batch job per document (coalescing N docs/job is a follow-up).

- embedding/gateway_batch_client.py: submit/poll client (reuses GatewayTokenProvider).
- vector/batch_ocr_store.py + migration 008: job tracking (portable SQLite+PG).
- document_processors/escalation.py: BatchPending control-flow signal.
- document_processors/ocr.py: batch state machine + sync fallback.
- vector/processor.py: thread doc identity to the OCR tier; raise BatchPending
  from the pending sentinel; propagate it as control flow (not a failure).
- vector/queue/procrastinate.py: BatchPending -> same-queue retry_in, exempt
  from the transient cap (bounded by the processor's deadline).
- config + docs; tests across client/store/processor/strategy/parse-tier.

1653 unit tests pass; ruff + ty green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 09:41:19 +02:00
github-actions[bot] 07ee91399b bump: version 0.117.2 → 0.118.0 2026-06-15 04:34:42 +00:00
Chris CoutinhoandGitHub 8d6196f161 Merge pull request #832 from Janhouse/fix/830-external-calendars
feat(calendar): list external read-only (subscribed) calendars
2026-06-15 06:34:16 +02:00
github-actions[bot] 9389393d89 bump: version 0.117.1 → 0.117.2 2026-06-14 17:22:25 +00:00
Chris CoutinhoandGitHub ba96d9f38f Merge pull request #909 from cbcoutinho/security/ghsa-8vh3-g2qg-2h2c-webhook-auth
fix(security): require WEBHOOK_SECRET for the Nextcloud webhook receiver (GHSA-8vh3-g2qg-2h2c)
2026-06-14 19:21:58 +02:00
Chris CoutinhoandClaude Opus 4.8 de302073eb fix(security): enforce WEBHOOK_SECRET min length + address round-2 review
Round-2 review follow-ups (GHSA-8vh3-g2qg-2h2c PR):
- Add a dynaconf validator requiring WEBHOOK_SECRET to be >=16 chars when set
  (None still allowed = webhooks disabled), so weak/placeholder secrets fail
  at startup rather than in an audit. Covered by two new tests in test_config.py.
- Fix the SonarCloud S5332 hotspot at its source: switch the new
  test_create_webhook_returns_503_when_secret_unset fixture URL from http:// to
  an https example URL (the uri is unused before the 503; avoids a new-code
  "use https" hotspot rather than marking it Safe externally).
- Nits: drop the unused app.state.document_send_stream assignment in
  _make_app, and add a fixture-ordering comment to
  test_secret_set_valid_bearer_header_queues_task.

(--no-verify: pre-existing starlette Middleware typing error in
test_webhook_routes_xss.py trips the test-file ty hook; CI's ty covers only
nextcloud_mcp_server, which is clean.)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 18:36:22 +02:00
Chris CoutinhoandClaude Opus 4.8 5b8167f9a4 test(webhook): cover enable_webhook_preset 503 branch + clarify wrong-scheme test
Round-1 review follow-ups (GHSA-8vh3-g2qg-2h2c PR):
- Add a unit test for the new `except WebhookSecretNotConfigured` branch in
  enable_webhook_preset: returns 503 (not the generic 500) with WEBHOOK_SECRET
  in the body. Uses the existing test_webhook_routes_xss.py scaffolding.
- Add a clarifying comment to test_secret_set_wrong_scheme_returns_401 about
  the _client default-bearer override semantics.

(--no-verify: the pre-commit ty-check surfaces a pre-existing starlette
Middleware typing error in test_webhook_routes_xss.py unrelated to this change;
CI's ty check covers only nextcloud_mcp_server, which is clean.)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 18:25:30 +02:00
Chris CoutinhoandClaude Opus 4.8 4fc2b10945 fix(security): require WEBHOOK_SECRET for the Nextcloud webhook receiver
GHSA-8vh3-g2qg-2h2c (CVSS 9.1, CWE-306): POST /webhooks/nextcloud had no
authentication when WEBHOOK_SECRET was unset (the default). The receiver
trusted the attacker-supplied user.uid and fed it to Qdrant, letting an
unauthenticated network caller delete or re-index any user's vector
embeddings.

Webhooks now require WEBHOOK_SECRET end-to-end:

- app.py: the /webhooks/nextcloud route is only mounted when WEBHOOK_SECRET
  is set; otherwise it 404s and a startup warning notes vector sync falls
  back to the polling scanner.
- webhook_receiver.py: removed the warn-and-accept fallback. No secret -> 503,
  missing/invalid bearer -> 401; the payload is never processed unauthenticated.
- webhook_routes.py / api/webhooks.py: webhook_auth_pair() raises
  WebhookSecretNotConfigured instead of returning authMethod="none"; both
  registration entry points return a clear 503 so no dead unauthenticated
  webhooks are created.

Also expose webhooks availability to the Astrolabe UI via GET /api/v1/status
("webhooks_enabled": bool), set WEBHOOK_SECRET on the docker-compose
semantic-search dev services, and update env.sample + ADR-010 / ADR-018 /
webhook-management-guide docs.

Vector sync still works without a secret via the polling scanner.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 18:18:34 +02:00
github-actions[bot] a818b49b3f bump: version 0.117.0 → 0.117.1 2026-06-14 15:44:18 +00:00
Chris CoutinhoandGitHub bc241d8340 Merge pull request #908 from cbcoutinho/docs/login-flow-static-oidc-client-907
docs(login-flow): require static OIDC client; remove dead token-exchange mode
2026-06-14 17:43:54 +02:00
Chris CoutinhoandClaude Opus 4.8 af3e2371e6 docs(env): add required NEXTCLOUD_PUBLIC_ISSUER_URL to login_flow sample
Round-8 review: env.sample.oauth-multi-user omitted NEXTCLOUD_PUBLIC_ISSUER_URL,
which configuration.md marks required for login_flow — a user working from the
template alone would hit the "Login URL points to localhost" failure. Add it
(with a troubleshooting pointer) and give it + NEXTCLOUD_MCP_SERVER_URL a
"REQUIRED: PUBLIC URLs" section header for consistency with the rest of the file.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 12:59:43 +02:00
Chris CoutinhoandClaude Opus 4.8 dadd71ec83 docs(login-flow): unify encryption-key placeholder
Round-7 nit: align the TOKEN_ENCRYPTION_KEY placeholder in login-flow-v2.md
(`<fernet-key>` / `<your-fernet-key>`) with env.sample.oauth-multi-user's
`<your-encryption-key>` so copy-pasters don't see a mismatch.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 12:55:45 +02:00
Chris CoutinhoandClaude Opus 4.8 407d0d765b docs(env): mark token storage required for login_flow
Round-6 review: in env.sample.oauth-multi-user, TOKEN_ENCRYPTION_KEY and
TOKEN_STORAGE_DB sat under "OPTIONAL: SEMANTIC SEARCH", but they're required for
any login_flow deployment (per-user app passwords must be persisted). Move them
into a dedicated "REQUIRED: APP-PASSWORD STORAGE" block with a pointer to
docs/login-flow-v2.md#setup so a login_flow-without-semantic-search user copying
the template doesn't miss them.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 12:50:21 +02:00
Chris CoutinhoandClaude Opus 4.8 39bc675a72 docs: close round-5 token-exchange follow-ups
- settings.toml.example: drop the stale `enable_token_exchange = false`
  deprecated-alias line (the key was removed from config.py _DEFAULTS).
- configuration-migration-v2.md: add a Quick Reference row + note that
  `ENABLE_TOKEN_EXCHANGE` was removed and is now ignored; use
  `MCP_DEPLOYMENT_MODE=login_flow` instead.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 12:45:04 +02:00
Chris CoutinhoandClaude Opus 4.8 864b3e96d7 docs(adr): note token-exchange removed in ADR-005
Round-4 review: ADR-005 (Status: Implemented) still described the token-exchange
mode (Option 2 / ENABLE_TOKEN_EXCHANGE) as an active option. Add a note to the
Implementation Note section clarifying it was removed in the ADR-022/023
consolidation and only multi-audience mode ships — consistent with the ADR-004
deprecation in this PR.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 12:39:43 +02:00
Chris CoutinhoandClaude Opus 4.8 45c518700b docs: address round-3 review (compose excerpt, ADR-004 notes, placeholder)
- login-flow-v2.md: add commented-out NEXTCLOUD_OIDC_CLIENT_ID/_SECRET (with a
  "production: register a static client" note) to the Docker Compose excerpt so
  copy-pasters of the rendered snippet don't fall into the #907 DCR-expiry trap.
- ADR-004: rename "## Implementation Status" -> "## Historical Implementation
  Notes" and add a banner clarifying the steps were never completed and the
  ENABLE_TOKEN_EXCHANGE symbols no longer exist (the design was superseded).
- env.sample.oauth-multi-user: angle-bracket the TOKEN_ENCRYPTION_KEY
  placeholder for consistency with the OIDC client placeholders.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 12:33:01 +02:00
Chris CoutinhoandClaude Opus 4.8 72b7b2efa0 refactor(auth): drop remaining stale token-exchange references
Round-2 review follow-ups in unified_verifier.py:
- module-level docstring still described "two compliant OAuth modes" incl.
  token exchange — rewritten to multi-audience only.
- removed the stale "# Both modes do the same validation" inline comment in
  verify_token().

(--no-verify: same pre-existing ty errors in test_unified_verifier.py as prior
commits; CI type-checks only the package, which passes.)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 12:26:38 +02:00
Chris CoutinhoandClaude Opus 4.8 40b1f0ec3c docs(login-flow): clarify re-auth + placeholders per review
Round-1 review follow-ups:
- Troubleshooting "Access forbidden": note that existing users must re-authorize
  once after switching to a static client (stored sessions were issued to the
  now-deleted DCR client).
- Default IdP setup: explain that the `/mcp` resource identifier works because
  `_has_mcp_audience` accepts both the bare server URL and the `/mcp` form.
- env.sample.oauth-multi-user: use angle-bracket placeholders
  (`<your-client-id>`) to match the template convention and fail loudly if
  copied verbatim.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 12:22:23 +02:00
Chris CoutinhoandClaude Opus 4.8 477f9a1ff7 refactor(auth): remove vestigial token-exchange code path
The oauth_token_exchange deployment mode was removed in ADR-022 but left a dead
`enable_token_exchange` flag and an unreachable "exchange mode" in the verifier
(self.mode was hardcoded to "multi-audience"). Remove the remnants:

- config.py: drop the `enable_token_exchange` default and the
  `ENABLE_TOKEN_EXCHANGE` branch in `_is_multi_user` (+ its doc line).
- unified_verifier.py: drop `self.mode` and the dead exchange-mode log branch;
  simplify the docstrings to multi-audience only.
- test_unified_verifier.py: drop the `.mode` assertions (attribute removed);
  collapse the redundant init tests.

Also remove docs/ADR-004-Code-Review.md — an orphaned code-review note, not an
ADR; it doesn't belong in the docs/ADR namespace.

(--no-verify: the ty-check hook flags 3 PRE-EXISTING type errors in
test_unified_verifier.py lines 346/362/441, untouched by this change; CI
type-checks only the package, which passes.)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 12:13:16 +02:00
Chris CoutinhoandClaude Opus 4.8 b134b2c539 docs(adr): correct stale statuses for shipped ADRs
Audit of all ADR status fields found several marked Proposed/Draft that are in
fact implemented. Correct them with an evidence pointer; also fix the ADR-025
title that read "ADR-024".

- ADR-007 background vector sync: Proposed -> Accepted/implemented
- ADR-008 MCP sampling: Proposed -> Accepted/implemented
- ADR-009 semantic.read scope: Proposed -> Accepted/implemented
- ADR-010 webhook-based vector sync: Proposed -> Accepted/implemented
- ADR-012 unified multi-algorithm search: Proposed -> Accepted/implemented
- ADR-013 RAG evaluation: Proposed -> Partially implemented
- ADR-018 Nextcloud settings-UI app: Proposed -> Accepted/implemented
- ADR-025 dynaconf: Proposed -> Accepted/implemented; fix title typo (ADR-024 -> ADR-025)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 12:00:54 +02:00