Round-3 review on PR #893:
- record_qdrant_operation("upsert","error") now fires only when the exhausted
retry was actually a Qdrant failure (reason=="qdrant"); an embed/connection
failure exhausts retries before Qdrant is called, so attributing it to
mcp_qdrant_operations_total{error} inflated that signal. The cause is still
captured by record_ingest_dropped.
- Add test_mistral_embed_retries_on_5xx: exercises the full Mistral retry path
(5xx SDKError then success), not just the predicate.
- Add test_generate_does_not_retry_on_bad_request: generate() fast-fails on a
permanent 4xx.
- Move astrolabe_vector_ingest_dropped_total's definition into the astrolabe_
pipeline-metrics block (was in the mcp_ section).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Round-2 review on PR #893:
- Add test_generate_retries_on_connection_error (generate() shares the transient
retry; guards the decorator against accidental removal).
- Add test_process_document_records_drop_on_exhausted_retries: drives
process_document to retry-exhaustion and asserts record_ingest_dropped is
called once with the classified reason (processor-level coverage, not just the
_drop_reason unit).
- Note in _drop_reason that a multi-failure group is labelled by its first leaf
(best-effort, no "mixed" bucket).
SonarCloud: the quality gate was failing on new_security_hotspots_reviewed
(S5332 "use https") from http:// URLs in the test _req() helpers — switched to
https:// (mirrors commit 98c9d58e).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Round-1 review on PR #893:
- _drop_reason now descends through nested ExceptionGroups to the first leaf
(was single-level), so a doubly-wrapped cause isn't mislabelled "other";
added a nested-group test. Commented why both the httpx and openai isinstance
branches exist (raw Nextcloud-API errors vs SDK-wrapped variants).
- Documented that generate() intentionally shares the broadened transient retry
(RAG sampling path), with the worst-case latency note.
- Added a docstring note to process_document on how the provider-level retry
(5x) layers over the outer loop (3x in-process / 1x procrastinate).
- Added test_embed_batch_retries_on_connection_error for the batch path.
- Renamed test_retry_reraises_non_rate_limit_immediately ->
test_retry_reraises_when_predicate_returns_false (it tests the predicate, not
a specific status).
SonarCloud:
- S5708 (BLOCKER) on the helper's dynamic `except exception_type`: the type is
constrained to BaseException/tuple by the signature; suppressed with a
justified NOSONAR.
- S7503 (async without await) in the embed-retry test: use AsyncMock side_effect
instead of a hand-rolled async function.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
From card 309 (OHR-Bench smoke-test triage): during a backend-pod rollover the
embedding endpoint was briefly unreachable, and openai.APIConnectionError /
ConnectError propagated unretried (the provider only retried 429). Documents
exhausted the 3 in-process retries and were dropped for that scan cycle.
Broaden the provider-level retry to the transient set -- APIConnectionError,
APITimeoutError, 429, and 5xx -- on the existing exponential backoff (2s->60s,
5 attempts), so a few seconds of retry rides through the rollover. Permanent
4xx (auth, bad request) still re-raise immediately. Generalize the shared
_retry helper (retry_on_rate_limit -> retry_on_transient, predicate renamed to
should_retry, accurate log label) with a back-compat alias; Mistral gets 429+5xx
for parity. The production gateway path inherits this via GatewayProvider, which
delegates to the decorated OpenAIProvider methods.
Add astrolabe_vector_ingest_dropped_total{reason}, incremented when a document
exhausts retries, classified (connection|timeout|rate_limit|server|qdrant|other)
by _drop_reason so the embed-drop rate is alertable per cause. Dropped docs are
NOT marked failed, so the next full scan re-picks them (re-queue via scan loop).
Refs: Deck board 12 card 309 (AC #1 no permanently-dropped docs; embed-drop
metric for AC #5).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Round-6 review: the docstrings listed preserved fields but omitted
assignedUsers. Verified empirically (Deck 1.15.9) that the update route's
board-change handling only remaps labels and leaves user assignments
untouched, so assignees carry over. Documented in both the client and MCP
tool docstrings, with the caveat that an assignee lacking access to the
target board stays assigned but cannot act on the card. Added
test_move_card_to_board_preserves_assigned_users to lock it in.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The move-card unit tests added a mock httpx.Request with an http:// URL,
which SonarCloud flags as a new security hotspot (insecure protocol),
failing the new-code quality gate. The URL is never dialed (it only labels
a synthetic HTTPStatusError), but switch it to https to keep the gate green.
Also simplify the done-PUT mock to a bare 200 response, since that response
is discarded by the implementation.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Round-4 review polish on PR #885:
- The post-move done re-mark is now best-effort: the move PUT has already
committed by then, so if the /done call (or its re-fetch) fails, log a
warning with the card's new location and return the moved card instead of
raising as if the whole move failed. Documented in the docstring.
- Note that duedate is sent explicitly as None (vs update_card omitting it) —
equivalent for this route.
- Add unit coverage for the swallowed done-restore failure, and an integration
test for a card that is both done and archived (exercises the done-restore
re-fetch on an archived card).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Round-3 review polish on PR #885:
- deck_move_card_to_board now captures the moved DeckCard and returns its
post-move label titles in CardOperationResponse.labels, so LLM clients can
confirm the cross-board label remap (the tool's headline behaviour) without
a follow-up deck_get_card. The field is optional and defaults to None for
the other card operations that share this response model.
- Tighten test_move_card_to_board_restores_done_state to assert the returned
card reflects the restored done state.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Round-2 review polish on PR #885:
- Document in the deck_move_card_to_board tool that the move reassigns the
card owner to the calling user and resets the done timestamp (both are
limitations of Deck's move route), so an LLM reading only the tool
description isn't misled about preserved fields.
- Fix the done integration-test docstring to say "done state (not timestamp)".
- Add test_move_card_to_board_preserves_archived_status to lock in the
documented archived-preservation behaviour.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Addresses the round-1 review on PR #885:
- Preserve `done` across a cross-board move. The internal card-update route
(the only one that works cross-board — the board/stack-scoped route 404s for
a card not already on that board) does not accept a done value, so a "done"
card is re-marked done after the move. Deck stamps the current time there, so
the original timestamp isn't preserved — documented as a route limitation.
(`archived` is already preserved: CardService only mutates it when sent.)
- Validate that target_stack_id is on target_board_id before moving, so the
parameter is load-bearing and a mismatch fails loudly instead of misreporting.
- Skip the same-board guard's get_stacks round-trip on a same-stack reorder.
- Add unit coverage (done-restore call, destination validation, same-stack
skip) and integration coverage (done preservation, target-board mismatch).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
deck_reorder_card only relocated a card between stacks on the same board.
Moving a card to another board now has a dedicated tool that goes through
Deck's card-update route (CardService::update), which remaps the card's
board-scoped labels to the destination board by title instead of leaving
orphaned labels behind. Card identity (id, comments, attachments) is
preserved.
reorder_card is now restricted to same-board moves: it rejects a
target_stack_id on another board (which Deck's reorder route would accept
but with orphaned labels), steering clients to deck_move_card_to_board.
Verified empirically against Deck 1.15.9: the reorder route leaves a moved
card carrying its source board's label (boardId mismatch); the update route
remaps it to the destination board's same-titled label.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- _default_mcp_server_url() replaces the hardcoded localhost:8000 fallback,
deriving the port from settings.port so a custom PORT is honoured and all
fallback sites share one source of truth (removes the footgun where
settings.port looked wired but the OAuth-audience fallback ignored it).
- Clear _readiness_cache.statuses at loop start so dependency entries from a
prior lifespan run in the same process (integration matrix) don't linger as
stale, confusing checks output.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the blanket tg.cancel_scope.cancel() with a per-task CancelScope: the
readiness loop reports its scope via task_status, and the lifespan cancels just
that scope at shutdown. The task group's exit then waits for the scanner/
processor tasks to drain on their shutdown_event instead of force-cancelling
them mid-work, restoring the graceful drain the pre-refactor code had.
Also use docstrings instead of bare `return` in the no-op mode closures.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
_readiness_refresh_loop is started with tg.start_soon and loops forever with no
shutdown_event check. anyio waits for start_soon tasks on normal task-group exit
rather than cancelling them, so graceful shutdown hung until uvicorn's timeout.
Cancel the task group's scope after teardown() to stop the loop and any
stragglers, mirroring _maybe_login_flow_cleanup.
Also document the cache ttl_seconds startup-override and the inclusive is_stale
boundary.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- _DEFAULTS keys for NEXTCLOUD_OIDC_TOKEN_TYPE / NEXTCLOUD_OIDC_SCOPES were
registered as oidc_* (uppercasing to OIDC_*), so dynaconf
(ignore_unknown_envvars) never read the NEXTCLOUD_-prefixed env vars and the
fields stayed at their defaults. Prefix the keys to match _field_map; add a
regression test.
- Add gte=1 validator for HEALTH_READY_REFRESH_INTERVAL and a 1..65535 range
validator for PORT.
- Tie ReadinessCache.ttl_seconds to 2x the configured refresh interval so
is_stale() stays meaningful when the interval is tuned.
- Raise the refresh-loop exception log from DEBUG to WARNING.
- Make health_ready a sync handler (no awaits); dedupe the localhost fallback
into _DEFAULT_MCP_SERVER_URL; use pytest.approx for the float default.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Round-4 review (real bug): get_background_sync_status now returns provisioned_at
as Unix seconds (the wire/pact value), but ProvisioningStatus.provisioned_at is
str | None (ISO). Constructing it for a provisioned user raised a Pydantic
ValidationError — a path that was unreachable before the has_access fix.
Convert int -> ISO at the oauth_tools boundary (mirroring the existing
refresh_token branch), keeping the model schema and the int-asserting contract
pact/unit tests intact. Add a regression test that drives the full
_get_provisioning_status round-trip with an integer timestamp.
Also surface dropped provider-state params in the verifier's _dispatch_state
no-op branch (round-4 nit).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Fixes MCP reconnect timeouts on tenant servers (Deck #302). Three changes:
- /health/ready now gates only on local config. Nextcloud/Qdrant health is
refreshed by a background loop, cached, and reported but NON-gating, so a
single-replica tenant Pod is no longer pulled from its Service on a transient
dependency blip (which dropped every MCP streamable-HTTP session and caused
reconnect timeouts). The probe path performs no external I/O.
- Refactor starlette_lifespan: collapse the four near-identical per-mode
task-group + session + yield + teardown skeletons into one shared task group
that also runs the readiness refresh loop; each mode contributes a
(start, teardown) pair. eviction_task_group is now always present.
- Migrate app.py off os.getenv: all config is read through dynaconf Settings
(adds health_ready_refresh_interval, oidc_token_type, oidc_scopes, port).
Inline/dynamic defaults preserved at each call site.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The `env` context is not available in a job-level `if:` (only `github`/`needs`/
`vars`/`inputs` are), so `if: ... && env.PACT_BROKER != ''` on the job was an
invalid-context error that failed the whole workflow to parse. Move the broker
guard onto each step (matching the consumer/provider jobs) and keep the job
`if` on the master-branch check only.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- pact.yml: guard `can-i-deploy` job on `env.PACT_BROKER != ''` so a secret
rotation/fork can't break every master merge (the CLI errors on empty URL)
- pact.yml: pin install.sh to the v2.6.1 commit SHA (immune to tag force-push)
- astrolabe_client.py: `_token_cache` Optional[dict] -> `dict | None` and drop
the now-unused `Optional` import (CLAUDE.md union syntax)
- add tests/unit/test_astrolabe_client.py: mocked unit coverage for
get_background_sync_status field mapping (200 provisioned / 200 not-provisioned
/ 404) — the layer that would have caught the original silent app_password bug
- consumer pact test: note the 404 branch is internal defensive handling (covered
by the unit test), not a contract obligation
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- pact.yml: pin tailscale/github-action@v3 to commit SHA (3 jobs) and
pact-ruby-standalone install.sh to v2.6.1 (2 jobs) — supply-chain hardening
- pact.yml: drop redundant `-o "addopts=..."` override (pyproject.toml already
sets the same addopts; the override would silently mask future additions)
- test_mcp_provider_verification.py: remove dead `pytestmark` shadowed by the
list assignment; gate the module skip on PACT_USERNAME/PACT_PASSWORD too so a
broker-set-but-creds-missing CI skips cleanly instead of raising KeyError
- conftest.py: drop the unused `pact_dir` fixture
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Introduce consumer-driven contract testing between nextcloud-mcp-server and the
astrolabe Nextcloud app, published to the homelab Pact Broker and verified in CI.
- pact-python dev dep + `contract` pytest marker
- tests/contract/test_astrolabe_credentials_consumer.py: consumer pact for the
background-sync *status* call (provisioned -> has_background_access:true,
sync_type:"app_password", integer provisioned_at; unprovisioned -> false/null)
- tests/contract/test_mcp_provider_verification.py: env-gated Verifier harness
for this server's /api/v1/* provider role (provider-state handlers stubbed
pending astrolabe's published pacts)
- .github/workflows/pact.yml: join tailnet -> publish pacts -> provider verify
-> can-i-deploy; broker steps skip when PACT_BROKER is unset (forks)
- docs/ADR-029-pact-contract-testing.md
Fix astrolabe_client.get_background_sync_status: it previously read a
non-existent `app_password` field (always reporting no-access). Rewrite it to
read the real status contract (has_background_access / sync_type /
provisioned_at) and drop the unsatisfiable get_user_app_password.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Round-2 review nit (PR #879): lock the intentional record ordering
(tokens_embedded before the conditional pages_embedded) with an
assertion in test_parsed_file_records_pages_and_tokens, so a refactor
that reverses it fails a test rather than only contradicting a comment.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Round-1 review follow-ups (PR #879):
- Gate pages_embedded on `page_count and page_count > 0` so a malformed
negative count meters as "no pages" rather than emitting a negative
billing row (matches the documented call-site intent).
- Exclude bool at the call-site narrowing (`isinstance(int) and not
isinstance(bool)`) — bool is an int subclass, so a stray page_count=True
would otherwise record pages=1.
- Document chunk_count's role (empty-batch no-op guard) and the
intentional tokens-before-pages ordering in the docstring/comments.
- Add test_negative_pages_skips_pages.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`pages_embedded` carried an interim chunk count (`len(chunk_texts)`,
TODO #282). Reframe it as a charge for *parsing* (PDF page extraction /
OCR) rather than a normalized content size:
- Parsed files (PDFs) record `pages_embedded` = real `page_count` from
the document processor metadata.
- Text content (notes, deck cards, news items) is never parsed, carries
no `page_count`, and records no `pages_embedded` row — only
`tokens_embedded`. There is deliberately no chars/tokens-per-page
constant; pages map 1:1 to parsed document pages.
`record_indexing_usage` now takes `page_count` and records the two
dimensions independently, gating `pages_embedded` on a truthy page count
(not the doc_type) so a future non-PDF parsed type stays correct. Stays
flag-gated + best-effort. Tests cover parsed-file, text-only, and
zero-page cases.
Deck #282 (board 8). Billing-model ADR corrected in
astrolabe-cloud-website docs/control-plane/usage-metering.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Addresses round-1 review on #878:
- Move the eager `document_processors` imports out of the API startup graph:
`app.py` (get_registry now imported inside initialize_document_processors,
after the disabled early-return) and `vector/processor.py` (get_registry now
imported at its single use site). Importing `app` + `cli` no longer loads
`document_processors` / `_isolation` at all -- the #877 stack is fully out of
startup (pymupdf still loads via search/pdf_highlighter, a Windows-compatible
and separately-tracked concern).
- Make `tests/unit/test_pdf_parse_isolation.py` importable on Windows: guard the
top-level `import resource` with try/except and skip the three rlimit
computation tests via a `requires_resource` marker when the module is absent.
The Windows no-op / import-guard tests don't use the real module and still run.
- Fix the `# pragma: no cover` comment on the win32 branch to be accurate.
- Add `enable-cache: true` to the package-smoke setup-uv step.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`document_processors/_isolation.py` did an unconditional module-level
`import resource`, a POSIX-only stdlib module absent on Windows. It was
pulled into the API startup path via
`server/webdav.py -> utils/document_parser -> document_processors`, so
the MCP server failed to start on Windows since 0.101.2 with
`ModuleNotFoundError: No module named 'resource'`.
- Guard the import behind `sys.platform`; bind `resource = None` on
win32. `_apply_mem_limit()` degrades to a logged no-op when the module
is unavailable (the RLIMIT_AS cap is a Linux-pod safety measure, not a
correctness requirement).
- Make the document-parser import in `server/webdav.py` lazy so server
startup never loads the ingest document stack
(document_processors -> pymupdf -> _isolation) at all -- it is only
needed when a file is actually read and parsed. This both fixes#877
and decouples the API layer from ingest-only deps.
- Add unit regressions for the no-op path and the win32 import guard.
- Add a cross-platform `package-smoke` CI job (ubuntu + windows) that
installs the package isolated and runs the CLI, exercising the
cli -> server -> webdav import chain that crashed in #877.
Fixes#877
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Round-8 claude-review (no blockers; comment-only):
- 🟡 Documented that Ollama's /api/embed prompt_eval_count is assumed
batch-level total and is unverified against a live instance (Ollama isn't the
Cloud billing provider); if it proves last-item-only, switch to per-item
summing. The char estimate already covers versions that omit the field.
- 🟡 Noted on the astrolabe_embedding_tokens_total counter that operation="query"
is recorded pre-Qdrant, so it can legitimately exceed the billing-store
tokens_embedded aggregate when a search fails post-embed — dashboards
shouldn't alert on that healthy gap.
Deferred (reviewer: "minor nit, acceptable"): record_indexing_usage awaited in
the task group — the group awaits all child tasks regardless, the write is
best-effort + fast, and start_soon would need the tg threaded into the closure
for marginal gain.
Deck #284.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
