fix(vector-sync): address round-5 review — partial-failure signal, markers

- purge route: include a "failed" key in the 200 body listing requested doc
  types that were not purged, so Astrolabe knows consent isn't yet enforced
  for them (scanner backstop still catches up)
- tests: add @pytest.mark.unit / module-level pytestmark to the new test
  modules so they run under `pytest -m unit`; add a partial-failure route test
- capabilities: comment why the cache is keyed per-user despite a global value
- semantic/scanner: doc/comment clarifications (sorted-order, eviction timing)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Chris Coutinho
2026-06-16 01:47:44 +02:00
co-authored by Claude Opus 4.8
parent b0751102d7
commit d0db530ac9
10 changed files with 64 additions and 7 deletions
+14 -2
View File
@@ -117,8 +117,20 @@ async def purge_doc_types_route(request: Request) -> JSONResponse:
return JSONResponse({"purged": {}})
purged = await purge_doc_types(doc_types)
logger.info("Vector-sync purge by admin %s: %s", user_id, purged)
return JSONResponse({"purged": purged})
# Surface a partial-failure signal so Astrolabe knows which types were
# NOT purged (consent not yet enforced for them) — the scanner backstop
# still catches these, but the caller shouldn't assume full success.
failed = [dt for dt in dict.fromkeys(doc_types) if dt not in purged]
body: dict = {"purged": purged}
if failed:
body["failed"] = failed
logger.info(
"Vector-sync purge by admin %s: purged=%s failed=%s",
user_id,
purged,
failed,
)
return JSONResponse(body)
except ProvisioningRequiredError as e:
logger.info("Provisioning required for user %s: %s", user_id, e)
+6
View File
@@ -28,6 +28,12 @@ logger = logging.getLogger(__name__)
# changes rarely, but search/scan paths consult it frequently, so trade a little
# staleness for keeping the OCS round-trip off the hot path. Mirrors the
# list_accessible_owners cache in search/access_filter.py.
#
# Keyed by user_id even though enabled_doc_types is an admin-wide value: the OCS
# call is authenticated per-user (and ``installed`` resolves per-user on the
# Astrolabe side), so we cache per-user for correctness. The redundancy is
# bounded by _CACHE_MAXSIZE; on an admin change all entries reconverge within
# one TTL window.
_CACHE_TTL_SECONDS = 30.0
_CACHE_MAXSIZE = 1024
# user_id -> (monotonic_ts, frozenset[doc_type] | None). None = no restriction.
+6 -4
View File
@@ -64,10 +64,12 @@ def _consent_narrowed_doc_types(
Caller has already established ``allowed is not None`` (a concrete allow-set;
``None`` means "no restriction" and is handled by skipping this call). When
no explicit ``doc_types`` are requested, restrict to the full allow-set;
otherwise intersect (preserving the caller's order). An empty result means
nothing the caller asked for is admin-approved — the caller short-circuits
to an empty response rather than falling through to an all-types search.
no explicit ``doc_types`` are requested, restrict to the full allow-set
(returned ``sorted`` for determinism only — order is a filter, not a ranking
hint); otherwise intersect (preserving the caller's order). An empty result
means nothing the caller asked for is admin-approved — the caller
short-circuits to an empty response rather than falling through to an
all-types search.
"""
if doc_types is None:
return sorted(allowed)
+4 -1
View File
@@ -377,7 +377,10 @@ async def _enqueue_deletes_for_disabled_types(
if len(_consent_backstop_done) >= _CONSENT_BACKSTOP_MAX:
# Evict oldest-first down to half capacity (insertion-ordered dict),
# so overflow re-fires the backstop for only the oldest markers
# rather than the whole fleet at once.
# rather than the whole fleet at once. Placed inside the per-doc_type
# loop: markers added earlier in *this* call are the newest, so they
# survive eviction; only genuinely old entries are dropped (and a
# re-fire is idempotent regardless).
overage = len(_consent_backstop_done) - _CONSENT_BACKSTOP_MAX // 2
logger.info(
"consent backstop tracking hit %d entries; evicting %d oldest",