fix(vector-sync): sweep placeholder orphans at Pod startup (#101)

When the per-tenant nextcloud-mcp-server Pod OOMKills mid-batch, the
in-memory anyio processor queue is lost but the placeholder Qdrant
points (is_placeholder=true, status=pending) survive. The next Pod's
scanner re-runs, sees the existing placeholders, applies the
5 × VECTOR_SYNC_SCAN_INTERVAL staleness gate (~5h with the deployed
1h scan interval), and skips them. Result: 0 documents indexed for
the duration of the gate after every restart.

Stamps a process-level instance_id (UUID per Pod-process) onto every
placeholder write. A new sweep_orphan_placeholders helper, called
once from starlette_lifespan after the Qdrant client is initialised
and before the scanner / user-manager spawns, scrolls the collection
and deletes any placeholder whose instance_id doesn't match the
current Pod's (including placeholders with no instance_id field —
back-compat for pre-fix Pod versions). The scanner's next cycle
naturally re-creates fresh placeholders and queues work normally;
no DocumentTask reconstruction needed.

Sweep is one-shot at startup, not periodic — the existing staleness
gate still covers same-Pod recovery, and the cross-Pod-restart gap
was the only failure mode. Failure is non-fatal (logged via
vector_sync.orphan_sweep_failed) so a transient Qdrant hiccup at
boot doesn't prevent the scanner from running.

Both lifespan branches (single-user BasicAuth, OAuth / multi-user
BasicAuth) call the sweep via a module-local helper. A new
VECTOR_SYNC_ORPHAN_SWEEP_ENABLED setting (default True) provides
an escape hatch.

Closes Deck #101.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Chris Coutinho
2026-05-22 21:03:23 +02:00
co-authored by Claude Opus 4.7
parent 32fc03cf67
commit a5cbe91b29
4 changed files with 337 additions and 0 deletions
+37
View File
@@ -129,6 +129,7 @@ from nextcloud_mcp_server.vector.oauth_sync import (
oauth_processor_task,
user_manager_task,
)
from nextcloud_mcp_server.vector.placeholder import sweep_orphan_placeholders
from nextcloud_mcp_server.vector.processor import processor_task
from nextcloud_mcp_server.vector.qdrant_client import get_qdrant_client
from nextcloud_mcp_server.vector.scanner import scanner_task
@@ -1437,6 +1438,34 @@ def get_app(transport: str = "streamable-http", enabled_apps: list[str] | None =
await stack.enter_async_context(_maybe_login_flow_cleanup(app))
yield
async def _sweep_orphan_placeholders_if_enabled() -> None:
"""One-shot Pod-startup sweep of cross-restart placeholder orphans.
See ``vector.placeholder.sweep_orphan_placeholders`` and Deck
card #101. Both lifespan branches (single-user BasicAuth and
OAuth / multi-user BasicAuth) call this after the qdrant
client is initialised and before the scanner / user-manager
tasks spawn. Failures are non-fatal — the existing staleness
gate will eventually re-queue orphans on the slow ~5h path.
"""
if not settings.vector_sync_orphan_sweep_enabled:
return
try:
qdrant_client = await get_qdrant_client()
swept, kept = await sweep_orphan_placeholders(
qdrant_client, settings.qdrant_collection
)
logger.info(
"vector_sync.orphan_sweep",
extra={
"swept": swept,
"kept": kept,
"collection": settings.qdrant_collection,
},
)
except Exception:
logger.exception("vector_sync.orphan_sweep_failed")
@asynccontextmanager
async def starlette_lifespan(app: Starlette):
# Set OAuth context for OAuth login routes (ADR-004)
@@ -1612,6 +1641,9 @@ def get_app(transport: str = "streamable-http", enabled_apps: list[str] | None =
f"Cannot start vector sync - Qdrant initialization failed: {e}"
) from e
# Orphan-sweep before scanner starts — card #101.
await _sweep_orphan_placeholders_if_enabled()
# Initialize shared state
send_stream, receive_stream = anyio.create_memory_object_stream(
max_buffer_size=settings.vector_sync_queue_max_size
@@ -1774,6 +1806,11 @@ def get_app(transport: str = "streamable-http", enabled_apps: list[str] | None =
f"Cannot start vector sync - Qdrant initialization failed: {e}"
) from e
# Orphan-sweep before scanners spawn — card #101. Runs once
# across the shared (per-tenant) collection regardless of
# how many per-user scanners the user-manager later starts.
await _sweep_orphan_placeholders_if_enabled()
# Clean up stale app passwords at startup (BasicAuth mode only)
if not oauth_enabled:
try: