feat: backend-agnostic vector-sync gauges (pending/documents/chunks)

The only queue metric, mcp_vector_sync_queue_size, was updated inline by the
single-user consumer (processor_task) but never by the multi-user consumer
(oauth_processor_task). On multi-user tenants (e.g. blackbox-demo, 5 users) the
gauge read 0 for 24h while the live anyio buffer held ~2214 pending documents
(shown by /api/v1/vector-sync/status). The "indexed" figure was also a chunk
count (16039 points ≈ 480 docs) mislabelled as documents.

Publish a consumer-independent snapshot from a periodic task
(vector/metrics_publisher.vector_sync_metrics_task), spawned in BOTH lifespan
task groups (single-user and multi-user) and every queue backend:
- mcp_vector_sync_pending_documents — outstanding work via
  ingest_status.get_ingest_pending() (anyio buffer depth or procrastinate
  todo+doing); also keeps the legacy queue_size gauge meaningful on all paths.
- mcp_vector_sync_indexed_documents — distinct documents, counted exactly and
  cheaply via the one chunk_index=0 point per document (no facet).
- mcp_vector_sync_indexed_chunks — total non-placeholder points.

The /api/v1/vector-sync/status endpoint now returns indexed_documents (distinct
docs) AND indexed_chunks separately, so documents and chunks are no longer
conflated. The publisher uses approximate Qdrant counts (every-N-seconds gauge);
the on-demand endpoint counts exactly. New knob:
VECTOR_SYNC_METRICS_REFRESH_INTERVAL (default 20s). Fail-safe: a metrics refresh
never disturbs ingest.

BREAKING CHANGE: /api/v1/vector-sync/status field `indexed_documents` now holds
the distinct-document count (was the chunk count); the chunk count moved to the
new `indexed_chunks` field. The Astrolabe UI + the nc_get_vector_sync_status MCP
tool / userinfo page are harmonized in a follow-up (Deck #195).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

nextcloud_mcp_server/api/management.py

@@ -20,13 +20,12 @@ import time
 from importlib.metadata import version
 from typing import Any
 
-from qdrant_client.models import Filter
 from starlette.requests import Request
 from starlette.responses import JSONResponse
 
 from nextcloud_mcp_server.config import get_settings
 from nextcloud_mcp_server.config_validators import AuthMode, detect_auth_mode
-from nextcloud_mcp_server.vector.placeholder import get_placeholder_filter
+from nextcloud_mcp_server.vector.metrics_publisher import count_indexed
 from nextcloud_mcp_server.vector.qdrant_client import get_qdrant_client
 
 logger = logging.getLogger(__name__)
@@ -307,28 +306,30 @@ async def get_vector_sync_status(request: Request) -> JSONResponse:
             ingest_queue=settings.ingest_queue,
         )
 
-        # Get Qdrant client and query indexed count (backend-independent)
-        indexed_count = 0
+        # Corpus size (backend-independent): distinct documents AND total
+        # chunks. A single "indexed" figure is ambiguous because each document
+        # fans out to ~N chunks, so both are reported (the UI shows both).
+        indexed_documents = 0
+        indexed_chunks = 0
         try:
             qdrant_client = await get_qdrant_client()
-
-            # Count documents in collection, excluding placeholders
-            count_result = await qdrant_client.count(
-                collection_name=settings.get_collection_name(),
-                count_filter=Filter(must=[get_placeholder_filter()]),
+            indexed_documents, indexed_chunks = await count_indexed(
+                qdrant_client, settings.get_collection_name()
             )
-            indexed_count = count_result.count
-
         except Exception as e:
-            logger.warning("Failed to query Qdrant for indexed count: %s", e)
-            # Continue with indexed_count = 0
+            logger.warning("Failed to query Qdrant for indexed counts: %s", e)
+            # Continue with zeroed counts
 
         # Determine status
         status = "syncing" if pending.pending > 0 else "idle"
 
         body: dict[str, object] = {
             "status": status,
-            "indexed_documents": indexed_count,
+            # indexed_documents is now the distinct-document count (was the chunk
+            # count before — the two differ by the per-document chunk fan-out).
+            # indexed_chunks exposes the raw point count separately.
+            "indexed_documents": indexed_documents,
+            "indexed_chunks": indexed_chunks,
             "pending_documents": pending.pending,
             "ingest_queue": settings.ingest_queue,
         }