fix(embedding): instantiate BM25 singleton off the event loop

The ``BM25SparseEmbeddingProvider.__init__`` calls
``fastembed.SparseTextEmbedding(model_name="Qdrant/bm25")`` which
downloads ~50 MB of model weights from HuggingFace and loads them
into memory — observed >5 s wall-clock in production. The inference
methods (``encode_async``, ``encode_batch_async``) already wrap work
in ``anyio.to_thread.run_sync``, so the design intent is clearly to
keep FastEmbed off the event loop. That protection just didn't
cover the constructor.

Symptom in the Astrolabe Cloud per-tenant deploy (deck #102 smoke):
~30–90 s after a user enables semantic search, the pod tips into a
SIGKILL-restart cycle. Loki shows a single log line

  Initializing BM25 sparse embedding provider: Qdrant/bm25

followed by nothing else from the event loop until exitCode 137.
Kubernetes ``/health/live`` httpGet probe timeout=5s fires 6 times
in a row, kubelet kills the container, restart, repeat.

Fix: switch ``get_bm25_service()`` to an async accessor that wraps
the first-time construction in ``anyio.to_thread.run_sync``. Two
existing call sites (``vector/processor.py:603``,
``search/bm25_hybrid.py:123``) update to ``await``. Both are
already inside async functions so the await is free.

New unit test pins the invariant by monkey-patching
``BM25SparseEmbeddingProvider.__init__`` with ``time.sleep(1)`` and
asserting a concurrent ``anyio.sleep(0.05)`` finishes promptly —
the test fails if the constructor ever runs back on the event loop.

Same pattern exists in ``OllamaEmbeddingProvider.__init__`` (sync
``httpx.get`` health-check). Ollama isn't enabled in any current
deploy; filed as a follow-up.

Refs:
- Astrolabe Cloud deck card #102 (smoke discovery)
- Sibling fix #799 (NullPool for cross-loop-asyncpg, same class
  of "anyio bites you in production" bug)

Verified:
- ``uv run pytest tests/unit/`` — 1027 passed
- ``uv run ruff check`` clean on touched files
- ``uv run ty check`` clean on touched files
- New tests pass

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Chris Coutinho
2026-05-17 19:33:24 +02:00
co-authored by Claude Opus 4.7
parent 63c1ff0bb4
commit e0b7afb4b1
4 changed files with 139 additions and 4 deletions
+29 -2
View File
@@ -6,6 +6,8 @@ New code should use nextcloud_mcp_server.providers.get_provider() directly.
import logging
import anyio
from nextcloud_mcp_server.providers import get_provider
from .bm25_provider import BM25SparseEmbeddingProvider
@@ -89,14 +91,39 @@ def get_embedding_service() -> EmbeddingService:
_bm25_service: BM25SparseEmbeddingProvider | None = None
def get_bm25_service() -> BM25SparseEmbeddingProvider:
async def get_bm25_service() -> BM25SparseEmbeddingProvider:
"""
Get singleton BM25 sparse embedding service instance.
Lazily instantiates the singleton off the event loop. The
``BM25SparseEmbeddingProvider`` constructor calls
``fastembed.SparseTextEmbedding(model_name="Qdrant/bm25")`` which
downloads ~50 MB of model weights from HuggingFace and loads them
into memory — observed >5 s wall-clock in production, enough to
stall the calling thread. The encode methods on the provider
already offload via ``anyio.to_thread.run_sync``; this routes the
first-time init through the same path so the event loop stays
responsive (kubernetes ``/health/live`` httpGet probe in
particular).
The singleton is process-wide so the per-pod cost is paid once.
Subsequent calls hit the warm path and return after a single
non-blocking await.
Concurrent first callers race: two coroutines can both observe
``_bm25_service is None`` and both enter ``run_sync``. We accept
the duplicate model load over an ``asyncio.Lock``, which has its
own cross-loop hazards under anyio TaskGroups (see PR #799 for
that class of bug). The duplicate is bounded — FastEmbed caches
the downloaded weights on disk after the first call, so loser(s)
pick up cheaply.
Returns:
Global BM25SparseEmbeddingProvider instance
"""
global _bm25_service
if _bm25_service is None:
_bm25_service = BM25SparseEmbeddingProvider()
_bm25_service = await anyio.to_thread.run_sync( # type: ignore[attr-defined] # ty: ignore[unresolved-attribute]
BM25SparseEmbeddingProvider
)
return _bm25_service