feat(embedding): gateway provider discovers dimension via GET /v1/models

External-mode tenant pods CrashLoop at startup: Qdrant collection init calls
get_dimension() before any embed(), but GatewayProvider only learns its
dimension lazily after the first embed, and the gateway model isn't an OpenAI
model so the base class can't know it statically.

- Add GatewayProvider._detect_dimension() — the async startup hook the
  vector-sync bootstrap already invokes (vector/qdrant_client.py:
  hasattr(provider, "_detect_dimension")) for Ollama. It GETs the gateway's
  GET /v1/models and sets _dimension from the entry whose id matches the
  configured model. Best-effort: any failure (old gateway, model absent,
  network) leaves _dimension unset so the inherited lazy detect-on-first-embed
  still applies — never fatal. Presents the M2M bearer when configured.
- Switch the default embedding_gateway_model to the gateway's provider-
  namespaced id "mistral/mistral-embed" (the gateway routes on the "/"-prefix
  and sends "mistral-embed" upstream); collapse a duplicated config field.

Pairs with astrolabe-cloud-website#229 (gateway /v1/models, namespaced ids).

Tests: discovery sets dim w/o embed, sends bearer, non-fatal on
404/absent/error, skips when already known.
This commit is contained in:
Chris Coutinho
2026-05-31 23:43:19 +02:00
parent c662d57c4f
commit d84a95842f
3 changed files with 195 additions and 5 deletions
@@ -147,6 +147,66 @@ class GatewayProvider(OpenAIProvider):
if self._token_provider is not None:
self.client.api_key = await self._token_provider.get_token()
async def _detect_dimension(self) -> None:
"""Resolve the embedding dimension from the gateway's ``GET /v1/models``
before the first embed.
Qdrant collection init needs the vector size at startup (it calls
``get_dimension()`` before any ``embed()``); the vector-sync bootstrap
invokes this hook first (``vector/qdrant_client.py`` —
``hasattr(provider, "_detect_dimension")``). The gateway is the
authority on the dimensions of the models it serves, so we read it from
there rather than hardcoding (``mistral-embed`` isn't an OpenAI model,
so the OpenAI-wire base class can't know its size statically).
Best-effort: any failure (old gateway without /v1/models, model absent,
network) leaves ``_dimension`` unset so the inherited lazy
detect-on-first-embed path still applies. Never raises.
"""
if self._dimension is not None:
return # already known (e.g. an OpenAI model in the static map)
# ``models`` is a sibling of ``embeddings`` under the gateway's base —
# derive it from the same base_url the OpenAI client uses for embeds so
# the two stay consistent (str(base_url) has a trailing slash).
models_url = str(self.client.base_url).rstrip("/") + "/models"
headers: dict[str, str] = {}
if self._token_provider is not None:
headers["Authorization"] = (
f"Bearer {await self._token_provider.get_token()}"
)
try:
async with httpx.AsyncClient(
timeout=httpx.Timeout(10.0, connect=5.0)
) as client:
resp = await client.get(models_url, headers=headers)
resp.raise_for_status()
catalog = resp.json().get("data", [])
for entry in catalog:
if entry.get("id") == self.embedding_model:
dim = entry.get("dimension")
if isinstance(dim, int):
self._dimension = dim
logger.info(
"Resolved embedding dimension %d for model %s via "
"gateway /v1/models",
dim,
self.embedding_model,
)
return
logger.warning(
"Gateway /v1/models reported no dimension for model %s; "
"falling back to lazy detection on first embed",
self.embedding_model,
)
except Exception as exc: # noqa: BLE001 - best-effort, never fatal
logger.warning(
"Could not fetch model dimensions from gateway %s: %s; "
"falling back to lazy detection on first embed",
models_url,
exc,
)
async def embed(self, text: str) -> list[float]:
await self._ensure_bearer()
return await super().embed(text)