fix: address PR #836 round-3 review (lock-key invariant, single open)

🟡 Document the _doc_queueing_lock ":" delimiter invariant (user_id and the
   controlled doc_type enum are colon-free, so the key is collision-safe; a
   future doc_type with ":" must not be added).
🟡 API pod no longer opens the procrastinate connector twice on startup: add
   ProcrastinateTaskProducer.ensure_schema() (applies the schema on the
   already-open pool) and have both lifespan branches build the producer then
   ensure_schema — one open/close cycle, matching the worker. build_producer now
   returns the concrete producer type.
🟢 Document in ports.py that a long-lived-connection producer may optionally
   provide drain() (lifespan probes via getattr).
🟢 Add a unit test that a non-credential pipeline error propagates (for
   procrastinate's RetryStrategy) and still closes the client via finally.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Chris Coutinho
2026-06-03 15:32:16 +02:00
co-authored by Claude Opus 4.8
parent 820b98dac1
commit b10ce15032
5 changed files with 75 additions and 25 deletions
+10 -5
View File
@@ -12,19 +12,24 @@ The transport is selected from ``INGEST_QUEUE``:
from __future__ import annotations
import logging
from typing import TYPE_CHECKING
from ...config import Settings
from .ports import TaskProducer
if TYPE_CHECKING:
from .procrastinate import ProcrastinateTaskProducer
logger = logging.getLogger(__name__)
async def build_producer(settings: Settings) -> TaskProducer:
async def build_producer(settings: Settings) -> ProcrastinateTaskProducer:
"""Build the Postgres (procrastinate) ingest producer.
Precondition: ``settings.ingest_queue == "postgres"`` (the memory transport
is constructed inline by the lifespan because it needs the paired receive
stream for the in-process processor pool).
Returns the concrete :class:`ProcrastinateTaskProducer` (not just the
``TaskProducer`` protocol) so the lifespan can call ``ensure_schema()`` on
the open connector. Precondition: ``settings.ingest_queue == "postgres"``
(the memory transport is constructed inline by the lifespan because it needs
the paired receive stream for the in-process processor pool).
"""
if settings.ingest_queue != "postgres":
raise ValueError(