refactor: address PR #851 review round 1 (ingest transport)
- Add IngestTransport.active_consumer_count (0 by default; LocalTransport stores the started count) so app.py logs the worker count without re-checking INGEST_QUEUE — the last backend-knowledge leak in the lifespan is gone. - Document that DistributedTransport is postgres/procrastinate-specific by design (aclose() calls ProcrastinateTaskProducer.drain()); other distributed backends would be separate IngestTransport subclasses. - Clarify the _wire_vector_sync_state log line (writes app.state + singleton, not only the singleton). - Strengthen the LocalTransport test: assert active_consumer_count transitions 0→N and that each worker receives a distinct cloned receive stream. Refs: Deck #196 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
e7bcdb1950
commit
655d608fb7
@@ -377,7 +377,7 @@ def _wire_vector_sync_state(
|
||||
# app.state (Starlette) + the module singleton share the same attribute names.
|
||||
_apply(app.state)
|
||||
_apply(_vector_sync_state)
|
||||
logger.info("Vector sync state stored in module singleton")
|
||||
logger.info("Vector sync state published (app.state + module singleton)")
|
||||
|
||||
# Also share with the mounted /app browser sub-app, if present.
|
||||
for route in app.routes:
|
||||
@@ -1763,9 +1763,7 @@ def get_app(transport: str = "streamable-http", enabled_apps: list[str] | None =
|
||||
|
||||
logger.info(
|
||||
"Background sync tasks started: 1 scanner + %s processors (queue=%s)",
|
||||
0
|
||||
if settings.ingest_queue == "postgres"
|
||||
else settings.vector_sync_processor_workers,
|
||||
transport.active_consumer_count,
|
||||
settings.ingest_queue,
|
||||
)
|
||||
|
||||
@@ -1969,9 +1967,7 @@ def get_app(transport: str = "streamable-http", enabled_apps: list[str] | None =
|
||||
|
||||
logger.info(
|
||||
"Background sync tasks started: 1 user manager + %s processors (queue=%s)",
|
||||
0
|
||||
if settings.ingest_queue == "postgres"
|
||||
else settings.vector_sync_processor_workers,
|
||||
transport.active_consumer_count,
|
||||
settings.ingest_queue,
|
||||
)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user