fix: make procrastinate ingest queue opt-in (default to in-process anyio)

An unset INGEST_QUEUE auto-derived "postgres" whenever DATABASE_URL was
PostgreSQL, silently starting the procrastinate ingest worker (schema
migration, reclaim cron, deferred jobs) on every Postgres-backed tenant —
even though none had opted into the api/worker split. Observed on
tenant-blackbox-demo (:0.98.0): ~600 "Deferred 1 job" log lines / 24h.

Resolve an unset INGEST_QUEUE to "memory" (the in-process anyio queue)
regardless of the database backend. procrastinate is now strictly opt-in
via an explicit INGEST_QUEUE=postgres; the existing guard still rejects
postgres against a SQLite DATABASE_URL. Docs + unit test updated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Chris Coutinho
2026-06-04 01:59:24 +02:00
co-authored by Claude Opus 4.8
parent 1e615c2bf1
commit ad211ee2da
3 changed files with 35 additions and 15 deletions
+13 -2
View File
@@ -59,13 +59,24 @@ class TestIngestQueueResolution:
with pytest.raises(ValueError, match="INGEST_QUEUE=postgres requires"):
Settings(ingest_queue="postgres")
def test_auto_postgres_when_database_url_is_postgres(self, monkeypatch):
def test_memory_default_even_on_postgres_url(self, monkeypatch):
# Procrastinate is opt-in: a Postgres DATABASE_URL with INGEST_QUEUE
# unset must NOT silently enable procrastinate. Default → memory.
monkeypatch.setattr(
config_module,
"get_database_url",
lambda: "postgresql+asyncpg://mcp:mcp@db/mcp",
)
assert Settings().ingest_queue == "postgres"
assert Settings().ingest_queue == "memory"
def test_explicit_postgres_on_postgres_url(self, monkeypatch):
# Opting in explicitly against a Postgres URL is the supported path.
monkeypatch.setattr(
config_module,
"get_database_url",
lambda: "postgresql+asyncpg://mcp:mcp@db/mcp",
)
assert Settings(ingest_queue="postgres").ingest_queue == "postgres"
def test_explicit_memory_on_postgres_url(self, monkeypatch):
monkeypatch.setattr(