Addresses three important findings from the latest reviewer comment:
- Add progress INFO log every 20 scroll batches (≈5120 points at
batch_size=256) in _backfill_doc_id_to_string so a long-running
migration on a large collection (50k+ points) doesn't look like a
startup hang. The line carries collection name, scanned count, and
rewritten count so it doubles as a heartbeat.
- Track non-400 failures in _ensure_keyword_payload_indexes and emit
a WARNING summary line listing every field that failed to get an
index. Per-field ERROR lines are easy to miss in startup noise; the
summary makes the partial-failure state visible at a glance.
- Split the sentinel upsert out of the data-scroll try/except in
_backfill_doc_id_to_string. A scroll-time failure still logs ERROR
with the new "scroll failed" wording (data is incomplete). A
sentinel-write failure now logs WARNING with "data succeeded but
sentinel write failed" wording — data is correct, only the
short-circuit marker is missing, and the next restart re-scrolls
an already-clean collection (idempotent zero-write) before retrying
the upsert.
Also fix the RuntimeWarning emitted by
test_backfill_logs_and_returns_when_scroll_raises: replace the bare
`RuntimeError` side_effect with an async-callable side_effect so
AsyncMock awaits the coroutine before the exception propagates.
Three new unit tests cover the new branches:
test_backfill_emits_progress_log_every_20_batches,
test_backfill_logs_warning_when_sentinel_upsert_fails,
test_ensure_keyword_payload_indexes_summarises_failed_fields.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
