refactor: migrate vector sync from asyncio.Queue to anyio memory object streams

Replace asyncio.Queue with anyio.create_memory_object_stream() throughout
the vector sync system for better library consistency and improved shutdown
semantics.

## Changes Made

**scanner.py**:
- Changed parameter type from `asyncio.Queue` to `MemoryObjectSendStream[DocumentTask]`
- Replaced all `await document_queue.put()` calls with `await send_stream.send()`
- Wrapped scanner loop in `async with send_stream:` context manager for automatic cleanup
- Updated log messages: "Queued" → "Sent"
- Removed `import asyncio` (no longer needed)

**processor.py**:
- Changed parameter type from `asyncio.Queue` to `MemoryObjectReceiveStream[DocumentTask]`
- Replaced `asyncio.wait_for(document_queue.get(), timeout=1.0)` with `anyio.fail_after(1.0)` + `await receive_stream.receive()`
- Removed all `document_queue.task_done()` calls (not needed with streams)
- Added `anyio.EndOfStream` exception handling for graceful shutdown when scanner closes
- Removed `import asyncio` (no longer needed)

**app.py**:
- Removed `import asyncio` from top-level imports
- Added `from anyio.streams.memory import MemoryObjectReceiveStream, MemoryObjectSendStream`
- Updated AppContext dataclass:
  - Replaced `document_queue: Optional[asyncio.Queue]` with:
    - `document_send_stream: Optional[MemoryObjectSendStream]`
    - `document_receive_stream: Optional[MemoryObjectReceiveStream]`
- Updated `app_lifespan_basic()`:
  - Replaced `asyncio.Queue(maxsize=...)` with `anyio.create_memory_object_stream(max_buffer_size=...)`
  - Pass `send_stream` to scanner_task
  - Pass `receive_stream.clone()` to each processor_task (enables multiple consumers)
  - Updated AppContext yield to include both streams
- Updated `starlette_lifespan()`:
  - Same changes as app_lifespan_basic for streamable-http transport
  - Removed `import asyncio as asyncio_module` (no longer needed)
  - Updated app.state storage to use send_stream and receive_stream

**semantic.py**:
- Updated `nc_get_vector_sync_status()` tool:
  - Access `document_receive_stream` instead of `document_queue` from lifespan context
  - Use `stream_stats.current_buffer_used` instead of `queue.qsize()` for pending count
  - More reliable metrics (qsize() was not guaranteed accurate)

## Benefits

1. **Library Consistency**: Pure anyio throughout codebase (was mixing asyncio.Queue with anyio.Event and anyio.create_task_group)
2. **Graceful Shutdown**: `async with send_stream:` automatically closes stream on exit, signaling EndOfStream to all processors
3. **Better Timeout Handling**: `anyio.fail_after()` is more idiomatic than `asyncio.wait_for()`
4. **Stream Cloning**: Easy to add multiple consumers via `receive_stream.clone()`
5. **Better Statistics**: `.statistics()` provides accurate buffer metrics (qsize() was unreliable)
6. **Type Safety**: Separate send/receive types prevent accidental misuse
7. **No task_done() tracking**: Streams handle completion automatically

## Testing

- ✅ All 69 unit tests passing
- ✅ All 5 smoke tests passing
- ✅ No regressions in functionality
- ✅ Graceful shutdown behavior improved

## References

- https://anyio.readthedocs.io/en/stable/why.html#queue-fix
- https://anyio.readthedocs.io/en/stable/streams.html#memory-object-streams

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

nextcloud_mcp_server/vector/scanner.py

@@ -3,12 +3,12 @@
 Periodically scans enabled users' content and queues changed documents for processing.
 """
 
-import asyncio
 import logging
 import time
 from dataclasses import dataclass
 
 import anyio
+from anyio.streams.memory import MemoryObjectSendStream
 from qdrant_client.models import FieldCondition, Filter, MatchValue
 
 from nextcloud_mcp_server.client import NextcloudClient
@@ -35,7 +35,7 @@ _potentially_deleted: dict[tuple[str, str], float] = {}
 
 
 async def scanner_task(
-    document_queue: asyncio.Queue,
+    send_stream: MemoryObjectSendStream[DocumentTask],
     shutdown_event: anyio.Event,
     wake_event: anyio.Event,
     nc_client: NextcloudClient,
@@ -47,7 +47,7 @@ async def scanner_task(
     For BasicAuth mode, scans a single user with credentials available at runtime.
 
     Args:
-        document_queue: Queue to enqueue changed documents
+        send_stream: Stream to send changed documents to processors
         shutdown_event: Event signaling shutdown
         wake_event: Event to trigger immediate scan
         nc_client: Authenticated Nextcloud client
@@ -56,44 +56,45 @@ async def scanner_task(
     logger.info(f"Scanner task started for user: {user_id}")
     settings = get_settings()
 
-    while not shutdown_event.is_set():
-        try:
-            # Scan user documents
-            await scan_user_documents(
-                user_id=user_id,
-                document_queue=document_queue,
-                nc_client=nc_client,
-            )
+    async with send_stream:
+        while not shutdown_event.is_set():
+            try:
+                # Scan user documents
+                await scan_user_documents(
+                    user_id=user_id,
+                    send_stream=send_stream,
+                    nc_client=nc_client,
+                )
 
-        except Exception as e:
-            logger.error(f"Scanner error: {e}", exc_info=True)
+            except Exception as e:
+                logger.error(f"Scanner error: {e}", exc_info=True)
 
-        # Sleep until next interval or wake event
-        try:
-            with anyio.move_on_after(settings.vector_sync_scan_interval):
-                # Wait for wake event or shutdown (whichever comes first)
-                await wake_event.wait()
-        except anyio.get_cancelled_exc_class():
-            # Shutdown, exit loop
-            break
+            # Sleep until next interval or wake event
+            try:
+                with anyio.move_on_after(settings.vector_sync_scan_interval):
+                    # Wait for wake event or shutdown (whichever comes first)
+                    await wake_event.wait()
+            except anyio.get_cancelled_exc_class():
+                # Shutdown, exit loop
+                break
 
-    logger.info("Scanner task stopped")
+    logger.info("Scanner task stopped - stream closed")
 
 
 async def scan_user_documents(
     user_id: str,
-    document_queue: asyncio.Queue,
+    send_stream: MemoryObjectSendStream[DocumentTask],
     nc_client: NextcloudClient,
     initial_sync: bool = False,
 ):
     """
-    Scan a single user's documents and queue changes.
+    Scan a single user's documents and send changes to processor stream.
 
     Args:
         user_id: User to scan
-        document_queue: Queue to enqueue changed documents
+        send_stream: Stream to send changed documents to processors
         nc_client: Authenticated Nextcloud client
-        initial_sync: If True, queue all documents (first-time sync)
+        initial_sync: If True, send all documents (first-time sync)
     """
     logger.info(f"Scanning documents for user: {user_id}")
 
@@ -102,9 +103,9 @@ async def scan_user_documents(
     logger.debug(f"Found {len(notes)} notes for {user_id}")
 
     if initial_sync:
-        # Queue everything on first sync
+        # Send everything on first sync
         for note in notes:
-            await document_queue.put(
+            await send_stream.send(
                 DocumentTask(
                     user_id=user_id,
                     doc_id=str(note["id"]),
@@ -113,7 +114,7 @@ async def scan_user_documents(
                     modified_at=note["modified"],
                 )
             )
-        logger.info(f"Queued {len(notes)} documents for initial sync: {user_id}")
+        logger.info(f"Sent {len(notes)} documents for initial sync: {user_id}")
         return
 
     # Get indexed state from Qdrant
@@ -154,9 +155,9 @@ async def scan_user_documents(
             )
             del _potentially_deleted[doc_key]
 
-        # Queue if never indexed or modified since last index
+        # Send if never indexed or modified since last index
         if indexed_at is None or note["modified"] > indexed_at:
-            await document_queue.put(
+            await send_stream.send(
                 DocumentTask(
                     user_id=user_id,
                     doc_id=doc_id,
@@ -183,12 +184,12 @@ async def scan_user_documents(
                 time_missing = current_time - first_missing_time
 
                 if time_missing >= grace_period:
-                    # Grace period elapsed, queue for deletion
+                    # Grace period elapsed, send for deletion
                     logger.info(
                         f"Document {doc_id} missing for {time_missing:.1f}s "
-                        f"(>{grace_period:.1f}s grace period), queueing deletion"
+                        f"(>{grace_period:.1f}s grace period), sending deletion"
                     )
-                    await document_queue.put(
+                    await send_stream.send(
                         DocumentTask(
                             user_id=user_id,
                             doc_id=doc_id,
@@ -198,7 +199,7 @@ async def scan_user_documents(
                         )
                     )
                     queued += 1
-                    # Remove from tracking after queueing deletion
+                    # Remove from tracking after sending deletion
                     del _potentially_deleted[doc_key]
                 else:
                     logger.debug(
@@ -213,6 +214,6 @@ async def scan_user_documents(
                 _potentially_deleted[doc_key] = current_time
 
     if queued > 0:
-        logger.info(f"Queued {queued} documents for incremental sync: {user_id}")
+        logger.info(f"Sent {queued} documents for incremental sync: {user_id}")
     else:
         logger.debug(f"No changes detected for {user_id}")