Round-8 claude-review (no blockers; comment-only):
- 🟡 Documented that Ollama's /api/embed prompt_eval_count is assumed
batch-level total and is unverified against a live instance (Ollama isn't the
Cloud billing provider); if it proves last-item-only, switch to per-item
summing. The char estimate already covers versions that omit the field.
- 🟡 Noted on the astrolabe_embedding_tokens_total counter that operation="query"
is recorded pre-Qdrant, so it can legitimately exceed the billing-store
tokens_embedded aggregate when a search fails post-embed — dashboards
shouldn't alert on that healthy gap.
Deferred (reviewer: "minor nit, acceptable"): record_indexing_usage awaited in
the task group — the group awaits all child tasks regardless, the write is
best-effort + fast, and start_soon would need the tg threaded into the closure
for marginal gain.
Deck #284.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Round-5 claude-review (merge-ready; all nits):
- 🟡 Added test_empty_doc_types_normalizes_to_null pinning doc_types=[] → None
in record_search_usage metadata (matches the None case).
- 🟡 record_search_usage docstring now notes nc_semantic_search_answer always
meters with doc_types=None (it exposes no doc_types parameter).
- 🟢 BM25HybridSearchAlgorithm.__init__ now sets query_embedding /
query_token_count alongside _embedded_query, so all three cache fields are
instance attributes from construction (was relying on the class-level
SearchAlgorithm defaults).
- 🟢 Ollama embed_batch_with_usage caches _dimension inline (mirrors
OpenAI/Mistral), so the dimension is set via any embed path.
- 🟢 record_indexing_usage documents the independent-record / partial-failure
semantics under SUM aggregation.
Deck #67.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Round-1 claude-review findings:
- 🔴 Multi-doc_type search billed N embedding calls as 1. nc_semantic_search
loops search() once per doc_type on one BM25HybridSearchAlgorithm instance,
and each call re-embedded the query, so only the last query_token_count was
recorded. Cache the dense embedding per query on the (per-request) instance
so the query is embedded — and metered — exactly once regardless of how many
doc_types are searched. This also removes the redundant per-type embed work
and avoids billing a user N× for one logical query.
- 🟡 Ollama embed() now delegates to embed_with_usage() so single and batch
embeds use the same /api/embed endpoint (was the legacy /api/embeddings),
keeping _detect_dimension and other embed() callers consistent.
- 🟢 round() instead of truncating int() when coercing provider-reported token
counts (forward-compatible if a provider ever returns a float).
Tests: per-instance query-embedding cache (embedded once across 3 doc_types;
re-embeds on a different query).
Deferred (stated on the PR): mistral/openai single-embed dual path (changes
tested error/request semantics on the cloud-critical path — separate refactor),
bedrock boto3 sync-in-async (pre-existing; no new invoke_model calls per doc).
Deck #67.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
embeddings_queries now records the embedding request's token count (the unit
upstream providers bill on) instead of an operation count, and fires on the
indexing path too. Previously only semantic search recorded it (value=1), so a
re-indexing run produced no embeddings_queries events at all — only pages_chunks.
- Provider layer: additive embed_with_usage / embed_batch_with_usage surface the
per-request token count (Mistral/OpenAI usage.total_tokens, Bedrock Titan
inputTextTokenCount, Ollama prompt_eval_count); a char-based estimate is the
fallback (Simple, and any provider/response without a token field). Gateway and
EmbeddingService forward through. The count travels as a return value / a
per-request SearchAlgorithm attribute — never on the singleton — so concurrent
indexing + search can't mis-attribute bills.
- Indexing (vector/processor.py): records embeddings_queries (value=batch tokens)
alongside the existing pages_chunks event.
- Search (server/semantic.py): value is now the query embedding's token count,
relayed from BM25HybridSearchAlgorithm via query_token_count.
The astrolabe_embeddings_queries Stripe meter (sum aggregation) now sums tokens
with no CP/Terraform change. The meter "queries"->tokens naming/unit
clarification (homelab-terraform #254) + CP rollup/portal copy is a follow-up.
Deck #67.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Switch from sequential loop to /api/embed batch endpoint
- Use 'input' array parameter instead of individual 'prompt' requests
- Process in chunks of 32 to avoid quality degradation (issue #6262)
- Reduces HTTP overhead: 128 texts = 4 requests instead of 128
- Maintains backward compatibility with embed() for single embeddings
Ref: ollama/ollama#6262
Refactored LLM provider infrastructure to support sustainable additions of new providers with both embedding and text generation capabilities.
## Major Changes
### Unified Provider Architecture (ADR-015)
- Created `nextcloud_mcp_server/providers/` with unified Provider ABC
- Providers now support optional capabilities (embeddings and/or generation)
- Auto-detection registry with priority: Bedrock → Ollama → Simple
- Backward compatible - existing code continues to work
### New Providers
- **BedrockProvider**: Full Amazon Bedrock integration
- Embeddings: Titan Embed, Cohere Embed models
- Generation: Claude, Llama, Titan Text, Mistral models
- Model-specific request/response handling
- AWS credential chain integration
- **OllamaProvider**: Migrated with both capabilities support
- **AnthropicProvider**: Moved from test code to production providers
- **SimpleProvider**: Migrated in-memory fallback provider
### Breaking Changes
None - full backward compatibility maintained:
- `embedding.get_embedding_service()` still works
- RAG evaluation tests updated to use unified providers
- All existing tests pass (127 unit tests)
### Testing
- Added 9 comprehensive Bedrock unit tests with mocked boto3
- All existing unit tests pass
- Type checking (ty) and linting (ruff) pass
- Verified backward compatibility
### Documentation
- `docs/ADR-015-unified-provider-architecture.md`: Comprehensive ADR
- `docs/bedrock-setup.md`: AWS setup guide with IAM permissions
- `CLAUDE.md`: Updated with provider architecture section
### Dependencies
- Added `boto3>=1.35.0` to dev dependencies (optional)
## Environment Variables
### Bedrock
- `AWS_REGION`: AWS region (e.g., "us-east-1")
- `BEDROCK_EMBEDDING_MODEL`: Model ID for embeddings
- `BEDROCK_GENERATION_MODEL`: Model ID for generation
- `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`: Optional credentials
### Ollama
- `OLLAMA_BASE_URL`: API URL
- `OLLAMA_EMBEDDING_MODEL`: Embedding model (default: "nomic-embed-text")
- `OLLAMA_GENERATION_MODEL`: Generation model
## AWS Bedrock Permissions Required
Minimal IAM policy:
```json
{
"Effect": "Allow",
"Action": ["bedrock:InvokeModel"],
"Resource": ["arn:aws:bedrock:*::foundation-model/*"]
}
```
See `docs/bedrock-setup.md` for detailed setup instructions.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
