Commit Graph
2 Commits
Author SHA1 Message Date
Chris CoutinhoandClaude Opus 4.8 b134b2c539 docs(adr): correct stale statuses for shipped ADRs
Audit of all ADR status fields found several marked Proposed/Draft that are in
fact implemented. Correct them with an evidence pointer; also fix the ADR-025
title that read "ADR-024".

- ADR-007 background vector sync: Proposed -> Accepted/implemented
- ADR-008 MCP sampling: Proposed -> Accepted/implemented
- ADR-009 semantic.read scope: Proposed -> Accepted/implemented
- ADR-010 webhook-based vector sync: Proposed -> Accepted/implemented
- ADR-012 unified multi-algorithm search: Proposed -> Accepted/implemented
- ADR-013 RAG evaluation: Proposed -> Partially implemented
- ADR-018 Nextcloud settings-UI app: Proposed -> Accepted/implemented
- ADR-025 dynaconf: Proposed -> Accepted/implemented; fix title typo (ADR-024 -> ADR-025)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 12:00:54 +02:00
Chris CoutinhoandClaude c272ddd82d feat: implement RAG evaluation framework with CLI tooling
- Add ADR-013 documenting RAG evaluation architecture
- Implement two-part evaluation: Context Recall (retrieval) + Answer Correctness (generation)
- Create Click CLI for ground truth generation and corpus upload
- Add pytest fixtures and tests for retrieval/generation quality
- Use BeIR/nfcorpus dataset with 5 selected test queries
- Support Ollama and Anthropic LLM providers
- Generate synthetic ground truth answers offline
- Add comprehensive documentation in tests/rag_evaluation/README.md

The framework separates one-time setup (generate/upload) from test execution,
making tests much faster (~6-12 min vs ~15-25 min per run).

Tests are manual only (not in CI) and require external LLM access.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-15 23:11:21 +01:00