feat(helm): Add document chunking configuration
Add support for configurable document chunking parameters to Helm chart to match docker-compose and application capabilities. Changes: 1. values.yaml: - Add documentChunking section with chunkSize (512) and chunkOverlap (50) - Include comprehensive comments explaining chunking strategies - Positioned between vectorSync and qdrant sections 2. templates/deployment.yaml: - Add DOCUMENT_CHUNK_SIZE and DOCUMENT_CHUNK_OVERLAP env vars - Always set (not conditional), used by vector sync processor - Environment variables follow same pattern as config.py defaults 3. README.md: - Add documentChunking parameter table in Vector Search section - Document chunking strategies (small/medium/large chunks) - Explain overlap recommendations (10-20% of chunk size) Validation: - helm lint: Passes - helm template: Environment variables correctly generated - Custom values: Work as expected (tested with chunkSize=1024) - Always present: Not conditional on vectorSync.enabled This maintains feature parity between Helm and docker-compose deployments, allowing users to tune chunking for their embedding models and use cases. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -219,6 +219,19 @@ Enable semantic search capabilities by deploying a vector database (Qdrant) and
|
||||
| `vectorSync.processorWorkers` | Number of concurrent processor workers | `3` |
|
||||
| `vectorSync.queueMaxSize` | Maximum queue size for pending documents | `10000` |
|
||||
|
||||
**Document Chunking Configuration:**
|
||||
|
||||
| Parameter | Description | Default |
|
||||
|-----------|-------------|---------|
|
||||
| `documentChunking.chunkSize` | Number of words per chunk for embedding | `512` |
|
||||
| `documentChunking.chunkOverlap` | Number of overlapping words between chunks | `50` |
|
||||
|
||||
**Chunking Strategy:**
|
||||
- **Small chunks (256-384)**: Better precision for searches, more storage overhead
|
||||
- **Medium chunks (512-768)**: Balanced approach (recommended for most use cases)
|
||||
- **Large chunks (1024+)**: Better context preservation, less precise matching
|
||||
- **Overlap**: Should be 10-20% of chunk size to preserve context across boundaries
|
||||
|
||||
**Qdrant Vector Database:**
|
||||
|
||||
Qdrant is deployed as a subchart when `qdrant.enabled` is `true`. All configuration values are passed through to the [qdrant/qdrant](https://github.com/qdrant/qdrant-helm) chart.
|
||||
|
||||
Reference in New Issue
Block a user