mcp-nextcloud

1,118 Commits 1 Branch 0 Tags

3 Commits

Include renames

Author	SHA1	Message	Date
Chris CoutinhoandClaude	4e43d15153	fix: Move grafana_folder from labels to annotations Fixes Kubernetes label validation error when deploying dashboard ConfigMap. Problem: - Kubernetes labels cannot contain spaces (validation regex: [A-Za-z0-9][-A-Za-z0-9_.]*[A-Za-z0-9]) - Previous implementation had grafana_folder: "Nextcloud MCP" as a label - Deployment failed with: "Invalid value: 'Nextcloud MCP'" Solution: - Move grafana_folder from labels to annotations (annotations allow spaces) - Keep grafana_dashboard="1" as label for ConfigMap discovery - Grafana sidecar reads folder name from folderAnnotation parameter Changes: - dashboard-configmap.yaml: Move grafana_folder to annotations section - dashboards/README.md: Fix kubectl commands to use annotations - values.yaml: Update comments to clarify annotation usage This follows the standard kube-prometheus-stack pattern where: - Labels are used for ConfigMap discovery (strict validation) - Annotations are used for metadata like folder names (relaxed validation) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>	2025-11-13 13:08:45 +01:00
Chris CoutinhoandClaude	4ea5ed72d4	feat: Add Grafana dashboard and vector sync metric instrumentation Implement comprehensive observability for vector database synchronization with Grafana dashboard and Prometheus metrics. ## Part 1: Grafana Dashboard Created all-in-one operations dashboard with 7 rows and 34 panels: ### Dashboard Structure: - Overview Row: Request rate, error rate, P95 latency, active requests - HTTP Metrics (RED): Request/error rates by endpoint, latency percentiles - MCP Tools: Call volume, error rates, execution duration by tool - Nextcloud API: API calls/latency by app, retry patterns - OAuth & Authentication: Token validations, exchanges, cache hit rate - Dependencies & Health: Status for Nextcloud/Qdrant/Keycloak/Unstructured - Vector Sync: Processing throughput, queue depth, Qdrant operations ### Helm Chart Integration: - Added dashboard-configmap.yaml template for automatic provisioning - Configured Grafana sidecar auto-discovery (label: grafana_dashboard="1") - Added dashboards configuration section in values.yaml (opt-in) - Updated Chart.yaml with dashboard annotations - Enhanced NOTES.txt with dashboard deployment instructions - Comprehensive documentation in dashboards/README.md Dashboard supports dynamic filtering via variables: - datasource: Prometheus data source selection - namespace: Filter by Kubernetes namespace - pod: Multi-select pod filtering - interval: Query interval (1m/5m/10m/30m/1h) ## Part 2: Vector Sync Metric Instrumentation Implemented metric recording throughout vector sync pipeline: ### metrics.py: Added convenience functions: - record_vector_sync_scan() - Track documents per scan - record_vector_sync_processing() - Track processing duration/status - record_qdrant_operation() - Track database operations - update_vector_sync_queue_size() - Track queue depth ### scanner.py: - Record number of documents found in each scan - Enables monitoring of scan throughput ### processor.py: - Record processing duration for each document - Track success/failure status with timing - Record Qdrant upsert/delete operations - Handle all code paths (success, deletion, error) ### semantic.py: - Wrap Qdrant query_points with try/except - Record search operation success/failure ## Metrics Exposed: - mcp_vector_sync_documents_scanned_total - mcp_vector_sync_documents_processed_total{status} - mcp_vector_sync_processing_duration_seconds (histogram) - mcp_vector_sync_queue_size (gauge) - mcp_qdrant_operations_total{operation,status} This enables monitoring of: - Scan and processing throughput - Processing latency (P50/P95/P99) - Error rates for processing and Qdrant operations - Queue depth trends - Complete observability of vector sync pipeline ## Testing: Verified locally that metrics are recorded correctly: - 36 documents scanned - 3 documents processed (avg 7.5s each) - 3 successful Qdrant upsert operations - Search operations tracked ## Deployment: Enable dashboard provisioning in Helm values: ```yaml dashboards: enabled: true grafanaFolder: "Nextcloud MCP" ``` 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>	2025-11-13 11:49:20 +01:00
Chris CoutinhoandClaude	093ac5b5ba	feat(helm): Add observability support with ServiceMonitor and Grafana dashboard Add comprehensive observability configuration to Helm chart: Helm Values: - Add observability configuration section for metrics, tracing, and logging - Add serviceMonitor configuration (disabled by default) - Add prometheusRule configuration (disabled by default) Templates: - Update deployment to include observability environment variables - Update deployment to expose metrics port (9090) - Update service to expose metrics port - Add ServiceMonitor template for Prometheus Operator - Add PrometheusRule template with critical and warning alerts Dashboards: - Add comprehensive Grafana dashboard JSON with 6 panels: - Request Rate (by method and endpoint) - Error Rate (5xx errors percentage) - Request Latency (P50/P95 by endpoint) - Top MCP Tools (by invocation volume) - Nextcloud API Latency (by app) - Vector Sync Queue Size - Add dashboard README with import instructions Alert Rules: - Critical: Server down, high error rate (>5%), high latency (>1s), dependency down - Warning: Token validation errors (>1%), vector sync queue high (>100), Qdrant slow (>500ms) All features are opt-in via values.yaml configuration. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>	2025-11-09 09:10:11 +01:00

Author

SHA1

Message

Date

Chris CoutinhoandClaude

4e43d15153

fix: Move grafana_folder from labels to annotations

Fixes Kubernetes label validation error when deploying dashboard ConfigMap.

Problem:
- Kubernetes labels cannot contain spaces (validation regex: [A-Za-z0-9][-A-Za-z0-9_.]*[A-Za-z0-9])
- Previous implementation had grafana_folder: "Nextcloud MCP" as a label
- Deployment failed with: "Invalid value: 'Nextcloud MCP'"

Solution:
- Move grafana_folder from labels to annotations (annotations allow spaces)
- Keep grafana_dashboard="1" as label for ConfigMap discovery
- Grafana sidecar reads folder name from folderAnnotation parameter

Changes:
- dashboard-configmap.yaml: Move grafana_folder to annotations section
- dashboards/README.md: Fix kubectl commands to use annotations
- values.yaml: Update comments to clarify annotation usage

This follows the standard kube-prometheus-stack pattern where:
- Labels are used for ConfigMap discovery (strict validation)
- Annotations are used for metadata like folder names (relaxed validation)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

2025-11-13 13:08:45 +01:00

Chris CoutinhoandClaude

4ea5ed72d4

feat: Add Grafana dashboard and vector sync metric instrumentation

Implement comprehensive observability for vector database synchronization
with Grafana dashboard and Prometheus metrics.

## Part 1: Grafana Dashboard

Created all-in-one operations dashboard with 7 rows and 34 panels:

### Dashboard Structure:
- **Overview Row**: Request rate, error rate, P95 latency, active requests
- **HTTP Metrics (RED)**: Request/error rates by endpoint, latency percentiles
- **MCP Tools**: Call volume, error rates, execution duration by tool
- **Nextcloud API**: API calls/latency by app, retry patterns
- **OAuth & Authentication**: Token validations, exchanges, cache hit rate
- **Dependencies & Health**: Status for Nextcloud/Qdrant/Keycloak/Unstructured
- **Vector Sync**: Processing throughput, queue depth, Qdrant operations

### Helm Chart Integration:
- Added dashboard-configmap.yaml template for automatic provisioning
- Configured Grafana sidecar auto-discovery (label: grafana_dashboard="1")
- Added dashboards configuration section in values.yaml (opt-in)
- Updated Chart.yaml with dashboard annotations
- Enhanced NOTES.txt with dashboard deployment instructions
- Comprehensive documentation in dashboards/README.md

Dashboard supports dynamic filtering via variables:
- datasource: Prometheus data source selection
- namespace: Filter by Kubernetes namespace
- pod: Multi-select pod filtering
- interval: Query interval (1m/5m/10m/30m/1h)

## Part 2: Vector Sync Metric Instrumentation

Implemented metric recording throughout vector sync pipeline:

### metrics.py:
Added convenience functions:
- record_vector_sync_scan() - Track documents per scan
- record_vector_sync_processing() - Track processing duration/status
- record_qdrant_operation() - Track database operations
- update_vector_sync_queue_size() - Track queue depth

### scanner.py:
- Record number of documents found in each scan
- Enables monitoring of scan throughput

### processor.py:
- Record processing duration for each document
- Track success/failure status with timing
- Record Qdrant upsert/delete operations
- Handle all code paths (success, deletion, error)

### semantic.py:
- Wrap Qdrant query_points with try/except
- Record search operation success/failure

## Metrics Exposed:

- mcp_vector_sync_documents_scanned_total
- mcp_vector_sync_documents_processed_total{status}
- mcp_vector_sync_processing_duration_seconds (histogram)
- mcp_vector_sync_queue_size (gauge)
- mcp_qdrant_operations_total{operation,status}

This enables monitoring of:
- Scan and processing throughput
- Processing latency (P50/P95/P99)
- Error rates for processing and Qdrant operations
- Queue depth trends
- Complete observability of vector sync pipeline

## Testing:

Verified locally that metrics are recorded correctly:
- 36 documents scanned
- 3 documents processed (avg 7.5s each)
- 3 successful Qdrant upsert operations
- Search operations tracked

## Deployment:

Enable dashboard provisioning in Helm values:
```yaml
dashboards:
  enabled: true
  grafanaFolder: "Nextcloud MCP"
```

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

2025-11-13 11:49:20 +01:00

Chris CoutinhoandClaude

093ac5b5ba

feat(helm): Add observability support with ServiceMonitor and Grafana dashboard

Add comprehensive observability configuration to Helm chart:

**Helm Values:**
- Add observability configuration section for metrics, tracing, and logging
- Add serviceMonitor configuration (disabled by default)
- Add prometheusRule configuration (disabled by default)

**Templates:**
- Update deployment to include observability environment variables
- Update deployment to expose metrics port (9090)
- Update service to expose metrics port
- Add ServiceMonitor template for Prometheus Operator
- Add PrometheusRule template with critical and warning alerts

**Dashboards:**
- Add comprehensive Grafana dashboard JSON with 6 panels:
  - Request Rate (by method and endpoint)
  - Error Rate (5xx errors percentage)
  - Request Latency (P50/P95 by endpoint)
  - Top MCP Tools (by invocation volume)
  - Nextcloud API Latency (by app)
  - Vector Sync Queue Size
- Add dashboard README with import instructions

**Alert Rules:**
- Critical: Server down, high error rate (>5%), high latency (>1s), dependency down
- Warning: Token validation errors (>1%), vector sync queue high (>100), Qdrant slow (>500ms)

All features are opt-in via values.yaml configuration.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

2025-11-09 09:10:11 +01:00