docs: Emphasize server-side processing in ADR-012 viz pane

Updates ADR-012 to clarify that all search and filtering operations
must happen server-side, not in the browser.

Key changes:
- Enhanced viz pane data flow showing server-side processing
- Added performance benefits section (384x bandwidth reduction)
- Detailed server-side filtering approach:
  * Query execution via search/algorithms.py
  * User ID filtering (multi-tenant security)
  * Document type filtering
  * PCA reduction (768-dim → 2D) on server
  * Only 2D coordinates + metadata sent to client
- Updated Phase 3 implementation plan:
  * Remove ALL client-side search logic
  * Implement /app/vector-viz server endpoint
  * htmx form submission for queries
  * Performance optimizations (caching, streaming)

This ensures:
- Minimal bandwidth usage (only 2 floats per doc vs 768)
- Client handles only visualization, not computation
- Can visualize 10,000+ documents without client lag
- Raw vectors never leave server (security)
- Same search logic as MCP tool (consistency)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Chris Coutinho
2025-11-15 00:02:54 +01:00
co-authored by Claude
parent 5e67277049
commit 56bd85c0f7
+59 -17
View File
@@ -131,17 +131,34 @@ We will implement a **unified multi-algorithm search architecture** with the fol
7. Return ranked SearchResponse to client 7. Return ranked SearchResponse to client
``` ```
#### Viz Pane Request #### Viz Pane Request (Server-Side Processing)
``` ```
1. User navigates to /app (Vector Visualization tab) 1. User navigates to /app (Vector Visualization tab)
2. Browser loads vector-viz fragment via htmx 2. Browser loads vector-viz fragment via htmx
3. User adjusts algorithm selector and weight sliders 3. User enters query and adjusts algorithm/weights
4. JavaScript calls same search/algorithms.py backend 4. htmx sends request to /app/vector-viz endpoint
5. PCA reduces vectors to 2D for visualization 5. Server executes search via search/algorithms.py:
6. Plotly.js renders interactive scatter plot - Filters by user_id (multi-tenant security)
7. Matching results highlighted, non-matches grayed out - Applies selected algorithm (semantic/keyword/fuzzy/hybrid)
- Filters by document type (notes/files/calendar/contacts)
- Retrieves matching results + metadata
6. Server performs PCA reduction (768-dim → 2D):
- Converts matching results to 2D coordinates
- Only sends coordinates + metadata (not full vectors)
- Dramatically reduces bandwidth (e.g., 768 floats → 2 floats per doc)
7. Server returns JSON: {results: [...], coordinates_2d: [...], stats: {...}}
8. Browser receives lightweight response
9. Plotly.js renders interactive scatter plot
10. Matching results highlighted (blue), non-matches grayed (40% opacity)
``` ```
**Performance Benefits of Server-Side Processing**:
- **Bandwidth reduction**: ~384x less data (2 floats vs 768 floats per document)
- **Client efficiency**: Browser only handles visualization, not computation
- **Scalability**: Can visualize 10,000+ documents without client-side lag
- **Security**: Raw vectors never leave server
- **Consistency**: Same search logic as MCP tool (no drift)
### 1. Core Search Algorithms ### 1. Core Search Algorithms
Four search algorithms will be available: Four search algorithms will be available:
@@ -238,10 +255,19 @@ nextcloud_mcp_server/
Update viz pane (`nextcloud_mcp_server/auth/userinfo_routes.py`) to: Update viz pane (`nextcloud_mcp_server/auth/userinfo_routes.py`) to:
1. **Use shared algorithms**: Import from `search/algorithms.py` 1. **Use shared algorithms**: Import from `search/algorithms.py`
2. **Remove client-side filtering**: Call server-side search methods 2. **Server-side filtering**: All search and filtering operations happen server-side
3. **User accessibility**: Available to all users with vector sync enabled - Query execution via shared search backend
4. **Security**: Filter results by `user_id` (only show user's own documents) - Document type filtering (notes, files, calendar, contacts)
5. **Interactive testing**: Allow users to: - User ID filtering for multi-tenant security
- Only matching results + metadata sent to client
- Reduces bandwidth and improves performance
3. **PCA reduction**: Server performs dimensionality reduction (768-dim → 2D)
- Only 2D coordinates sent to browser for visualization
- Dramatically reduces data transfer vs sending full vectors
- Enables visualization of large document collections
4. **User accessibility**: Available to all users with vector sync enabled
5. **Security**: Filter results by `user_id` (only show user's own documents)
6. **Interactive testing**: Allow users to:
- Select algorithm type - Select algorithm type
- Adjust weights (hybrid mode) - Adjust weights (hybrid mode)
- Compare results across algorithms - Compare results across algorithms
@@ -403,13 +429,29 @@ def reciprocal_rank_fusion(
### Phase 3: Update Viz Pane (Week 2) ### Phase 3: Update Viz Pane (Week 2)
1. Remove client-side search filtering **Critical: All processing must happen server-side**
2. Call shared `search/algorithms.py` methods
3. Add user_id filtering for multi-user security 1. **Remove client-side search filtering**
4. Add algorithm selector dropdown - Delete JavaScript-based keyword/fuzzy matching
5. Add weight adjustment controls (sliders) - Remove client-side document type filtering
6. Update visualization to show algorithm-specific metadata - No search logic in browser
7. Add side-by-side comparison mode 2. **Implement server-side endpoint** (`/app/vector-viz`)
- Accept query, algorithm, weights, doc_type filters
- Execute search via `search/algorithms.py`
- Filter results by user_id (security)
- Perform PCA reduction (768-dim → 2D)
- Return JSON with 2D coordinates + metadata only
3. **Update frontend**
- htmx form submission to `/app/vector-viz`
- Algorithm selector dropdown
- Weight adjustment sliders (htmx updates on change)
- Document type checkboxes
- Plotly.js visualization of server response
4. **Performance optimization**
- Limit results to user's documents only
- Cache PCA transformation (invalidate on new vectors)
- Stream large result sets if needed
- Add loading indicators for server processing
### Phase 4: Documentation and Testing (Week 2-3) ### Phase 4: Documentation and Testing (Week 2-3)