Chris CoutinhoandClaude Sonnet 4.5
20404cf3f2
Adds comprehensive vector search support for Nextcloud Deck cards,
including semantic search indexing, chunk preview in the vector viz UI,
and proper deep linking to cards.
**Vector Search Indexing**
- Add deck_card scanning in scanner.py (scan_deck_cards function)
- Index cards from non-archived, non-deleted boards
- Store metadata: board_id, board_title, stack_id, stack_title, card_type, duedate, owner
- Content structure: title + "\n\n" + description (matches indexing format)
- Incremental sync based on lastModified timestamp
- Deletion tracking with grace period
**Vector Visualization Support**
- Add deck_card handler in context.py for chunk preview expansion
- Include board_id in search result metadata (bm25_hybrid.py, semantic.py)
- Expose metadata in viz_routes.py JSON responses
- Update vector-viz.js to construct proper Deck URLs: /apps/deck/board/{board_id}/card/{card_id}
- Update vector_viz.html filter label from "Deck" to "Deck Cards"
**Bug Fixes**
- Skip soft-deleted boards (deletedAt > 0) to prevent 403 Forbidden errors
- Applies to scanner, processor, and context expansion code paths
- Deck API returns deleted boards but rejects stack access with 403
**Testing**
- Add integration tests in test_deck_vector_search.py:
- test_deck_card_semantic_search: Filtered search with doc_type="deck_card"
- test_deck_card_appears_in_cross_app_search: Cross-app search includes deck cards
- test_deck_card_chunk_context: Chunk context fetching for viz preview
**Documentation**
- Update README.md: Add Deck cards to semantic search feature list
- Update semantic-search-architecture.md: Document deck_card support
- Update nc_semantic_search tool documentation
**Type Safety**
- Fix type narrowing for page_boundaries (could be None) using cast()
- Fix scanner.py payload None check for type safety
Resolves vector search for Deck cards across indexing, search, and visualization.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
185 lines
12 KiB
HTML
185 lines
12 KiB
HTML
<div x-data="vizApp()">
|
|
<div class="viz-layout">
|
|
<!-- Top: Search Controls -->
|
|
<div class="viz-card viz-controls-card">
|
|
<form @submit.prevent="executeSearch">
|
|
<div class="viz-controls-grid">
|
|
<div class="viz-control-group">
|
|
<label>Search Query</label>
|
|
<input type="text" x-model="query" placeholder="Enter search query..." required />
|
|
</div>
|
|
|
|
<div class="viz-control-group">
|
|
<label>Algorithm</label>
|
|
<select x-model="algorithm">
|
|
<option value="semantic">Semantic (Dense)</option>
|
|
<option value="bm25_hybrid" selected>BM25 Hybrid</option>
|
|
</select>
|
|
</div>
|
|
|
|
<div class="viz-control-group">
|
|
<label>Fusion</label>
|
|
<select x-model="fusion" :disabled="algorithm !== 'bm25_hybrid'" :style="algorithm !== 'bm25_hybrid' ? 'opacity: 0.5; cursor: not-allowed;' : ''">
|
|
<option value="rrf" selected>RRF</option>
|
|
<option value="dbsf">DBSF</option>
|
|
</select>
|
|
</div>
|
|
|
|
<div class="viz-control-group">
|
|
<label> </label>
|
|
<button type="submit" class="viz-btn">Search</button>
|
|
</div>
|
|
|
|
<div class="viz-control-group">
|
|
<label> </label>
|
|
<button type="button" class="viz-btn-secondary" @click="showAdvanced = !showAdvanced">
|
|
<span x-text="showAdvanced ? 'Hide' : 'Advanced'"></span>
|
|
</button>
|
|
</div>
|
|
</div>
|
|
|
|
<!-- Advanced Options (Collapsible) -->
|
|
<div x-show="showAdvanced" style="margin-top: 16px;">
|
|
<div class="viz-controls-grid" style="grid-template-columns: repeat(auto-fit, minmax(150px, 1fr));">
|
|
<div class="viz-control-group">
|
|
<label>Document Types</label>
|
|
<div style="display: grid; grid-template-columns: 1fr 1fr; gap: 8px; font-size: 13px;">
|
|
<label style="display: flex; align-items: center; cursor: pointer; font-weight: normal;">
|
|
<input type="checkbox" x-model="docTypes" value="" style="margin-right: 4px;">
|
|
<span>All</span>
|
|
</label>
|
|
<label style="display: flex; align-items: center; cursor: pointer; font-weight: normal;">
|
|
<input type="checkbox" x-model="docTypes" value="note" style="margin-right: 4px;">
|
|
<span>Notes</span>
|
|
</label>
|
|
<label style="display: flex; align-items: center; cursor: pointer; font-weight: normal;">
|
|
<input type="checkbox" x-model="docTypes" value="file" style="margin-right: 4px;">
|
|
<span>Files</span>
|
|
</label>
|
|
<label style="display: flex; align-items: center; cursor: pointer; font-weight: normal;">
|
|
<input type="checkbox" x-model="docTypes" value="calendar" style="margin-right: 4px;">
|
|
<span>Calendar</span>
|
|
</label>
|
|
<label style="display: flex; align-items: center; cursor: pointer; font-weight: normal;">
|
|
<input type="checkbox" x-model="docTypes" value="contact" style="margin-right: 4px;">
|
|
<span>Contacts</span>
|
|
</label>
|
|
<label style="display: flex; align-items: center; cursor: pointer; font-weight: normal;">
|
|
<input type="checkbox" x-model="docTypes" value="deck_card" style="margin-right: 4px;">
|
|
<span>Deck Cards</span>
|
|
</label>
|
|
<label style="display: flex; align-items: center; cursor: pointer; font-weight: normal;">
|
|
<input type="checkbox" x-model="docTypes" value="news_item" style="margin-right: 4px;">
|
|
<span>News</span>
|
|
</label>
|
|
</div>
|
|
</div>
|
|
|
|
<div class="viz-control-group">
|
|
<label>Score Threshold</label>
|
|
<input type="number" x-model.number="scoreThreshold" min="0" max="1" step="any" />
|
|
</div>
|
|
|
|
<div class="viz-control-group">
|
|
<label>Result Limit</label>
|
|
<input type="number" x-model.number="limit" min="1" max="1000" />
|
|
</div>
|
|
|
|
<div class="viz-control-group">
|
|
<label>Display Options</label>
|
|
<label style="display: flex; align-items: center; cursor: pointer; font-weight: normal; margin-top: 4px;">
|
|
<input type="checkbox" x-model="showQueryPoint" @change="updatePlot()" style="margin-right: 6px;">
|
|
<span>Show Query Point</span>
|
|
</label>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
</form>
|
|
</div>
|
|
|
|
<!-- Plot -->
|
|
<div class="viz-card viz-card-plot">
|
|
<div id="viz-plot-container">
|
|
<div x-show="loading" class="viz-loading-overlay" x-transition.opacity.duration.200ms>
|
|
Executing search and computing PCA projection...
|
|
</div>
|
|
<div id="viz-plot" x-show="!loading" x-transition.opacity.duration.200ms></div>
|
|
</div>
|
|
</div>
|
|
|
|
<!-- Results -->
|
|
<div class="viz-card" style="flex: 0 0 auto;">
|
|
<h3 style="margin-top: 0;">Search Results (<span x-text="loading ? '...' : results.length"></span>)</h3>
|
|
|
|
<div x-show="loading" class="viz-loading" x-transition.opacity.duration.200ms>
|
|
Loading results...
|
|
</div>
|
|
|
|
<div x-show="!loading && results.length === 0" class="viz-no-results" x-transition.opacity.duration.200ms>
|
|
No results found. Try a different query or adjust your search parameters.
|
|
</div>
|
|
|
|
<template x-if="!loading && results.length > 0">
|
|
<div x-transition.opacity.duration.200ms>
|
|
<template x-for="result in results" :key="`${result.doc_type}_${result.id}_${result.chunk_start_offset || 0}`">
|
|
<div style="padding: 12px; border-bottom: 1px solid #eee;">
|
|
<a :href="getNextcloudUrl(result)" target="_blank" style="font-weight: 500; color: #0066cc; text-decoration: none;">
|
|
<span x-text="result.title"></span>
|
|
</a>
|
|
<div style="font-size: 14px; color: #666; margin-top: 4px;"
|
|
x-text="result.excerpt.length > 200 ? result.excerpt.substring(0, 200) + '...' : result.excerpt"></div>
|
|
<div style="font-size: 12px; color: #999; margin-top: 4px;">
|
|
Raw Score: <span x-text="result.original_score.toFixed(3)"></span>
|
|
(<span x-text="(result.score * 100).toFixed(0)"></span>% relative) |
|
|
Type: <span x-text="result.doc_type"></span>
|
|
</div>
|
|
|
|
<!-- Show Chunk button (only if chunk position is available) -->
|
|
<template x-if="hasChunkPosition(result)">
|
|
<button
|
|
class="chunk-toggle-btn"
|
|
@click="toggleChunk(result)"
|
|
x-text="isChunkExpanded(`${result.doc_type}_${result.id}_${result.chunk_start_offset || 0}`) ? 'Hide Chunk' : 'Show Chunk'"
|
|
></button>
|
|
</template>
|
|
|
|
<!-- Chunk context (expanded inline) -->
|
|
<template x-if="isChunkExpanded(`${result.doc_type}_${result.id}_${result.chunk_start_offset || 0}`)">
|
|
<div class="chunk-context" x-transition.opacity.duration.200ms>
|
|
<template x-if="chunkLoading[`${result.doc_type}_${result.id}_${result.chunk_start_offset || 0}`]">
|
|
<div style="color: #666; font-style: italic;">Loading chunk...</div>
|
|
</template>
|
|
<template x-if="!chunkLoading[`${result.doc_type}_${result.id}_${result.chunk_start_offset || 0}`]">
|
|
<div>
|
|
<!-- Highlighted page image for PDFs -->
|
|
<template x-if="expandedChunks[`${result.doc_type}_${result.id}_${result.chunk_start_offset || 0}`]?.highlighted_page_image">
|
|
<div class="chunk-image-container">
|
|
<div class="chunk-image-header">
|
|
<span>Page <span x-text="expandedChunks[`${result.doc_type}_${result.id}_${result.chunk_start_offset || 0}`]?.page_number"></span></span>
|
|
</div>
|
|
<img
|
|
:src="'data:image/png;base64,' + expandedChunks[`${result.doc_type}_${result.id}_${result.chunk_start_offset || 0}`]?.highlighted_page_image"
|
|
:alt="'Page ' + expandedChunks[`${result.doc_type}_${result.id}_${result.chunk_start_offset || 0}`]?.page_number"
|
|
class="chunk-highlighted-image"
|
|
/>
|
|
</div>
|
|
</template>
|
|
<!-- Text context -->
|
|
<template x-if="expandedChunks[`${result.doc_type}_${result.id}_${result.chunk_start_offset || 0}`]?.has_more_before">
|
|
<span class="chunk-ellipsis">...</span>
|
|
</template>
|
|
<span class="chunk-text" x-text="expandedChunks[`${result.doc_type}_${result.id}_${result.chunk_start_offset || 0}`]?.before_context"></span><span class="chunk-matched" x-text="expandedChunks[`${result.doc_type}_${result.id}_${result.chunk_start_offset || 0}`]?.chunk_text"></span><span class="chunk-text" x-text="expandedChunks[`${result.doc_type}_${result.id}_${result.chunk_start_offset || 0}`]?.after_context"></span><template x-if="expandedChunks[`${result.doc_type}_${result.id}_${result.chunk_start_offset || 0}`]?.has_more_after">
|
|
<span class="chunk-ellipsis">...</span>
|
|
</template>
|
|
</div>
|
|
</template>
|
|
</div>
|
|
</template>
|
|
</div>
|
|
</template>
|
|
</div>
|
|
</template>
|
|
</div><!-- Search Results -->
|
|
</div><!-- .viz-layout -->
|
|
</div><!-- x-data="vizApp()" -->
|