feat(search): ADR-027 Phase 2 — file-path filter

Add a path_prefix filter to semantic search, honoured on both the MCP tool and
the dense-only visualization/API paths through the shared filter contract.

- build_base_filter_conditions: append FieldCondition(file_path,
  MatchText(path_prefix)) when set. file_path is only on doc_type == "file"
  points, so a non-empty path_prefix implicitly restricts to files.
- Promote path_prefix to an explicit keyword param on the SearchAlgorithm ABC
  and both algorithms; thread it through nc_semantic_search (blank ⇒ no filter),
  the /api/v1 search endpoints, and the viz route.
- Add a file_path TEXT payload index to _PAYLOAD_INDEX_FIELDS (no content
  re-index; idempotent startup migration). MatchText tokenizes on server Qdrant
  and matches by substring on local/embedded qdrant-client — both serve folder
  scoping.
- Update ADR-027 (Phase 2 implemented; readiness table; semantics note). Tests.

Refs ADR-027 Phase 2. Deck #177.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Chris Coutinho
2026-06-03 00:51:12 +02:00
co-authored by Claude Opus 4.8
parent c2c8dc1a08
commit ab128bef5b
11 changed files with 121 additions and 10 deletions
+15 -7
View File
@@ -1,6 +1,6 @@
# ADR-027: Rich Search Filters for Semantic Search
**Status**: Accepted (Phase 1 implemented; Phases 23 deferred)
**Status**: Accepted (Phases 12 implemented; Phase 3 deferred)
**Date**: 2026-06-02
**Depends On**: ADR-012 (Unified Multi-Algorithm Search), ADR-014 (BM25 Search), ADR-019 (Verify-on-Read for Semantic Search)
**Tracking**: Astrolabe Cloud POC Deck card #177
@@ -68,7 +68,7 @@ Filters can only be applied to fields that exist in the Qdrant payload (built in
|---|---|---|---|
| Modified-date range | `modified_at` | `int` (Unix ts) | ✅ **Ready** — value present on every point; needs a payload index (added in Phase 1, no content re-index) |
| Document type | `doc_type` | keyword-indexed `str` | ✅ Implemented |
| Directory / path | `file_path` (files only) | `str` | ⚠️ Stored but **not keyword-indexed** — prefix/match needs a payload index |
| Directory / path | `file_path` (files only) | `str` | ✅ **Ready (Phase 2)** — value present on file points; TEXT payload index added (no content re-index), filtered with `MatchText` |
| Tags | — | — | ❌ **Not indexed** — no `tags` field is written during scanning |
| Category (notes) | — | — | ❌ Not in payload — fetched from the Notes API at verify time only |
@@ -186,11 +186,19 @@ express:
it on the answer path. When demand appears it can be threaded through using exactly the same
parameter + `parse_modified_timestamp` pattern; doing it now would add an unused parameter and
widen the change for no user-visible gain.
- **Phase 2 — directory / path.** Create a Qdrant payload index on `file_path`, add a `path_prefix`
parameter, and add an `NcFilePicker` folder chooser. Scoped to `doc_type == "file"`: the frontend
must **hide/disable the path filter unless the file doc type is selected**, because `file_path`
is only written to the payload for `doc_type == "file"` (`processor.py`) — a path condition on
any other type matches nothing and silently returns zero results.
- **Phase 2 — directory / path (implemented).** Add a `path_prefix` parameter threaded through the
same shared contract (`build_base_filter_conditions` → `FieldCondition(key="file_path",
match=MatchText(text=path_prefix))`), and a `file_path` **TEXT** payload index in
`_PAYLOAD_INDEX_FIELDS` (no content re-index — `file_path` is already on every file point).
`MatchText` semantics differ by backend: **server Qdrant** tokenizes (AND-of-tokens, so
`/Projects/Reports` matches files whose path contains both the `Projects` and `Reports` tokens),
while **local/embedded qdrant-client** matches by substring containment — both serve folder
scoping, neither is a strict left-anchored prefix (a future strict-prefix would need an indexed
ancestor-path array, i.e. a re-index, so it stays out of Phase 2). Because `file_path` is only
written for `doc_type == "file"`, a non-empty `path_prefix` implicitly restricts results to
files; the frontend uses a native path text input enabled only when the **Files** doc type is in
scope (rather than `NcFilePicker`, to avoid a hard `@nextcloud/vue` component-version dependency —
same rationale as the date inputs). A blank value is treated as "no filter".
- **Phase 3 — tags (and optionally category).** Add a `tags: list[str]` payload field in
`processor.py`, propagate Nextcloud system tags during scanning, trigger a re-index, then wire
`NcSelectTags` (`MatchAny` over tags). Re-index cost lives here, isolated from the cheap wins.