feat(search): ADR-027 Phase 1 — modified-date range filter

Add a modified_after/modified_before date-range filter to semantic search,
honoured on both the MCP tool path (BM25HybridSearchAlgorithm) and the
dense-only visualization/API path (SemanticSearchAlgorithm) through one shared
contract.

- Promote modified_after/modified_before to explicit keyword params on the
  SearchAlgorithm ABC and both concrete algorithms; factor the shared
  placeholder+ownership+doc_type+date filter into
  access_filter.build_base_filter_conditions so new filters land in one place.
- nc_semantic_search: accept RFC 3339 / ISO 8601 (or Unix seconds) bounds via
  utils.validation.parse_modified_timestamp; Annotated/Field constraints on the
  numeric args; explicit McpError guard for after > before. Thread the parsed
  bounds through the cross-app and per-doc_type dispatch.
- /api/v1 search endpoints + viz route parse the same formats and 400 on bad or
  inverted ranges.
- Add a modified_at INTEGER payload index to _PAYLOAD_INDEX_FIELDS; the
  idempotent _ensure_payload_indexes() startup path migrates existing
  collections with no content re-index.
- Update ADR-027 to resolve the review feedback (validation placement, shared
  algorithm contract, deferral of nc_semantic_search_answer, payload index,
  RFC-3339-at-the-boundary rationale). Add unit tests.

Refs ADR-027. Deck #177.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Chris Coutinho
2026-06-03 00:35:20 +02:00
co-authored by Claude Opus 4.8
parent f6ab04b2d9
commit c2c8dc1a08
13 changed files with 624 additions and 75 deletions
+120 -30
View File
@@ -1,6 +1,6 @@
# ADR-027: Rich Search Filters for Semantic Search # ADR-027: Rich Search Filters for Semantic Search
**Status**: Proposed **Status**: Accepted (Phase 1 implemented; Phases 23 deferred)
**Date**: 2026-06-02 **Date**: 2026-06-02
**Depends On**: ADR-012 (Unified Multi-Algorithm Search), ADR-014 (BM25 Search), ADR-019 (Verify-on-Read for Semantic Search) **Depends On**: ADR-012 (Unified Multi-Algorithm Search), ADR-014 (BM25 Search), ADR-019 (Verify-on-Read for Semantic Search)
**Tracking**: Astrolabe Cloud POC Deck card #177 **Tracking**: Astrolabe Cloud POC Deck card #177
@@ -66,16 +66,33 @@ Filters can only be applied to fields that exist in the Qdrant payload (built in
| Desired filter | Payload field | Type | Status | | Desired filter | Payload field | Type | Status |
|---|---|---|---| |---|---|---|---|
| Modified-date range | `modified_at` | `int` (Unix ts) | ✅ **Ready** — numeric, range-filterable today | | Modified-date range | `modified_at` | `int` (Unix ts) | ✅ **Ready** — value present on every point; needs a payload index (added in Phase 1, no content re-index) |
| Document type | `doc_type` | keyword-indexed `str` | ✅ Implemented | | Document type | `doc_type` | keyword-indexed `str` | ✅ Implemented |
| Directory / path | `file_path` (files only) | `str` | ⚠️ Stored but **not keyword-indexed** — prefix/match needs a payload index | | Directory / path | `file_path` (files only) | `str` | ⚠️ Stored but **not keyword-indexed** — prefix/match needs a payload index |
| Tags | — | — | ❌ **Not indexed** — no `tags` field is written during scanning | | Tags | — | — | ❌ **Not indexed** — no `tags` field is written during scanning |
| Category (notes) | — | — | ❌ Not in payload — fetched from the Notes API at verify time only | | Category (notes) | — | — | ❌ Not in payload — fetched from the Notes API at verify time only |
`modified_at` is a deliberate cross-app normalization the scanner already performs: the Notes API
and Deck return int Unix seconds natively, News timestamps are unit-normalized to seconds, and
WebDAV's RFC-1123 `getlastmodified` *string* is parsed to `int(dt.timestamp())`
(`client/webdav.py`). One `int` field therefore covers every doc type.
Two consequences: Two consequences:
- **`modified_at` is the cheap win.** It is already a numeric Unix timestamp on every point, so a - **`modified_at` is the cheap win — but it still needs a payload *index*.** The value is on every
`Range` condition works against the existing index with no re-index. point, so **no content re-index** is required. However, Qdrant only evaluates a `Range`
efficiently against an *indexed* field; without an index every dated query full-scans the
collection (and 400s on Qdrant Cloud strict payload-validation mode). Phase 1 therefore adds a
single `modified_at: INTEGER` entry to `_PAYLOAD_INDEX_FIELDS`
(`vector/qdrant_client.py`); the existing idempotent `_ensure_payload_indexes()` startup path
creates it on new **and** existing collections with no operator action. `INTEGER` (not `FLOAT`)
because the stored value is an int Unix-second timestamp.
- **Why not a Qdrant `datetime` index?** Qdrant's `datetime` index / `DatetimeRange` would let us
filter with RFC 3339 strings directly, but its indexer only ingests *string* payload values
(`value.as_str()` in Qdrant's `numeric_index/value_indexer.rs`) — it silently skips integer
payloads. Adopting it would mean re-storing `modified_at` as RFC 3339 strings on every point,
i.e. the full re-index Phase 1 is designed to avoid. We instead keep int storage + a numeric
`Range` and accept RFC 3339 only at the request boundary (see §3).
- **Tags / path / category are not free.** `file_path` filtering needs a Qdrant payload index - **Tags / path / category are not free.** `file_path` filtering needs a Qdrant payload index
before `MatchText`/prefix matching is performant; `tags` and `category` are not in the payload at before `MatchText`/prefix matching is performant; `tags` and `category` are not in the payload at
all and require extending `processor.py` plus a full re-index. Conflating these with the date all and require extending `processor.py` plus a full re-index. Conflating these with the date
@@ -83,40 +100,97 @@ Two consequences:
## Decision ## Decision
### 1. Generalise the filter contract ### 1. Generalise the filter contract through one shared helper
Every structured filter follows the `doc_type` path: **tool parameter → `search()` keyword arg → Every structured filter follows the `doc_type` path: **tool parameter → explicit `search()`
`FieldCondition` appended to `filter_conditions` → `Filter(must=[...])` on both prefetch branches.** keyword arg → `FieldCondition` in the shared `filter_conditions` builder → `Filter(must=[...])`
Filters are always applied at the Qdrant layer, **before** verify-on-read (ADR-019), so that on both prefetch branches.** Filters are always applied at the Qdrant layer, **before**
`verified_chunk_count` / `dropped_document_count` describe the already-filtered set and the verifier verify-on-read (ADR-019), so that `verified_chunk_count` / `dropped_document_count` describe the
never wastes Nextcloud round-trips on documents the filter excluded. already-filtered set and the verifier never wastes Nextcloud round-trips on documents the filter
excluded.
Date/range bounds use `qdrant_client.models.Range`: **The filter is added to the `SearchAlgorithm` ABC contract, not just one algorithm.** The two
algorithms that back the search surfaces — `BM25HybridSearchAlgorithm` (the MCP tool path,
`nc_semantic_search`) and `SemanticSearchAlgorithm` (the dense-only visualization / `/api/v1`
path, `api/visualization.py` + `auth/viz_routes.py`) — today build a *byte-identical*
placeholder + ownership + `doc_type` filter block. Rather than copy the new condition into both,
the common block moves into one helper, `search/access_filter.py::build_base_filter_conditions`,
and both algorithms call it. New filters are therefore honoured on **every** path (hybrid and
dense-only) by editing one function:
```python ```python
from qdrant_client.models import FieldCondition, Range # search/access_filter.py — the single source of the filter contract
def build_base_filter_conditions(
if modified_after is not None or modified_before is not None: user_id, accessible_owners=None, doc_type=None,
filter_conditions.append( modified_after=None, modified_before=None,
) -> list[Condition]:
conditions = [get_placeholder_filter(), build_ownership_filter(user_id, accessible_owners)]
if doc_type:
conditions.append(FieldCondition(key="doc_type", match=MatchValue(value=doc_type)))
if modified_after is not None or modified_before is not None:
conditions.append(
FieldCondition( FieldCondition(
key="modified_at", key="modified_at",
range=Range(gte=modified_after, lte=modified_before), # None bounds are open-ended range=Range(gte=modified_after, lte=modified_before), # None bounds are open-ended
) )
) )
return conditions
``` ```
`Range` treats `None` bounds as open, so the same condition serves after-only, before-only, and `Range` treats `None` bounds as open, so the same condition serves after-only, before-only, and
both-bounds queries. Validation that `modified_after <= modified_before` lives in the Pydantic both-bounds queries. New range/match filters are added here once; each algorithm wraps the
request model, not the algorithm. returned list in `Filter(must=...)` and may append its own additive conditions afterward (e.g.
`SemanticSearchAlgorithm`'s opt-in ACL pre-filter, which `BM25HybridSearchAlgorithm` does **not**
apply).
`modified_after` / `modified_before` are promoted to **explicit named keyword params** on
`SearchAlgorithm.search()` (and both concrete impls), exactly as `accessible_owners` was — not
left in `**kwargs`. Explicit params keep them discoverable and make a misspelled keyword a type
error instead of a silently-ignored filter. When `doc_types` is a list, the tool's per-type
dispatch loop forwards the same `modified_after`/`modified_before` to each per-type
`search()` call.
**Input validation.** FastMCP exposes no request-model object to hang a Pydantic
`@model_validator` on, so validation is split across three layers, each handling what it can
express:
- **Per-argument scalar bounds** use `Annotated[..., Field(...)]` on the tool signature — FastMCP
builds the input schema from these and rejects bad values before the body runs. This is the
repo's first use of `Annotated`/`Field` on a tool; the existing numeric knobs (`limit` `ge=1`,
`score_threshold` `0.01.0`, `context_chars` `ge=0`) are tightened the same way and the pattern
is what future filters reuse.
- **Format normalization for the date bounds.** `modified_after` / `modified_before` are typed
`str | int | None` and accept an **RFC 3339 / ISO 8601 datetime** (e.g. `"2026-01-01T00:00:00Z"`,
naive ⇒ UTC) *or* Unix seconds. A shared `utils/validation.py::parse_modified_timestamp` helper
normalizes either to int Unix seconds (the payload representation) and raises `ValueError` on an
unparseable value; the tool converts that to `McpError`, the HTTP endpoints to a 400. The same
helper is reused by `nc_semantic_search`, the `/api/v1` search endpoints, and the viz route so
every surface accepts identical formats.
- **The cross-field invariant `modified_after <= modified_before`** can't be a per-field
constraint, so it is an explicit guard (on the *parsed* int values) that raises
`McpError(ErrorData(code=-1, ...))` in the tool — the established input-error idiom (mirrors the
`VECTOR_SYNC_ENABLED` guard) — and a 400 on the HTTP endpoints.
### 2. Phase the rollout by payload readiness ### 2. Phase the rollout by payload readiness
- **Phase 1 — modified-date range (this ADR's committed scope).** Add `modified_after` / - **Phase 1 — modified-date range (this ADR's committed scope).** Add `modified_after` /
`modified_before` (Unix seconds, UTC) to `nc_semantic_search` and `bm25_hybrid.search()`. No `modified_before` (RFC 3339 / ISO 8601 at the boundary, Unix seconds accepted too) to the
re-index. Ship the frontend chip UX against this plus the existing doc-type filter to prove the `SearchAlgorithm` contract and both algorithm impls (so the MCP tool *and* the dense-only
end-to-end plumbing on fields that already exist. `/api/v1` path honour it), plus a `modified_at` INTEGER payload index. No content re-index. Ship
the frontend chip UX against this plus the existing doc-type filter to prove the end-to-end
plumbing on fields that already exist.
- **`nc_semantic_search_answer` is explicitly deferred, not part of Phase 1.** It is a thin RAG
wrapper that today threads only `query`/`limit`/`score_threshold`/`fusion`/context options to
`nc_semantic_search` and does not even expose `doc_types` — it always searches cross-app. Date
scoping is a search-box affordance, not an answer-synthesis one, and there is no UI surface for
it on the answer path. When demand appears it can be threaded through using exactly the same
parameter + `parse_modified_timestamp` pattern; doing it now would add an unused parameter and
widen the change for no user-visible gain.
- **Phase 2 — directory / path.** Create a Qdrant payload index on `file_path`, add a `path_prefix` - **Phase 2 — directory / path.** Create a Qdrant payload index on `file_path`, add a `path_prefix`
parameter, and add an `NcFilePicker` folder chooser. Scoped to `doc_type == "file"`. parameter, and add an `NcFilePicker` folder chooser. Scoped to `doc_type == "file"`: the frontend
must **hide/disable the path filter unless the file doc type is selected**, because `file_path`
is only written to the payload for `doc_type == "file"` (`processor.py`) — a path condition on
any other type matches nothing and silently returns zero results.
- **Phase 3 — tags (and optionally category).** Add a `tags: list[str]` payload field in - **Phase 3 — tags (and optionally category).** Add a `tags: list[str]` payload field in
`processor.py`, propagate Nextcloud system tags during scanning, trigger a re-index, then wire `processor.py`, propagate Nextcloud system tags during scanning, trigger a re-index, then wire
`NcSelectTags` (`MatchAny` over tags). Re-index cost lives here, isolated from the cheap wins. `NcSelectTags` (`MatchAny` over tags). Re-index cost lives here, isolated from the cheap wins.
@@ -124,17 +198,32 @@ request model, not the algorithm.
### 3. Frontend: filter chips, structured payload ### 3. Frontend: filter chips, structured payload
The Astrolabe app adds filter controls to the existing collapsible advanced panel and renders each The Astrolabe app adds filter controls to the existing collapsible advanced panel and renders each
**active** filter as a closable `NcChip` (the same component Nextcloud Unified Search uses): **active** filter as a closable chip:
- Modified-date range → `NcDateTimePicker type="datetime-range"` (model is `[Date, Date]`). - Modified-date range → two native `<input type="datetime-local">` fields (a local wall-clock
picker the browser validates), serialized to RFC 3339 UTC via `Date.toISOString()`. Phase 1
deliberately uses native inputs + lightweight CSS chips rather than taking a hard dependency on a
specific `@nextcloud/vue` component version (`NcDateTimePicker` / `NcChip` remain a later option);
this matches the component's existing native `<input type="range">` controls.
- Doc types → existing checkbox grid, now also echoed as chips. - Doc types → existing checkbox grid, now also echoed as chips.
- (Phase 2/3) path → `NcFilePicker`; tags → `NcSelectTags :fetch-tags`. - (Phase 2/3) path → `NcFilePicker`; tags → `NcSelectTags :fetch-tags`.
The `/apps/astrolabe/api/search` endpoint moves from `GET` with query params to **`POST` with a JSON Dates cross the wire as **RFC 3339 / ISO 8601 strings** (e.g. `"2026-01-01T00:00:00Z"`) — the format
body**, because the filter set is structured and multi-valued and will keep growing. Dates are sent Nextcloud's date pickers and Unified Search use. The MCP/HTTP boundary parses RFC 3339 (and bare
as **Unix seconds (UTC)** to match the `modified_at` payload representation exactly — no timezone or Unix seconds, for resilience) to int Unix seconds via `parse_modified_timestamp` before filtering,
string-parsing ambiguity crosses the wire. Empty or partially-filled filters are omitted from the so the friendly wire format never forces a re-index of the integer `modified_at` payload.
body rather than sent as nulls.
**Transport.** Phase 1 keeps the existing `GET /apps/astrolabe/api/search`: the two scalar date
strings ride as query params alongside the already comma-joined `doc_types`, so no transport change
is needed yet. The move to **`POST` with a JSON body** is deferred to Phase 2, when the filter set
becomes genuinely structured/multi-valued (path + tags) and outgrows query params.
The **Astrolabe UI validates the date range client-side** — the native picker constrains each field
to a real datetime, and `performSearch` rejects an *after > before* range before issuing the request
(`McpServerClient` forwards the RFC 3339 strings; `ApiController` re-validates and returns a 400 on a
bad/inverted range). The server-side `parse_modified_timestamp` + cross-field guard is the
authoritative backstop. (Astrolabe-side work is tracked on Deck card **#177**, label
`repo:astrolabe`.)
## Consequences ## Consequences
@@ -150,8 +239,9 @@ body rather than sent as nulls.
- Path and tag filters require index work (a payload index; a new payload field + full re-index) - Path and tag filters require index work (a payload index; a new payload field + full re-index)
that this ADR explicitly defers — the readiness table makes that cost visible rather than implicit. that this ADR explicitly defers — the readiness table makes that cost visible rather than implicit.
- Moving `/api/search` to POST is a breaking change to that endpoint's contract; the PHP app and the - The eventual `/api/search` GET→POST migration (Phase 2) will be a breaking change to that
MCP backend must ship together. endpoint's contract; the PHP app and the MCP backend must ship together when it lands. Phase 1
avoids it by keeping GET.
- Pre-fusion filtering on a very selective `Range` can still under-fill `limit` if the candidate - Pre-fusion filtering on a very selective `Range` can still under-fill `limit` if the candidate
pool (`limit * 2`) is exhausted; if this proves a problem we revisit the prefetch multiplier pool (`limit * 2`) is exhausted; if this proves a problem we revisit the prefetch multiplier
rather than filtering post-fusion. rather than filtering post-fusion.
+49 -1
View File
@@ -37,7 +37,10 @@ from nextcloud_mcp_server.search.context import (
get_chunk_with_context, get_chunk_with_context,
) )
from nextcloud_mcp_server.search.verification import verify_search_results from nextcloud_mcp_server.search.verification import verify_search_results
from nextcloud_mcp_server.utils.validation import is_valid_nextcloud_doc_id from nextcloud_mcp_server.utils.validation import (
is_valid_nextcloud_doc_id,
parse_modified_timestamp,
)
from nextcloud_mcp_server.vector.oauth_sync import ( from nextcloud_mcp_server.vector.oauth_sync import (
NotProvisionedError, NotProvisionedError,
get_user_client_basic_auth, get_user_client_basic_auth,
@@ -198,6 +201,22 @@ async def unified_search(request: Request) -> JSONResponse:
1.0, 1.0,
"score_threshold", "score_threshold",
) )
# ADR-027 modified-date range filter. Accepts RFC 3339 / ISO 8601
# datetimes or Unix seconds; normalized to int Unix seconds for the
# numeric Range filter. Absent bound ⇒ open-ended.
modified_after = parse_modified_timestamp(
body.get("modified_after"), param_name="modified_after"
)
modified_before = parse_modified_timestamp(
body.get("modified_before"), param_name="modified_before"
)
if (
modified_after is not None
and modified_before is not None
and modified_after > modified_before
):
raise ValueError("modified_after must be <= modified_before")
except ValueError as e: except ValueError as e:
return JSONResponse({"error": str(e)}, status_code=400) return JSONResponse({"error": str(e)}, status_code=400)
@@ -245,6 +264,8 @@ async def unified_search(request: Request) -> JSONResponse:
limit=search_limit, limit=search_limit,
doc_type=doc_type, doc_type=doc_type,
accessible_owners=owners, accessible_owners=owners,
modified_after=modified_after,
modified_before=modified_before,
) )
) )
# Sort, then cap to a fixed over-fetch budget before the result # Sort, then cap to a fixed over-fetch budget before the result
@@ -262,6 +283,8 @@ async def unified_search(request: Request) -> JSONResponse:
user_id=user_id, user_id=user_id,
limit=search_limit, limit=search_limit,
accessible_owners=owners, accessible_owners=owners,
modified_after=modified_after,
modified_before=modified_before,
) )
return results return results
@@ -413,6 +436,17 @@ async def vector_search(request: Request) -> JSONResponse:
limit = min(body.get("limit", 10), 50) # Enforce max limit limit = min(body.get("limit", 10), 50) # Enforce max limit
include_pca = body.get("include_pca", True) include_pca = body.get("include_pca", True)
doc_types = body.get("doc_types") # Optional list of document types doc_types = body.get("doc_types") # Optional list of document types
# ADR-027 modified-date range filter. Accepts RFC 3339 / ISO 8601
# datetimes or Unix seconds; normalized to int Unix seconds. None ⇒ open.
try:
modified_after = parse_modified_timestamp(
body.get("modified_after"), param_name="modified_after"
)
modified_before = parse_modified_timestamp(
body.get("modified_before"), param_name="modified_before"
)
except ValueError as e:
return JSONResponse({"error": str(e)}, status_code=400)
if not query: if not query:
return JSONResponse( return JSONResponse(
@@ -420,6 +454,16 @@ async def vector_search(request: Request) -> JSONResponse:
status_code=400, status_code=400,
) )
if (
modified_after is not None
and modified_before is not None
and modified_after > modified_before
):
return JSONResponse(
{"error": "modified_after must be <= modified_before"},
status_code=400,
)
# Validate algorithm # Validate algorithm
valid_algorithms = {"semantic", "bm25", "hybrid"} valid_algorithms = {"semantic", "bm25", "hybrid"}
if algorithm not in valid_algorithms: if algorithm not in valid_algorithms:
@@ -455,6 +499,8 @@ async def vector_search(request: Request) -> JSONResponse:
limit=limit, limit=limit,
doc_type=doc_type, doc_type=doc_type,
accessible_owners=owners, accessible_owners=owners,
modified_after=modified_after,
modified_before=modified_before,
) )
) )
# Sort merged results by score and limit # Sort merged results by score and limit
@@ -467,6 +513,8 @@ async def vector_search(request: Request) -> JSONResponse:
user_id=user_id, user_id=user_id,
limit=limit, limit=limit,
accessible_owners=owners, accessible_owners=owners,
modified_after=modified_after,
modified_before=modified_before,
) )
return results return results
+33 -1
View File
@@ -39,7 +39,10 @@ from nextcloud_mcp_server.search.context import (
get_chunk_with_context, get_chunk_with_context,
) )
from nextcloud_mcp_server.search.verification import verify_search_results from nextcloud_mcp_server.search.verification import verify_search_results
from nextcloud_mcp_server.utils.validation import is_valid_nextcloud_doc_id from nextcloud_mcp_server.utils.validation import (
is_valid_nextcloud_doc_id,
parse_modified_timestamp,
)
from nextcloud_mcp_server.vector.oauth_sync import ( from nextcloud_mcp_server.vector.oauth_sync import (
NotProvisionedError, NotProvisionedError,
get_user_client_basic_auth, get_user_client_basic_auth,
@@ -141,6 +144,31 @@ async def vector_visualization_search(request: Request) -> JSONResponse:
doc_types_param = request.query_params.get("doc_types", "") doc_types_param = request.query_params.get("doc_types", "")
doc_types = doc_types_param.split(",") if doc_types_param else None doc_types = doc_types_param.split(",") if doc_types_param else None
# Parse ADR-027 modified-date range filter. Accepts RFC 3339 / ISO 8601
# datetimes or Unix seconds; normalized to int Unix seconds. Absent ⇒
# open-ended. Unparseable input or an inverted range returns 400.
try:
modified_after = parse_modified_timestamp(
request.query_params.get("modified_after"), param_name="modified_after"
)
modified_before = parse_modified_timestamp(
request.query_params.get("modified_before"), param_name="modified_before"
)
except ValueError as exc:
return JSONResponse(
{"success": False, "error": str(exc)},
status_code=400,
)
if (
modified_after is not None
and modified_before is not None
and modified_after > modified_before
):
return JSONResponse(
{"success": False, "error": "modified_after must be <= modified_before"},
status_code=400,
)
logger.info( logger.info(
"Viz search: user=%s, query='%s', algorithm=%s, fusion=%s, limit=%s, doc_types=%s", "Viz search: user=%s, query='%s', algorithm=%s, fusion=%s, limit=%s, doc_types=%s",
username, username,
@@ -202,6 +230,8 @@ async def vector_visualization_search(request: Request) -> JSONResponse:
doc_type=None, # Search all types doc_type=None, # Search all types
score_threshold=score_threshold, score_threshold=score_threshold,
accessible_owners=accessible_owners, accessible_owners=accessible_owners,
modified_after=modified_after,
modified_before=modified_before,
) )
all_results.extend(unverified_results) all_results.extend(unverified_results)
else: else:
@@ -222,6 +252,8 @@ async def vector_visualization_search(request: Request) -> JSONResponse:
doc_type=doc_type, doc_type=doc_type,
score_threshold=score_threshold, score_threshold=score_threshold,
accessible_owners=accessible_owners, accessible_owners=accessible_owners,
modified_after=modified_after,
modified_before=modified_before,
) )
all_results.extend(unverified_results) all_results.extend(unverified_results)
# Sort by score, then cap to the same limit*2 over-fetch budget # Sort by score, then cap to the same limit*2 over-fetch budget
+67 -1
View File
@@ -29,7 +29,16 @@ import time
from collections import OrderedDict from collections import OrderedDict
from typing import Any, Protocol from typing import Any, Protocol
from qdrant_client.models import Condition, FieldCondition, Filter, MatchAny, MatchValue from qdrant_client.models import (
Condition,
FieldCondition,
Filter,
MatchAny,
MatchValue,
Range,
)
from nextcloud_mcp_server.vector.placeholder import get_placeholder_filter
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -177,3 +186,60 @@ def build_ownership_filter(
0, FieldCondition(key="owner_id", match=MatchAny(any=other_owners)) 0, FieldCondition(key="owner_id", match=MatchAny(any=other_owners))
) )
return Filter(should=conditions) return Filter(should=conditions)
def build_base_filter_conditions(
user_id: str,
accessible_owners: list[str] | None = None,
doc_type: str | None = None,
modified_after: int | None = None,
modified_before: int | None = None,
) -> list[Condition]:
"""Build the common ``must`` conditions shared by every search algorithm.
This is the single place the structured-filter contract (ADR-027) lives, so
both the BM25-hybrid (MCP tool) and dense-only (visualization/API) algorithms
apply identical placeholder/ACL/doc_type/date filtering. Each algorithm wraps
the returned list in ``Filter(must=...)`` and may append its own additive
conditions afterward (e.g. the dense algorithm's opt-in ACL pre-filter).
The conditions, in order:
1. ``get_placeholder_filter()`` — exclude in-flight placeholder points.
2. ``build_ownership_filter(...)`` — ACL-aware ``owner_id``/``user_id`` scope.
3. ``doc_type`` exact match — only when ``doc_type`` is truthy.
4. ``modified_at`` range — only when at least one bound is given.
Args:
user_id: Querying user.
accessible_owners: Owner UIDs the user can read (see
``build_ownership_filter``). ``None`` ⇒ self-only.
doc_type: Optional single document-type filter.
modified_after: Inclusive lower bound on ``modified_at`` (Unix seconds).
modified_before: Inclusive upper bound on ``modified_at`` (Unix seconds).
Returns:
A list of Qdrant ``Condition`` objects for a parent ``must`` clause.
"""
conditions: list[Condition] = [
get_placeholder_filter(),
build_ownership_filter(user_id, accessible_owners),
]
if doc_type:
conditions.append(
FieldCondition(key="doc_type", match=MatchValue(value=doc_type))
)
# ``Range`` treats ``None`` bounds as open-ended, so the same condition serves
# after-only, before-only, and both-bounds queries. Appended only when at
# least one bound is set so unfiltered searches add no condition.
if modified_after is not None or modified_before is not None:
conditions.append(
FieldCondition(
key="modified_at",
range=Range(gte=modified_after, lte=modified_before),
)
)
return conditions
@@ -289,6 +289,8 @@ class SearchAlgorithm(ABC):
doc_type: str | None = None, doc_type: str | None = None,
*, *,
accessible_owners: list[str] | None = None, accessible_owners: list[str] | None = None,
modified_after: int | None = None,
modified_before: int | None = None,
**kwargs: Any, **kwargs: Any,
) -> list[SearchResult]: ) -> list[SearchResult]:
"""Execute search with the given parameters. """Execute search with the given parameters.
@@ -304,6 +306,12 @@ class SearchAlgorithm(ABC):
buried in ``**kwargs`` — so a misspelled keyword is a type error buried in ``**kwargs`` — so a misspelled keyword is a type error
instead of a silent fall back to self-only scope. ``None`` means instead of a silent fall back to self-only scope. ``None`` means
self-only (``[user_id]``). self-only (``[user_id]``).
modified_after: Optional inclusive lower bound on the document's
``modified_at`` payload field (Unix seconds, UTC). Declared
explicitly for the same discoverability/type-safety reason as
``accessible_owners`` (ADR-027). ``None`` ⇒ open-ended.
modified_before: Optional inclusive upper bound on ``modified_at``
(Unix seconds, UTC). ``None`` ⇒ open-ended.
**kwargs: Algorithm-specific parameters **kwargs: Algorithm-specific parameters
Returns: Returns:
+17 -16
View File
@@ -4,19 +4,18 @@ import logging
from typing import Any from typing import Any
from qdrant_client import models from qdrant_client import models
from qdrant_client.models import FieldCondition, Filter, MatchValue from qdrant_client.models import Filter
from nextcloud_mcp_server.config import get_settings from nextcloud_mcp_server.config import get_settings
from nextcloud_mcp_server.embedding import get_bm25_service, get_embedding_service from nextcloud_mcp_server.embedding import get_bm25_service, get_embedding_service
from nextcloud_mcp_server.observability.metrics import record_qdrant_operation from nextcloud_mcp_server.observability.metrics import record_qdrant_operation
from nextcloud_mcp_server.observability.tracing import trace_operation from nextcloud_mcp_server.observability.tracing import trace_operation
from nextcloud_mcp_server.search.access_filter import build_ownership_filter from nextcloud_mcp_server.search.access_filter import build_base_filter_conditions
from nextcloud_mcp_server.search.algorithms import ( from nextcloud_mcp_server.search.algorithms import (
SearchAlgorithm, SearchAlgorithm,
SearchResult, SearchResult,
build_search_result_from_point, build_search_result_from_point,
) )
from nextcloud_mcp_server.vector.placeholder import get_placeholder_filter
from nextcloud_mcp_server.vector.qdrant_client import get_qdrant_client from nextcloud_mcp_server.vector.qdrant_client import get_qdrant_client
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -73,6 +72,8 @@ class BM25HybridSearchAlgorithm(SearchAlgorithm):
doc_type: str | None = None, doc_type: str | None = None,
*, *,
accessible_owners: list[str] | None = None, accessible_owners: list[str] | None = None,
modified_after: int | None = None,
modified_before: int | None = None,
**kwargs: Any, **kwargs: Any,
) -> list[SearchResult]: ) -> list[SearchResult]:
""" """
@@ -94,6 +95,10 @@ class BM25HybridSearchAlgorithm(SearchAlgorithm):
accessible_owners: Owner UIDs the user can read (self + share accessible_owners: Owner UIDs the user can read (self + share
senders), pre-computed by the caller from the OCS Sharing API. senders), pre-computed by the caller from the OCS Sharing API.
Defaults to ``[user_id]`` (self-only) when ``None``. Defaults to ``[user_id]`` (self-only) when ``None``.
modified_after: Inclusive lower bound on ``modified_at`` (Unix
seconds, UTC); ``None`` ⇒ open-ended (ADR-027).
modified_before: Inclusive upper bound on ``modified_at`` (Unix
seconds, UTC); ``None`` ⇒ open-ended (ADR-027).
**kwargs: Additional parameters (score_threshold override) **kwargs: Additional parameters (score_threshold override)
Returns: Returns:
@@ -134,19 +139,15 @@ class BM25HybridSearchAlgorithm(SearchAlgorithm):
len(sparse_embedding["indices"]), len(sparse_embedding["indices"]),
) )
# Build Qdrant filter # Build Qdrant filter (placeholder + ACL + doc_type + modified_at range).
filter_conditions = [ # Shared with the dense-only SemanticSearchAlgorithm via the common
get_placeholder_filter(), # Always exclude placeholders from user-facing queries # ADR-027 helper so every search surface applies one filter contract.
build_ownership_filter(user_id, accessible_owners), filter_conditions = build_base_filter_conditions(
] user_id=user_id,
accessible_owners=accessible_owners,
# Add doc_type filter if specified doc_type=doc_type,
if doc_type: modified_after=modified_after,
filter_conditions.append( modified_before=modified_before,
FieldCondition(
key="doc_type",
match=MatchValue(value=doc_type),
)
) )
query_filter = Filter(must=filter_conditions) query_filter = Filter(must=filter_conditions)
+18 -16
View File
@@ -3,20 +3,19 @@
import logging import logging
from typing import Any from typing import Any
from qdrant_client.models import FieldCondition, Filter, MatchAny, MatchValue from qdrant_client.models import FieldCondition, Filter, MatchAny
from nextcloud_mcp_server.acl_hash import accessible_hash_set from nextcloud_mcp_server.acl_hash import accessible_hash_set
from nextcloud_mcp_server.config import get_settings from nextcloud_mcp_server.config import get_settings
from nextcloud_mcp_server.embedding import get_embedding_service from nextcloud_mcp_server.embedding import get_embedding_service
from nextcloud_mcp_server.observability.metrics import record_qdrant_operation from nextcloud_mcp_server.observability.metrics import record_qdrant_operation
from nextcloud_mcp_server.search.access_filter import build_ownership_filter from nextcloud_mcp_server.search.access_filter import build_base_filter_conditions
from nextcloud_mcp_server.search.algorithms import ( from nextcloud_mcp_server.search.algorithms import (
SearchAlgorithm, SearchAlgorithm,
SearchResult, SearchResult,
build_search_result_from_point, build_search_result_from_point,
) )
from nextcloud_mcp_server.vector.payload_keys import ACL_HASH from nextcloud_mcp_server.vector.payload_keys import ACL_HASH
from nextcloud_mcp_server.vector.placeholder import get_placeholder_filter
from nextcloud_mcp_server.vector.qdrant_client import get_qdrant_client from nextcloud_mcp_server.vector.qdrant_client import get_qdrant_client
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -53,6 +52,8 @@ class SemanticSearchAlgorithm(SearchAlgorithm):
doc_type: str | None = None, doc_type: str | None = None,
*, *,
accessible_owners: list[str] | None = None, accessible_owners: list[str] | None = None,
modified_after: int | None = None,
modified_before: int | None = None,
**kwargs: Any, **kwargs: Any,
) -> list[SearchResult]: ) -> list[SearchResult]:
"""Execute semantic search using vector similarity. """Execute semantic search using vector similarity.
@@ -73,6 +74,10 @@ class SemanticSearchAlgorithm(SearchAlgorithm):
accessible_owners: Owner UIDs the user can read (self + share accessible_owners: Owner UIDs the user can read (self + share
senders), pre-computed by the caller from the OCS Sharing API. senders), pre-computed by the caller from the OCS Sharing API.
Defaults to ``[user_id]`` (self-only) when ``None``. Defaults to ``[user_id]`` (self-only) when ``None``.
modified_after: Inclusive lower bound on ``modified_at`` (Unix
seconds, UTC); ``None`` ⇒ open-ended (ADR-027).
modified_before: Inclusive upper bound on ``modified_at`` (Unix
seconds, UTC); ``None`` ⇒ open-ended (ADR-027).
**kwargs: **kwargs:
- score_threshold (float): override the instance default - score_threshold (float): override the instance default
@@ -103,19 +108,16 @@ class SemanticSearchAlgorithm(SearchAlgorithm):
"Generated embedding for query (dimension=%s)", len(query_embedding) "Generated embedding for query (dimension=%s)", len(query_embedding)
) )
# Build Qdrant filter # Build Qdrant filter (placeholder + ACL + doc_type + modified_at range).
filter_conditions = [ # Shared with BM25HybridSearchAlgorithm via the common ADR-027 helper so
get_placeholder_filter(), # Always exclude placeholders from user-facing queries # the dense-only (API/visualization) and hybrid (MCP tool) paths apply
build_ownership_filter(user_id, accessible_owners), # one filter contract.
] filter_conditions = build_base_filter_conditions(
user_id=user_id,
# Add doc_type filter if specified accessible_owners=accessible_owners,
if doc_type: doc_type=doc_type,
filter_conditions.append( modified_after=modified_after,
FieldCondition( modified_before=modified_before,
key="doc_type",
match=MatchValue(value=doc_type),
)
) )
# ACL pre-filter (design §11), opt-in via ACL_PREFILTER_ENABLED and OFF # ACL pre-filter (design §11), opt-in via ACL_PREFILTER_ENABLED and OFF
+71 -3
View File
@@ -1,6 +1,7 @@
"""Semantic search MCP tools using vector database.""" """Semantic search MCP tools using vector database."""
import logging import logging
from typing import Annotated
import anyio import anyio
from httpx import RequestError from httpx import RequestError
@@ -16,6 +17,7 @@ from mcp.types import (
TextContent, TextContent,
ToolAnnotations, ToolAnnotations,
) )
from pydantic import Field
from qdrant_client.models import Filter from qdrant_client.models import Filter
from nextcloud_mcp_server.auth import require_scopes from nextcloud_mcp_server.auth import require_scopes
@@ -34,6 +36,7 @@ from nextcloud_mcp_server.search.access_filter import list_accessible_owners
from nextcloud_mcp_server.search.bm25_hybrid import BM25HybridSearchAlgorithm from nextcloud_mcp_server.search.bm25_hybrid import BM25HybridSearchAlgorithm
from nextcloud_mcp_server.search.context import get_chunk_with_context from nextcloud_mcp_server.search.context import get_chunk_with_context
from nextcloud_mcp_server.search.verification import verify_search_results from nextcloud_mcp_server.search.verification import verify_search_results
from nextcloud_mcp_server.utils.validation import parse_modified_timestamp
from nextcloud_mcp_server.vector.placeholder import get_placeholder_filter from nextcloud_mcp_server.vector.placeholder import get_placeholder_filter
from nextcloud_mcp_server.vector.qdrant_client import get_qdrant_client from nextcloud_mcp_server.vector.qdrant_client import get_qdrant_client
@@ -55,12 +58,32 @@ def configure_semantic_tools(mcp: FastMCP):
async def nc_semantic_search( async def nc_semantic_search(
query: str, query: str,
ctx: Context, ctx: Context,
limit: int = 10, limit: Annotated[int, Field(ge=1, le=100)] = 10,
doc_types: list[str] | None = None, doc_types: list[str] | None = None,
score_threshold: float = 0.0, score_threshold: Annotated[float, Field(ge=0.0, le=1.0)] = 0.0,
fusion: str = "rrf", fusion: str = "rrf",
include_context: bool = False, include_context: bool = False,
context_chars: int = 300, context_chars: Annotated[int, Field(ge=0)] = 300,
modified_after: Annotated[
str | int | None,
Field(
description=(
"Only return documents modified at or after this time. "
"RFC 3339 / ISO 8601 datetime (e.g. '2026-01-01T00:00:00Z') "
"or Unix seconds. None = no lower bound."
),
),
] = None,
modified_before: Annotated[
str | int | None,
Field(
description=(
"Only return documents modified at or before this time. "
"RFC 3339 / ISO 8601 datetime or Unix seconds. "
"None = no upper bound."
),
),
] = None,
) -> SemanticSearchResponse: ) -> SemanticSearchResponse:
""" """
Search Nextcloud content using BM25 hybrid search with cross-app support. Search Nextcloud content using BM25 hybrid search with cross-app support.
@@ -86,6 +109,13 @@ def configure_semantic_tools(mcp: FastMCP):
DBSF: Uses distribution-based normalization, may better balance different score ranges DBSF: Uses distribution-based normalization, may better balance different score ranges
include_context: Whether to expand results with surrounding context (default: False) include_context: Whether to expand results with surrounding context (default: False)
context_chars: Number of characters to include before/after matched chunk (default: 300) context_chars: Number of characters to include before/after matched chunk (default: 300)
modified_after: Only return documents whose last-modified time is at or after this
instant. Accepts an RFC 3339 / ISO 8601 datetime (e.g. "2026-01-01T00:00:00Z";
a naive datetime is treated as UTC) or Unix seconds. None = no lower bound
(default).
modified_before: Only return documents whose last-modified time is at or before this
instant. Same formats as modified_after. None = no upper bound (default). Must be
>= modified_after when both are supplied.
Returns: Returns:
SemanticSearchResponse with matching documents ranked by fusion scores. SemanticSearchResponse with matching documents ranked by fusion scores.
@@ -122,6 +152,40 @@ def configure_semantic_tools(mcp: FastMCP):
) )
) )
# Normalize the RFC 3339 / Unix-seconds date bounds to int Unix seconds
# for the numeric ``modified_at`` Range filter (ADR-027). A bad format
# surfaces as a clean McpError rather than a 500.
try:
modified_after_ts = parse_modified_timestamp(
modified_after, param_name="modified_after"
)
modified_before_ts = parse_modified_timestamp(
modified_before, param_name="modified_before"
)
except ValueError as exc:
raise McpError(ErrorData(code=-1, message=str(exc))) from exc
# Cross-field invariant: a per-parameter pydantic ``Field`` constraint
# (validated by FastMCP from the signature) bounds each date on its own
# but cannot express the relationship between them. Guard it here so an
# inverted range surfaces a clean McpError rather than silently
# returning zero results (ADR-027).
if (
modified_after_ts is not None
and modified_before_ts is not None
and modified_after_ts > modified_before_ts
):
raise McpError(
ErrorData(
code=-1,
message=(
"modified_after must be <= modified_before "
f"(got modified_after={modified_after!r}, "
f"modified_before={modified_before!r})"
),
)
)
# Expand the caller's identity to every owner whose content they # Expand the caller's identity to every owner whose content they
# have read access to via Nextcloud shares. Lets a user find files # have read access to via Nextcloud shares. Lets a user find files
# owners have shared with them without having to re-index those # owners have shared with them without having to re-index those
@@ -166,6 +230,8 @@ def configure_semantic_tools(mcp: FastMCP):
doc_type=None, # Signal to search all types doc_type=None, # Signal to search all types
score_threshold=score_threshold, score_threshold=score_threshold,
accessible_owners=accessible_owners, accessible_owners=accessible_owners,
modified_after=modified_after_ts,
modified_before=modified_before_ts,
) )
all_results.extend(unverified_results) all_results.extend(unverified_results)
else: else:
@@ -191,6 +257,8 @@ def configure_semantic_tools(mcp: FastMCP):
doc_type=dtype, doc_type=dtype,
score_threshold=score_threshold, score_threshold=score_threshold,
accessible_owners=accessible_owners, accessible_owners=accessible_owners,
modified_after=modified_after_ts,
modified_before=modified_before_ts,
) )
all_results.extend(unverified_results) all_results.extend(unverified_results)
+71
View File
@@ -1,6 +1,7 @@
"""Shared validators for primitive types crossing system boundaries.""" """Shared validators for primitive types crossing system boundaries."""
import re import re
from datetime import datetime, timezone
# Nextcloud object IDs are unsigned ints from MySQL AUTO_INCREMENT, which # Nextcloud object IDs are unsigned ints from MySQL AUTO_INCREMENT, which
# starts at 1. Restrict to ASCII positive integers to exclude Unicode digit # starts at 1. Restrict to ASCII positive integers to exclude Unicode digit
@@ -9,7 +10,77 @@ import re
# and leading zeros. # and leading zeros.
_NEXTCLOUD_DOC_ID_RE = re.compile(r"^[1-9][0-9]*$") _NEXTCLOUD_DOC_ID_RE = re.compile(r"^[1-9][0-9]*$")
# A bare non-negative integer string is accepted as Unix seconds (the
# pre-RFC-3339 wire format) so older callers keep working.
_UNIX_SECONDS_RE = re.compile(r"^[0-9]+$")
def is_valid_nextcloud_doc_id(value: str) -> bool: def is_valid_nextcloud_doc_id(value: str) -> bool:
"""True iff `value` is the str form of a positive ASCII integer (>= 1).""" """True iff `value` is the str form of a positive ASCII integer (>= 1)."""
return bool(_NEXTCLOUD_DOC_ID_RE.fullmatch(value)) return bool(_NEXTCLOUD_DOC_ID_RE.fullmatch(value))
def parse_modified_timestamp(
value: str | int | float | None,
*,
param_name: str = "modified_at",
) -> int | None:
"""Normalize a search date-filter bound to an int Unix-second timestamp.
ADR-027: callers (the MCP tool, the ``/api/v1`` search endpoints, and the
visualization route) accept **RFC 3339 / ISO 8601** datetimes at the
boundary — the ergonomic, Nextcloud-Unified-Search-style format — while the
``modified_at`` Qdrant payload stays an int Unix-second timestamp (a
cross-app normalization done by the scanner). This converts the former to
the latter so the numeric ``Range`` filter and INTEGER payload index work
without re-indexing.
Accepts:
- ``None`` / empty string ⇒ ``None`` (open-ended bound).
- ``int`` / ``float`` ⇒ truncated to int seconds.
- A bare non-negative integer string ⇒ Unix seconds (legacy wire format).
- An RFC 3339 / ISO 8601 string, e.g. ``"2026-01-01T00:00:00Z"`` or
``"2026-01-01T00:00:00+02:00"``. A naive datetime (no offset) is assumed
to be UTC, matching the payload representation.
Args:
value: The raw bound from the request.
param_name: Field name used in error messages.
Returns:
Int Unix-second timestamp (UTC), or ``None`` for an open bound.
Raises:
ValueError: If the value is negative or cannot be parsed.
"""
if value is None:
return None
# bool is an int subclass — reject it explicitly so True/False aren't
# silently read as 1/0 seconds.
if isinstance(value, bool):
raise ValueError(f"{param_name} must be a datetime string or Unix seconds")
if isinstance(value, (int, float)):
seconds = int(value)
if seconds < 0:
raise ValueError(f"{param_name} must be >= 0, got {seconds}")
return seconds
if isinstance(value, str):
text = value.strip()
if not text:
return None
if _UNIX_SECONDS_RE.fullmatch(text):
return int(text)
# RFC 3339 / ISO 8601. ``fromisoformat`` accepts a trailing "Z" only on
# Python 3.11+; normalize it for safety across versions.
try:
parsed = datetime.fromisoformat(text.replace("Z", "+00:00"))
except ValueError as exc:
raise ValueError(
f"{param_name} must be an RFC 3339 datetime or Unix seconds, "
f"got {value!r}"
) from exc
if parsed.tzinfo is None:
parsed = parsed.replace(tzinfo=timezone.utc)
return int(parsed.timestamp())
raise ValueError(f"{param_name} must be a datetime string or Unix seconds")
@@ -52,6 +52,16 @@ _PAYLOAD_INDEX_FIELDS: dict[str, PayloadSchemaType] = {
"chunk_index": PayloadSchemaType.INTEGER, "chunk_index": PayloadSchemaType.INTEGER,
"chunk_start_offset": PayloadSchemaType.INTEGER, "chunk_start_offset": PayloadSchemaType.INTEGER,
"chunk_end_offset": PayloadSchemaType.INTEGER, "chunk_end_offset": PayloadSchemaType.INTEGER,
# modified_at is the ADR-027 date-range filter field: searches apply
# Range(key="modified_at", gte=..., lte=...) (see
# search/access_filter.build_base_filter_conditions). Qdrant requires a
# payload index to evaluate a Range efficiently — without one every dated
# query full-scans the collection (and 400s on Qdrant Cloud strict mode).
# INTEGER (not FLOAT): modified_at is an int Unix-second timestamp on every
# point (vector/processor.py, vector/placeholder.py). _ensure_payload_indexes
# is idempotent, so existing collections gain this index at startup with no
# content re-index and no operator action.
"modified_at": PayloadSchemaType.INTEGER,
} }
# Sentinel point that records "this collection has been backfilled to str # Sentinel point that records "this collection has been backfilled to str
+74
View File
@@ -5,9 +5,11 @@ from __future__ import annotations
from unittest.mock import AsyncMock from unittest.mock import AsyncMock
import pytest import pytest
from qdrant_client.models import FieldCondition, Range
from nextcloud_mcp_server.search import access_filter from nextcloud_mcp_server.search import access_filter
from nextcloud_mcp_server.search.access_filter import ( from nextcloud_mcp_server.search.access_filter import (
build_base_filter_conditions,
build_ownership_filter, build_ownership_filter,
clear_accessible_owners_cache, clear_accessible_owners_cache,
list_accessible_owners, list_accessible_owners,
@@ -186,3 +188,75 @@ class TestBuildOwnershipFilter:
(user_branch,) = flt.should (user_branch,) = flt.should
assert user_branch.key == "user_id" assert user_branch.key == "user_id"
assert user_branch.match.value == "alice" assert user_branch.match.value == "alice"
class TestBuildBaseFilterConditions:
"""The shared ADR-027 filter contract used by both search algorithms."""
@pytest.mark.unit
def test_minimal_is_placeholder_plus_ownership(self) -> None:
# No doc_type, no date bounds -> exactly placeholder + ownership.
conditions = build_base_filter_conditions("alice", None)
assert len(conditions) == 2
# No modified_at Range condition present.
assert not any(
isinstance(c, FieldCondition) and c.key == "modified_at" for c in conditions
)
@pytest.mark.unit
def test_doc_type_appends_match_condition(self) -> None:
conditions = build_base_filter_conditions("alice", None, doc_type="note")
doc_type_conds = [
c
for c in conditions
if isinstance(c, FieldCondition) and c.key == "doc_type"
]
assert len(doc_type_conds) == 1
assert doc_type_conds[0].match.value == "note"
@pytest.mark.unit
@pytest.mark.parametrize(
"after,before,expected_gte,expected_lte",
[
(100, 200, 100, 200),
(100, None, 100, None), # after-only
(None, 200, None, 200), # before-only
],
)
def test_modified_at_range_appended(
self, after, before, expected_gte, expected_lte
) -> None:
conditions = build_base_filter_conditions(
"alice", None, modified_after=after, modified_before=before
)
range_conds = [
c
for c in conditions
if isinstance(c, FieldCondition) and c.key == "modified_at"
]
assert len(range_conds) == 1
rng = range_conds[0].range
assert isinstance(rng, Range)
assert rng.gte == expected_gte
assert rng.lte == expected_lte
@pytest.mark.unit
def test_no_range_when_both_bounds_none(self) -> None:
conditions = build_base_filter_conditions(
"alice", None, modified_after=None, modified_before=None
)
assert not any(
isinstance(c, FieldCondition) and c.key == "modified_at" for c in conditions
)
@pytest.mark.unit
def test_all_filters_compose(self) -> None:
# placeholder + ownership + doc_type + modified_at range = 4 conditions.
conditions = build_base_filter_conditions(
"alice",
["alice", "bob"],
doc_type="file",
modified_after=100,
modified_before=200,
)
assert len(conditions) == 4
+65 -1
View File
@@ -1,8 +1,13 @@
"""Unit tests for shared boundary validators.""" """Unit tests for shared boundary validators."""
from datetime import datetime, timezone
import pytest import pytest
from nextcloud_mcp_server.utils.validation import is_valid_nextcloud_doc_id from nextcloud_mcp_server.utils.validation import (
is_valid_nextcloud_doc_id,
parse_modified_timestamp,
)
@pytest.mark.unit @pytest.mark.unit
@@ -52,3 +57,62 @@ def test_accepts_positive_ascii_integers(value):
def test_rejects_invalid_doc_ids(value, reason): def test_rejects_invalid_doc_ids(value, reason):
"""Reject empty/zero/leading-zero/non-ASCII/non-digit inputs.""" """Reject empty/zero/leading-zero/non-ASCII/non-digit inputs."""
assert is_valid_nextcloud_doc_id(value) is False, f"should reject: {reason}" assert is_valid_nextcloud_doc_id(value) is False, f"should reject: {reason}"
# parse_modified_timestamp (ADR-027) ---------------------------------------
_JAN_2026_UTC = int(datetime(2026, 1, 1, tzinfo=timezone.utc).timestamp())
@pytest.mark.unit
@pytest.mark.parametrize(
"value,expected",
[
(None, None),
("", None),
(" ", None),
# RFC 3339 / ISO 8601
("2026-01-01T00:00:00Z", _JAN_2026_UTC),
("2026-01-01T00:00:00+00:00", _JAN_2026_UTC),
# +02:00 offset is two hours earlier in UTC
("2026-01-01T02:00:00+02:00", _JAN_2026_UTC),
# Naive datetime is assumed UTC
("2026-01-01T00:00:00", _JAN_2026_UTC),
# Date-only ISO form
("2026-01-01", _JAN_2026_UTC),
# Bare Unix seconds (string and int) pass through
(str(_JAN_2026_UTC), _JAN_2026_UTC),
(_JAN_2026_UTC, _JAN_2026_UTC),
(0, 0),
(1767225600.9, 1767225600), # float truncates
],
)
def test_parse_modified_timestamp_accepts(value, expected):
"""RFC 3339 strings, Unix seconds, and None normalize to int seconds (UTC)."""
assert parse_modified_timestamp(value) == expected
@pytest.mark.unit
@pytest.mark.parametrize(
"value,reason",
[
("not-a-date", "unparseable string"),
("2026-13-01T00:00:00Z", "invalid month"),
(-1, "negative int"),
(-5.0, "negative float"),
(True, "bool is not a timestamp"),
(False, "bool is not a timestamp"),
([], "wrong type"),
],
)
def test_parse_modified_timestamp_rejects(value, reason):
"""Bad formats / negatives / bools raise ValueError (→ McpError or HTTP 400)."""
with pytest.raises(ValueError):
parse_modified_timestamp(value)
@pytest.mark.unit
def test_parse_modified_timestamp_error_names_param():
"""The param_name is surfaced in the error for caller-friendly messages."""
with pytest.raises(ValueError, match="modified_after"):
parse_modified_timestamp("nope", param_name="modified_after")
+15
View File
@@ -73,6 +73,21 @@ def _record(point_id: int | str, doc_id: int | str | None) -> SimpleNamespace:
return SimpleNamespace(id=point_id, payload=payload) return SimpleNamespace(id=point_id, payload=payload)
# ---------------------------------------------------------------------------
# _PAYLOAD_INDEX_FIELDS contract
# ---------------------------------------------------------------------------
@pytest.mark.unit
def test_modified_at_indexed_as_integer():
"""ADR-027: the date-range filter needs a numeric index on modified_at.
INTEGER (not FLOAT/DATETIME) because the payload stores an int Unix-second
timestamp; a numeric Range filters it without a content re-index.
"""
assert _PAYLOAD_INDEX_FIELDS.get("modified_at") == PayloadSchemaType.INTEGER
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# _ensure_payload_indexes # _ensure_payload_indexes
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------