Round-3 review on PR #891 (no blockers):
- Add test_encode_dav_path_encodes_exactly_once pinning the documented
decoded-input precondition ("already%20encoded.pdf" -> "already%2520...").
- format_exception_group: proper singular/plural ("1 sub-exception" vs
"N sub-exceptions") instead of "(s)".
- oauth_sync: use `if doc_task is not None:` to match processor_task's guard.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Round-2 review on PR #891 (non-blocking):
- Add a parametrised test_webdav_path_encoding covering empty path,
leading-slash stripping, '#'/comma/space, and a non-ASCII name — the single
source of truth for every caller-path builder's encoding, so write_file /
delete_resource / create_directory / attachments are covered transitively.
- Document the decoded-input precondition on _webdav_path (encode-exactly-once;
passing an already-encoded path would double-encode).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Round-1 review on PR #891:
- Guard processor_task's broad except handler against an unbound doc_task
(mirrors multi_user_processor_task): initialise doc_task=None before the loop
and branch the error log. Fixes a latent NameError if receive() raises a
non-TimeoutError/EndOfStream before the first document binds. Regression test
added.
- Drop the unnecessary `from __future__ import annotations` in vector/_errors.py
and express format_exception_group's non-group fast path as an explicit
isinstance check.
- Add a copy_resource Destination-header encoding test (analogue to MOVE);
strengthen the ExceptionGroup test to assert the full leaf repr survives.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two ingest-robustness fixes from card 309 (OHR-Bench smoke-test triage).
WebDAV paths flowed through the client already URL-decoded (unquote on the
PROPFIND/REPORT <d:href>, or raw MCP-tool input), so a '#' reached httpx as a
URL fragment and silently truncated the request -> spurious 404 on otherwise
valid files (e.g. law filenames with '#', commas, double/trailing spaces).
Route every caller-path builder through a new _webdav_path helper that
percent-encodes the path once (preserving separators); the MOVE/COPY
Destination header is encoded too.
Vector-sync runs inside anyio task groups, so a child-task failure surfaced as
a BaseExceptionGroup whose str() is the useless "unhandled errors in a
TaskGroup (N sub-exception)" -- hiding the real ConnectError operators need.
Add format_exception_group to flatten the group to its leaf exceptions and use
it at the broad catch/log sites in processor.py and oauth_sync.py.
Refs: Deck board 12 card 309 (AC #4 filename handling, AC #2 observability).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`<d:href>` is required by RFC 3986 to be percent-encoded, so non-ASCII
filenames (e.g. Chinese, Cyrillic) were leaking through `list_directory`
and the SEARCH-based tools (`find_by_name`, `find_by_type`,
`list_favorites`, `search_files`) as their URL-encoded form. Decode with
the already-imported `urllib.parse.unquote` before exposing to callers.
Fixes#776
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Hide sensitive files/folders from the WebDAV MCP tool surface by
tagging them with a configured Nextcloud system tag. Defence-in-depth
control for users who connect LLMs to accounts holding contracts,
medical records, credentials, etc.
A new EXCLUDED_TAGS env var (comma-separated tag names, empty by
default) gates an exclusion layer that runs at the start of every
WebDAV tool call: tag names are resolved to tag IDs, those IDs are
expanded to the set of tagged paths, then listings/searches are
filtered and read/write/delete/move/copy operations on excluded paths
raise ToolError. Tagged folders exclude their descendants via prefix
match. Empty EXCLUDED_TAGS disables the feature entirely.
The threat model is preventing accidental data exfiltration via the
LLM tool surface — not hiding files from a determined operator. The
docs explicitly recommend creating exclusion tags with
user_assignable=false so the credentials the MCP server uses cannot
remove the tag.
Implementation:
- config.py: add `excluded_tags` to _DEFAULTS, Settings, and the
_field_map alongside other comma-separated env vars.
- client/webdav.py: get_files_by_tag now requests <d:resourcetype/>
and surfaces is_directory so tagged directories can recursively
exclude descendants.
- server/tag_exclusion.py (new): get_excluded_tag_names,
get_excluded_file_paths, is_path_excluded.
- server/webdav.py: exclusion guards in all 11 WebDAV tools;
read/write/create/delete/move/copy raise ToolError, list/search
tools silently filter excluded entries. Existing f-string log
calls converted to lazy %-style.
- tests: 17 new unit tests covering path-matching edge cases
(shared-prefix non-match, descendants of excluded dirs), tag-name
parsing, and get_excluded_file_paths with mocked WebDAV; 1 new
client test asserting <d:resourcetype/> -> is_directory parsing.
- docs/configuration.md: new "Tag-Based File Exclusion" section with
per-tool effect table, security guidance, and per-call cost note.
- README.md: feature mention under Key Features.
Closes#710.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Round 7 raised 5 issues; this round addresses all of them and fixes
the underlying causes (not just the comments) where applicable so
they don't get re-flagged in future passes.
Critical:
- verified_count description in SemanticSearchResponse said "unique
documents" but the value is len(verified_results), a chunk count.
Description rewritten to accurately document chunk-level granularity
AND explicitly call out the asymmetry with dropped_count (which
counts unique (doc_id, doc_type) pairs).
- _verify_files false-eviction risk: the round-6 doc-only fix was
re-flagged. Address at the source — widen WebDAVClient.get_file_info
to raise HTTPStatusError on 404 (matching the rest of the client
convention) and reserve None for the genuinely ambiguous
malformed-PROPFIND case. _verify_files now keeps the result on None
(cannot tell whether the file exists) and evicts only on a
definitive HTTPStatusError 404. Tests updated; new test added for
the malformed-XML keep-result path.
Non-critical:
- News verifier semaphore lifetime now explicitly documented: one
slot held for one deduplicated fetch per search is the correct
backpressure behaviour.
- Cross-reference comments in _verify_notes / _verify_deck_cards no
longer claim "Mirrors X" pointing at functions defined later in
the file; now use direction-neutral "parallel implementation in".
- accessible_by_type is mutated by concurrent run_verifier tasks; a
comment explains why this is race-free under anyio's cooperative
multitasking (distinct keys per task, no await between read and
write) so a future reader doesn't add a redundant lock.
- Knock-on: tests/integration/test_rag.py wraps get_file_info in a
try/except for the new contract.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fixed 8 type checker errors across the codebase:
- vector/scanner.py: Handle None scroll results with null-safe iteration
- search/{bm25_hybrid,semantic}.py: Add None checks for result.payload
- auth/{unified_verifier,webhook_routes}.py: Assert non-None auth credentials
- client/webdav.py: Add None checks before int() conversions
- providers/openai.py: Assert embedding_model is not None
- search/algorithms.py: Explicitly type doc_types set and cast values
- observability/logging_config.py: Match parent class signature (log_data)
Also fixed test_create_tag_creates_system_tag to match WebDAV implementation
(was testing OCS API endpoint, now tests correct WebDAV endpoint with
Content-Location header).
Type checker: 0 errors (down from 8), 20 warnings (ignored)
Tests: All 192 unit tests passing
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Add get_file_info() to get file info including file ID via PROPFIND
- Add create_tag() to create system tags via OCS API
- Add get_or_create_tag() for idempotent tag creation
- Add assign_tag_to_file() to assign tags to files via WebDAV
- Add remove_tag_from_file() to remove tags from files
Also refactors RAG evaluation:
- Add indexed_manual_pdf fixture using existing nc_client/nc_mcp_client
- Remove manual tag creation steps from workflow (now handled by fixture)
- Add comprehensive unit tests for new WebDAV methods
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit fixes two critical issues with PDF processing:
1. **Text extraction mismatch (context expansion bug)**:
- Indexing used pymupdf4llm.to_markdown() producing markdown text
- Context expansion used page.get_text() producing plain text
- Different text formats caused character offset misalignment
- Search would find correct chunk, but expansion showed wrong section
- Fixed by making context.py use pymupdf4llm.to_markdown() consistently
2. **Diagnostic logging for page number assignment**:
- Added logging to verify page_boundaries exist in metadata
- Added logging to verify assign_page_numbers() assigns values
- Helps diagnose why page numbers show as null in search results
3. **mime_type storage bug**:
- Fixed incorrect field reference in processor.py:405
- Was using file_metadata.get("content_type", "")
- Should use content_type from WebDAV response
Changes:
- nextcloud_mcp_server/search/context.py: Use pymupdf4llm.to_markdown()
for PDF text extraction to match indexing method
- nextcloud_mcp_server/vector/processor.py: Add diagnostic logging for
page boundaries and assignment, fix mime_type storage
- tests/unit/client/test_webdav.py: Fix import sorting
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
