Add research context and artifact ingestion

.github/workflows/ci.yml

@@ -54,7 +54,7 @@ jobs:
         run: python -B 01-capture/tests/test_extension.py
 
   windows-pull:
-    name: Windows public pull, resolution, and fill safety
+    name: Windows pull, research ingest, and context safety
     runs-on: windows-latest
     steps:
       - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
@@ -76,6 +76,13 @@ jobs:
           if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
           & prov-sync-unresolved --help | Out-Null
           if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
+          & prov-ingest-research --help | Out-Null
+          if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
+          $contextVersion = & prov-context --version
+          if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
+          if (($contextVersion -join "`n").Trim() -ne '0.18.0') {
+            throw "Unexpected Context Capsule runtime version"
+          }
 
           $raw = & prov-sync-pull --capabilities --json
           $code = $LASTEXITCODE
@@ -98,3 +105,10 @@ jobs:
           }
       - name: Run Windows sync-layer tests
         run: python -m unittest discover -s tests -p "test_synclayer_*.py" -v
+      - name: Run Windows research context tests
+        run: >-
+          python -m unittest
+          tests.test_research_artifacts
+          tests.test_context_capsule
+          tests.test_provenance.TestVaultIngest
+          -v

README.md

@@ -38,6 +38,10 @@ the extension.
 - Organizes conversation records, builds deterministic preparation digests,
   exports a Markdown/Obsidian view, and creates a local SQLite FTS5 index.
 - Exposes read-only research context through a stdlib JSON-RPC/MCP server.
+- Ingests validated `parsed-paper/1.0`, `reading-note/1.0`, `review/1.0`, and
+  `lineage-graph/1.0` artifacts through an atomic, lock-protected local merge.
+- Builds a bounded Context Capsule that separates approved project state,
+  auto-applied low-risk progress, and reference-only literature leads.
 - Captures session-closeout events into a local queue and advances them through a
   resumable, lock-protected state machine.
 - Requires a resolved registered project before a scaffold or
@@ -141,14 +145,22 @@ are unofficial and can break independently of Provenance ingestion.
 
 ```powershell
 uv run --locked prov-ingest-library
+uv run --locked prov-ingest-research C:\ScriptoriumDemo\steward-output
 uv run --locked prov-ingest-vault C:\ScriptoriumDemo\workspace
+uv run --locked prov-context --project synthetic-catalyst --json
 uv run --locked prov-search "research gap"
 ```
 
 The default literature input is
 `%PROVENANCE_HOME%\kb\library.json`. The project portfolio comes from the
 workspace's `Projects/` frontmatter. Obsidian is optional; any Markdown workspace
-that follows the same contract can be used.
+that follows the same contract can be used. Research-artifact inputs must be
+explicit files/directories; the command does not discover personal folders or
+persist their source paths. It rejects symbolic links and Windows reparse points
+in every input or output path component, verifies file identity around reads and
+writes, and redacts absolute local paths including
+`file:` URIs while keeping read/write failures path-free. Capsule literature hints remain reference-only until
+the researcher reviews evidence and approves a claim.
 
 ### Read-only MCP
 
@@ -157,8 +169,8 @@ uv run --locked prov-mcp
 ```
 
 The server exposes `search_brain`, `get_profile`, `list_topics`, `get_portfolio`,
-and `get_current_context` over stdio. It performs no network request and does not
-write project state.
+`get_current_context`, and `get_context_capsule` over stdio. It performs no
+network request and does not write project state.
 
 ### Session writeback
 
@@ -203,13 +215,15 @@ mapping. See [the wiring runbook](docs/sync-layer-wiring.md).
 | `prov-ingest-takeout` | Import a selected Google Takeout ZIP |
 | `prov-ingest-agents` | Import supported local coding-agent logs |
 | `prov-ingest-library` | Import a `library-kb/1.x` snapshot |
+| `prov-ingest-research` | Atomically ingest explicit literature-reading artifacts |
 | `prov-ingest-vault` | Import project frontmatter |
 | `prov-ingest-notes` | Import private Markdown zones into protected state |
 | `prov-scrub` | Re-apply the current privacy transform |
 | `prov-organize` / `prov-prep` / `prov-diff` | Build deterministic derived views |
 | `prov-obsidian` | Export a local Markdown/Obsidian view |
 | `prov-search` | Build/query the local FTS5 index |
 | `prov-mcp` | Start the read-only stdio MCP server |
+| `prov-context` | Print one bounded, read-only project Context Capsule |
 | `prov-sync-pull` | Preview or run one lock-protected sync pass |
 | `prov-sync-pending` / `prov-sync-fill` | Review and submit one safe summary fill |
 | `prov-sync-unresolved` | Inspect unresolved mappings without mutation |

README.zh.md

@@ -31,6 +31,10 @@ Provenance 是 Scriptorium 套件中可独立部署的产品。它通过版本
 - 在记录进入工作层前擦除常见凭据与 PII 形态，并可应用仅保存在本地的别名表。
 - 生成确定性分类、精读输入、Markdown/Obsidian 视图和本地 SQLite FTS5 索引。
 - 通过标准库实现的只读 JSON-RPC/MCP 服务向 Agent 提供研究上下文。
+- 通过带锁的原子合并摄取经过验证的 `parsed-paper/1.0`、
+  `reading-note/1.0`、`review/1.0` 与 `lineage-graph/1.0` 工件。
+- 生成有长度上限的 Context Capsule，明确区分已批准项目状态、自动写入的
+  低风险进度，以及仅供参考的文献线索。
 - 将会话收尾事件写入本地队列，并通过可恢复、带锁的状态机推进。
 - 项目未解析时不生成 scaffold、timeline、draft 或 `session-summary/1.0`。
 - 高价值状态与结论先进入草稿，必须由用户审批后才能写回。
@@ -126,13 +130,20 @@ Provider 内部接口并非公开契约，可能独立于 Provenance 摄取端
 
 ```powershell
 uv run --locked prov-ingest-library
+uv run --locked prov-ingest-research C:\ScriptoriumDemo\steward-output
 uv run --locked prov-ingest-vault C:\ScriptoriumDemo\workspace
+uv run --locked prov-context --project synthetic-catalyst --json
 uv run --locked prov-search "research gap"
 ```
 
 默认文献输入为 `%PROVENANCE_HOME%\kb\library.json`；项目组合来自工作区
 `Projects/` 的 frontmatter。Obsidian 是可选示例，符合相同契约的 Markdown
-工作区也可以使用。
+工作区也可以使用。科研工件必须由用户显式指定文件或目录；命令不会发现个人
+文件夹，也不会保存输入路径；输入和输出路径任一层级中的符号链接、Junction 与
+Windows 重解析点都会被拒绝，读写前后还会核对文件身份，
+包括 `file:` URI 在内的绝对本地路径会被脱敏，读写失败也不会回显本地路径。
+Capsule 中的文献提示在研究者核对证据并批准
+论断前始终只是参考线索。
 
 ### 只读 MCP
 
@@ -141,7 +152,8 @@ uv run --locked prov-mcp
 ```
 
 服务通过 stdio 暴露 `search_brain`、`get_profile`、`list_topics`、
-`get_portfolio` 与 `get_current_context`，不发起网络请求，也不写项目状态。
+`get_portfolio`、`get_current_context` 与 `get_context_capsule`，不发起网络请求，
+也不写项目状态。
 
 ### 会话回写
 
@@ -185,13 +197,15 @@ uv run --locked prov-sync-fill <summary-id> --provenance-home $env:PROVENANCE_HO
 | `prov-ingest-takeout` | 摄取用户选择的 Google Takeout ZIP |
 | `prov-ingest-agents` | 摄取受支持的本地编程 Agent 日志 |
 | `prov-ingest-library` | 摄取 `library-kb/1.x` 快照 |
+| `prov-ingest-research` | 原子摄取显式选择的文献阅读工件 |
 | `prov-ingest-vault` | 摄取项目 frontmatter |
 | `prov-ingest-notes` | 将私人 Markdown 区域摄取进受保护状态 |
 | `prov-scrub` | 对既有记录重跑当前隐私转换 |
 | `prov-organize` / `prov-prep` / `prov-diff` | 生成确定性派生视图 |
 | `prov-obsidian` | 导出本地 Markdown/Obsidian 视图 |
 | `prov-search` | 构建或查询本地 FTS5 索引 |
 | `prov-mcp` | 启动只读 stdio MCP 服务 |
+| `prov-context` | 输出一个有边界、只读的项目 Context Capsule |
 | `prov-sync-pull` | 预览或执行一次带锁同步 |
 | `prov-sync-pending` / `prov-sync-fill` | 审阅并提交一个安全摘要 fill |
 | `prov-sync-unresolved` | 只读检查未解析映射 |

docs/OVERVIEW.md

@@ -1,6 +1,6 @@
 # Provenance 产品概览
 
-> 版本 0.17.0 · Scriptorium 套件中的本地科研记忆与证据桥梁。
+> 版本 0.18.0 · Scriptorium 套件中的本地科研记忆与证据桥梁。
 
 ## 1. 用户问题
 
@@ -14,7 +14,7 @@ Provenance 解决的不是单次问答，而是研究项目的连续性：
 - 把支持的本地来源规范化到一个用户控制的数据根；
 - 在进入工作层前执行凭据擦洗与可选假名化；
 - 把文献、项目状态和会话线索接入同一本地检索层；
-- 让 Agent 通过只读 MCP 恢复当前研究上下文；
+- 让 Agent 通过 CLI 或只读 MCP 获得有长度上限、带信任说明的 Context Capsule；
 - 把会话收尾变成可审阅的 timeline、草稿和人工审批，而不是自动改写事实。
 
 ## 2. 产品定位
@@ -42,7 +42,10 @@ Scriptorium 套件内，Steward 负责文献处理，Lectern 负责汇报材料
 脱敏、假名化、规范化、建立本地索引
         |
         v
-Agent 读取当前问题、证据、缺口、决定与下一步
+显式摄取已验证的阅读工件，生成只读 Context Capsule
+        |
+        v
+Agent 读取当前问题、参考证据、缺口、决定与下一步
         |
         v
 一次协作结束 -> 合成 scaffold -> 人工审阅 fill
@@ -67,6 +70,17 @@ Agent 读取当前问题、证据、缺口、决定与下一步
 脱敏降低误泄露概率，不把私人科研内容变成公共数据。附件按原始字节处理，
 尤其不能因文本记录已脱敏就假定附件安全。
 
+Context Capsule 进一步区分三类信息：
+
+| Capsule 内容 | 信任级别 | 使用方式 |
+|---|---|---|
+| 人工编写的项目字段、已批准论断 | `human_or_approved` | 可作为当前项目状态 |
+| 自动追加的低风险进展 | `auto_applied_low_risk_not_approved_claims` | 仅用于恢复过程上下文 |
+| 阅读工件、知识缺口与优先阅读线索 | `reference_only_not_approved_claims` | 只作为待核对参考 |
+
+原始会话、待批草稿、被拒绝论断和无关工件不会进入 Capsule。Markdown 输出
+最多 8,000 字符，JSON 输出最多 20,000 字符；绝对本地路径不会随输出暴露。
+
 ## 5. 首发能力与边界
 
 首发目标是 Windows 上可部署、可验证的本地 Python 产品，要求用户具备配置
@@ -78,6 +92,10 @@ GitHub 开源项目和本地 Agent 的基本能力。Obsidian、Zotero 均为可
 - 本地 Agent 日志、Google Takeout、兼容 ZIP、Markdown、文献库与项目组合摄取；
 - 凭据/PII 擦洗、别名表假名化、分类、精读输入、Obsidian 导出；
 - SQLite FTS5 本地检索与只读 MCP；
+- 显式摄取 `parsed-paper/1.0`、`reading-note/1.0`、`review/1.0` 与
+  `lineage-graph/1.0`，并执行稳定身份、原子幂等合并与搜索索引；
+- 通过 `prov-context` 和 MCP `get_context_capsule` 生成只读、带信任分层的
+  项目恢复上下文；
 - 带锁、可恢复、人工门控的事件/同步层；
 - 手动触发、同源请求、无第三方运行库的可选浏览器导出器；
 - 两个中立 Agent Skill 与跨平台测试。
@@ -87,7 +105,17 @@ GitHub 开源项目和本地 Agent 的基本能力。Obsidian、Zotero 均为可
 - 浏览器商店签名发行，以及对 Provider 未公开内部接口的稳定性保证；
 - 云端托管、后台定时模型调用或自动上传；
 - 未经人工确认的结论自动晋升；
-- 将所有来源内容自动视为事实。
+- 将所有来源内容自动视为事实；
+- `experiment-run/1.0` 与 `claim-evidence/1.0` 的摄取、检索或批准流程。
+
+研究工件必须由用户显式选择文件或目录。摄取器不发现个人目录、不保存来源
+路径，并拒绝符号链接和 Windows 重解析点。包括 `file:` URI 在内的绝对
+Windows、UNC 与 POSIX 路径会在落盘和建立索引前脱敏；读、解析、加锁和写入
+错误不会回显来源或目标路径。
+
+scriptorium-spec 发布实验运行与论断—证据契约只是前置条件，不代表 Provenance
+已经实现相应能力；后续版本仍需单独增加验证、信任与审批语义。0.18.0 会将
+这两种 schema 视为不支持，而不是默认为已有能力。
 
 ## 6. 质量证明
 
@@ -98,11 +126,15 @@ GitHub 开源项目和本地 Agent 的基本能力。Obsidian、Zotero 均为可
   往返和打包 allowlist；
 - 公开树门禁检查敏感文件名、凭据形态、非示例邮箱与已知私有夹具；
 - `skills/` 通过 Skill 结构校验；
-- public pull 默认只预览，写操作需要显式 `--run`。
+- public pull 默认只预览，写操作需要显式 `--run`；
+- 合成测试覆盖研究工件的幂等与并发写入、符号链接/重解析点拒绝、`file:`
+  URI 与路径脱敏、四类工件检索，以及 Capsule 的只读、长度限制和信任分层。
 
 ## 7. 下一阶段
 
-1. 完成浏览器商店发行评估、真实账号兼容性验收与接口变化监测方案。
-2. 提供从合成项目初始化到 MCP 恢复上下文的 Windows 演示路径。
-3. 建立可重复的用户验收：部署时长、首个可检索项目、一次完整 writeback。
+1. 在 scriptorium-spec 契约完成后，为后续版本实现 `experiment-run/1.0`
+   与 `claim-evidence/1.0` 的摄取、信任和审批语义。
+2. 完成浏览器商店发行评估、真实账号兼容性验收与接口变化监测方案。
+3. 建立可重复的 Windows 用户验收：部署时长、首个可检索项目、一次完整
+   Context Capsule 与 writeback。
 4. 为公开契约增加跨仓库兼容测试与版本迁移指南。

docs/SPEC.md

@@ -3,7 +3,7 @@
 | Field | Value |
 |---|---|
 | Status | Public living contract |
-| Product version | 0.17.0 |
+| Product version | 0.18.0 |
 | Runtime | Python standard library |
 | Primary release target | Windows 10/11 |
 | Data root | `PROVENANCE_HOME` |
@@ -27,8 +27,9 @@ Included:
 
 - installable `provenance` Python package;
 - local ingestion, privacy transforms, organization, search, Markdown export;
-- literature and project-memory ingestion;
-- read-only stdio MCP server;
+- literature, literature-reading artifact, and project-memory ingestion;
+- bounded project Context Capsules through the CLI and read-only stdio MCP
+  server;
 - event/sync queue, resolver, worker, pending/fill boundary, approvals;
 - optional clean-room browser producer under `01-capture/`;
 - neutral `skills/`, Windows wiring templates, synthetic tests.
@@ -52,7 +53,7 @@ PROVENANCE_HOME/
   inbox/          normalized working records and raw ZIP backup
   drop/           optional user-selected import queue
   kb/             versioned literature input
-  memory/         project, literature and approved-claim snapshots
+  memory/         project, literature, research-artifact and approved-claim snapshots
   output/         deterministic organization views
   _prep/          deterministic Skill inputs
   profile/        local profile artifacts
@@ -108,6 +109,74 @@ through the text redactor.
 RAG/search results and Agent-generated summaries are candidates, not accepted
 research facts.
 
+### 6.1 Research-artifact ingestion
+
+`prov-ingest-research` accepts only user-supplied files and directories. It does
+not discover personal folders, modify source files, or retain source paths. The
+0.18.0 implementation consumes exactly these scriptorium-spec contracts:
+
+- `parsed-paper/1.0`;
+- `reading-note/1.0`;
+- `review/1.0`;
+- `lineage-graph/1.0`.
+
+All selected inputs are validated and sanitized before one deterministic merge
+into `memory/research-artifacts.json`. Records receive stable content identities;
+the load-merge-write transaction is protected by a kernel-backed lock, uses an
+atomic replacement, and is idempotent for unchanged records.
+
+Parsed-paper and reading-note identities preserve the exact Better BibTeX
+citekey. Review and lineage identities include their `created` value, so a later
+snapshot of the same direction is retained while the same direction and
+timestamp remain idempotent.
+
+Input traversal and the destination store boundary fail closed when any path
+component is a symbolic link, junction or Windows reparse point, including
+ancestors and nested entries. Input bytes are read from one verified file
+descriptor and file identity is checked before and after the read. The target
+and kernel-lock identities are rechecked around the load-merge-write transaction.
+Absolute Windows, UNC and POSIX paths and `file:` URIs are redacted before
+persistence and search indexing. Read, parse, lock and write failures return
+bounded messages that do not echo source or destination paths.
+
+### 6.2 Context Capsule
+
+`prov-context --project PROJECT_ID` and MCP `get_context_capsule` return the same
+deterministic, read-only project resume view. The capsule:
+
+- reads human-authored project fields and explicitly approved claims as project
+  state;
+- includes recent auto-applied timeline entries only as low-risk progress, not
+  approved claims;
+- includes related literature-reading artifacts and their gap/priority-reading
+  leads only as reference material, not approved claims;
+- excludes raw conversations, pending drafts, rejected claims, source-path
+  fields and unrelated research artifacts;
+- omits absolute local paths from all returned text;
+- is bounded to 8,000 characters in Markdown and 20,000 characters in the JSON
+  envelope, with explicit truncation metadata.
+
+The trust envelope is part of the public output:
+
+```json
+{
+  "project_state": "human_or_approved",
+  "recent_progress": "auto_applied_low_risk_not_approved_claims",
+  "research_artifacts": "reference_only_not_approved_claims"
+}
+```
+
+Neither the CLI nor the MCP method mutates project or memory state.
+
+### 6.3 Reserved future contracts
+
+Provenance 0.18.0 does not consume `experiment-run/1.0` or
+`claim-evidence/1.0`. Publishing those formats in scriptorium-spec is a
+prerequisite, not an implementation in Provenance: a later Provenance release
+must still add explicit validation, trust semantics and tests. In 0.18.0,
+`prov-ingest-research` rejects them as unsupported schema versions; no
+documentation or Context Capsule output may imply otherwise.
+
 ## 7. Event/sync state machine
 
 ```text
@@ -200,5 +269,5 @@ quick_validate.py skills/session-summary
 ```
 
 Linux and Windows run the extension boundary tests. Linux runs the full Python
-suite; Windows additionally exercises lock, reparse-point, pending/fill and public
-pull behavior.
+suite; Windows additionally exercises lock, reparse-point, research-artifact
+ingestion, Context Capsule, pending/fill and public pull behavior.

provenance/__init__.py

@@ -1,4 +1,4 @@
 """Provenance — local-first capture/sanitize/organize/profile pipeline for your
 cross-AI conversation history. See docs/SPEC.md and docs/PRIVACY.md."""
 
-__version__ = "0.17.0"
+__version__ = "0.18.0"