docs: organize Dokimi features into an illustrated guide

.agent-eval/tasks/docs-new-page/check.py

@@ -23,7 +23,8 @@
     nav = Path("mkdocs.yml").read_text(encoding="utf-8")
     nav_part = nav[nav.find("\nnav:"):] if "\nnav:" in nav else ""
     m["in_mkdocs_nav"] = PAGE in nav_part
-    index = Path("docs/README.md").read_text(encoding="utf-8")
+    # The index is docs/documentation-index.md since the feature guide; the task base still has it in docs/README.md.
+    index = "".join(Path(f).read_text(encoding="utf-8") for f in ("docs/documentation-index.md", "docs/README.md") if Path(f).exists())
     m["in_docs_index"] = bool(re.search(r"\]\(\.?/?" + re.escape(PAGE) + r"\)", index))
 except Exception as exc:  # a broken workspace fails the checks, it does not crash the grader
     print(f"check error: {exc}")

.github/workflows/docs.yml

@@ -68,6 +68,14 @@ jobs:
       - run: pip install --no-deps --require-hashes -r requirements.lock.txt -r requirements-docs.lock.txt
       - name: mkdocs build
         run: mkdocs build --strict
+      - name: Save PR documentation preview
+        if: github.event_name == 'pull_request'
+        uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9  # v7.0.2
+        with:
+          name: docs-preview
+          path: dokimi/site
+          retention-days: 7
+          if-no-files-found: error
       - if: github.event_name != 'pull_request'
         uses: actions/upload-pages-artifact@fc324d3547104276b827a68afc52ff2a11cc49c9  # v5.0.0
         with:

AGENTS.md

@@ -68,8 +68,10 @@ also holds a PR to:
 - Executable tasks (Docker + oracle solution + pytest verifier) or harnesses:
   `dokimi/docs/contributing-terminal-bench-task.md`. Tasks live in a
   content repository; the framework reads a checkout via `--tasks-root`.
-- Anything else: the index, `dokimi/docs/README.md`. A new doc page also goes
-  in the `nav:` of `dokimi/mkdocs.yml`, the published site.
+- Anything else: the index, `dokimi/docs/documentation-index.md`. A new page
+  at any depth under `dokimi/docs/` goes in that index
+  (`dokimi/tests/test_docs_links.py` checks it) and in the `nav:` of
+  `dokimi/mkdocs.yml`, the published site.
 
 Registries, bundled in the package (`dokimi/dokimi/`) so they ship in the wheel:
 

README.md

@@ -9,67 +9,65 @@ SPDX-License-Identifier: Apache-2.0
 [![codecov](https://codecov.io/gh/intel/dokimi/graph/badge.svg?branch=main)](https://codecov.io/gh/intel/dokimi)
 [![OpenSSF Scorecard](https://api.scorecard.dev/projects/github.com/intel/dokimi/badge)](https://scorecard.dev/viewer/?uri=github.com/intel/dokimi)
 
-**Proof, not popularity: find out whether your agent's skills, docs and `AGENTS.md` actually work.**
+Find out whether your agent's skills, documentation and repository instructions improve its work. Dokimi runs the same questions or coding tasks under different context conditions, compares the outcomes, and helps you investigate the difference.
 
 ![The same eval set runs on agent A as it is and on agent B with one change; the same grader scores both, and the difference with its p-value decides](dokimi/docs/assets/readme/ab-test.png)
 
-**Documentation: <https://intel.github.io/dokimi/>** — the full tour, with every figure.
+Read the measured delta, uncertainty and task-level evidence together; one passing example does not establish a general benefit.
 
-## What it is
+![Assess with static checks, diagnose with novelty, reach and routing, change the skill or guidance, re-measure on the same tasks; keep a better result, reject none or worse with numbers, explain an unclear one, and reassess for a new model or harness](dokimi/docs/assets/readme/improvement-loop.png)
 
-Skills, instructions, docs, MCP servers and a repository's `AGENTS.md` are cheap
-to write and easy to over-claim. Dokimi runs the same questions or coding tasks
-with and without one change, grades both the same way, and reports the
-difference with its p-value, per model. Then it shows which parts of a skill
-the model already knew, which it added and which it broke, and proposes a
-shorter version that keeps what matters.
+## What can you find out?
 
-> **dokimi** — Greek *δοκιμή*, "test, proof". An artifact gets trusted because
-> it passed a test that could have failed, not because it is popular.
+| Your question | What to inspect |
+|---|---|
+| Does this context help? | [Controlled arms, models, harnesses and cost](dokimi/docs/features/experiments.md) |
+| Which parts of a skill matter? | [Knowledge probes, annotated passages, bundles and shortening](dokimi/docs/features/skills.md) |
+| Will the agent use it? | [Discovery, routing, tool preference and instruction loading](dokimi/docs/features/discovery.md) |
+| Does repository guidance work? | [Static claims, scopes, executable tasks and rule evidence](dokimi/docs/features/repositories.md) |
+| Do the tests cover the work? | [Task creation, coverage and grader controls](dokimi/docs/features/eval-design.md) |
+| Can the grades be trusted? | [Scoring methods, judge checks and grounding](dokimi/docs/features/judging.md) |
+| What did the agent do? | [Context use, execution paths and process contracts](dokimi/docs/features/traces.md) |
+| Will it keep working? | [Reports, baselines, CI and development directions](dokimi/docs/features/continuous-evaluation.md) |
 
-![1. Does it help? Ship it or cut it. 2. Which parts help? Edit it. 3. Does guidance land? Fix the repo](dokimi/docs/assets/readme/three-questions.png)
+**[Full feature tree: 39 individual cards](dokimi/docs/features/README.md)** · **[Documentation](https://intel.github.io/dokimi/)**
 
-**Which parts help: novelty.** Dokimi asks the model about each claim in a
-skill twice, without the skill and with it, and a different model grades both
-answers. Every claim lands in one of four outcomes
-([more](dokimi/docs/skill-novelty.md)):
+Each card states what works today and where it stops. Open PRs, partial workflows and research directions are included, with links, so they are discoverable without being advertised as released functionality.
+
+**Which parts of a skill help: novelty.** Dokimi asks the model about each claim in a skill twice, without the skill and with it, and a different model grades both answers. Every claim lands in one of four outcomes, per model ([more](dokimi/docs/skill-novelty.md)):
 
 ![A 2x2 grid. Wrong without, right with: added, keep it. Right without and with: redundant, cut it for this model. Wrong both times: still missing, rewrite it. Right without, wrong with: degraded, fix it first](dokimi/docs/assets/readme/novelty-outcomes.png)
 
-What measuring shows so far ([evidence](dokimi/docs/skill-novelty-evidence.md)):
-under lenient grading, 70% of what a skill says a frontier model already
-produces unprompted; under strict grading, which also checks exact values, 38%
-is still missing even with the skill in context; a plausible wrong value
-planted in a skill is repeated in 85–94% of cases in a small study; and
-what a small model needs, a large one may not, so every verdict is per model.
+## Choose your starting point
+
+[Skill author or reviewer](dokimi/docs/start/skills.md) · [Repository maintainer](dokimi/docs/start/repositories.md) · [Evaluator or researcher](dokimi/docs/start/experiments.md) · [Agent / CI operator](dokimi/docs/start/automation.md) · [Contributor](dokimi/docs/start/contributors.md)
+
+## Start with a free check
+
+From a checkout, with Python 3.11 or newer:
 
-## Start here
+```bash
+cd dokimi
+pip install -e .
+dokimi repo scan /path/to/your/repository
+```
+
+Replace the path with the repository you want to inspect. This static scan does not call an LLM. To check a skill collection instead:
 
 ```bash
-cd dokimi && pip install -e .     # installs the `dokimi` command
+dokimi catalog check --root /path/to/your/catalog --kind skills
 ```
 
-| You want to… | Start with | Then read |
-|---|---|---|
-| Check a skill | `dokimi catalog check --root <catalog> --kind skills` (free) | [Reviewing a skill](dokimi/docs/reviewing.md#a-skill) |
-| Check a repository's agent setup | `dokimi repo scan <repo>` (free) | [Reviewing a repository](dokimi/docs/reviewing.md#a-repositorys-agent-setup) |
-| See which parts of a skill help | `dokimi skill novelty analyze --artifact skill:<dir> --tier triage --estimate-only` (free estimate) | [Content novelty](dokimi/docs/skill-novelty.md) |
-| Compare with and without a change | `dokimi arms run --arms baseline,skill:<dir> …` | [Evaluating treatments](dokimi/docs/evaluating-treatments.md) |
-| Write evals you can trust | the [`dokimi-design-evals`](dokimi/data/skills/dokimi-design-evals/SKILL.md) skill | [External content](dokimi/docs/external-content.md) |
+Then follow [reviewing a skill or repository](dokimi/docs/reviewing.md) for the next step and its cost. Agent experiments need the selected provider or logged-in harness; executable coding tasks also need their supported container environment. Check [harness support](dokimi/docs/coding-harnesses.md) before choosing a runner.
+
+## Read the evidence
 
-Every step lists its cost; the free ones come first.
+A skill can teach something new without improving a task, and an instruction can exist without being loaded. Dokimi separates these questions. Knowledge novelty suggests edits to test; it does not mean familiar content is safe to delete. Grader controls help expose blind spots; they do not prove a verifier is infallible.
 
-**Coding agents.** Asked to "use dokimi to review this skill" or "this
-repository's agent setup"? Follow [reviewing.md](dokimi/docs/reviewing.md) step
-by step: it says which command to run, what each costs, and how to read the
-result. `dokimi --help` lists every command. To change dokimi itself, read
-[AGENTS.md](AGENTS.md).
+For methods and recorded studies, see [what Dokimi measures](dokimi/docs/what-it-measures.md), [methodology](dokimi/docs/methodology.md) and [skill novelty evidence](dokimi/docs/skill-novelty-evidence.md). Results belong to the evaluated artifact, task set, model, harness and grader.
 
-**More:** [what Dokimi measures](dokimi/docs/what-it-measures.md) ·
-[why it exists](VISION.md) · [contributing](CONTRIBUTING.md) ·
-[CLI reference](https://intel.github.io/dokimi/reference/cli/)
+**Coding agents:** use [reviewing.md](dokimi/docs/reviewing.md) to evaluate content, and [AGENTS.md](AGENTS.md) to change Dokimi. See also [contributing](CONTRIBUTING.md), [the CLI reference](https://intel.github.io/dokimi/reference/cli/) and [the project vision](VISION.md).
 
 ## License
 
-Licensed under the [Apache License, Version 2.0](LICENSE); see [NOTICE](NOTICE)
-for attribution.
+Licensed under the [Apache License, Version 2.0](LICENSE); see [NOTICE](NOTICE) for attribution.

dokimi/CONTRIBUTING.md

@@ -118,8 +118,8 @@ CI additionally holds a PR to:
   source tree whose file does not exist (an anchor into such a file is not
   checked), and on a list that nests differently on the site than on GitHub
   (Python-Markdown wants nested items indented 4 spaces and a blank line before
-  a list). A new page goes in two places: the `docs/README.md` index
-  (`tests/test_docs_links.py` checks it) and `nav:` in `mkdocs.yml` (the
+  a list). A new page goes in two places: the `docs/documentation-index.md`
+  index (`tests/test_docs_links.py` checks it) and `nav:` in `mkdocs.yml` (the
   build fails on a page missing from it). `mkdocs serve` previews the site.
 - **Dependencies match the imports** — `deptry .` (config in `[tool.deptry]` of
   `pyproject.toml`). A third-party module the package, `cli.py` or `scripts/`

dokimi/README.md

@@ -21,7 +21,11 @@ context layer and reports the delta together with its significance.
 Context layers it can measure: local Markdown/URL documentation, Context7 and
 real MCP servers, skills (`SKILL.md` and bundles), agent profiles (system
 prompts), plugins, and whole code repositories with their own tasks and graders
-(manifest validation and pinned sources today; the task-run contract is proposed).
+(validation, pinned sources, task runs, reports and comparisons are implemented;
+individual extensions remain proposed).
+
+Explore the [feature tree and 39 illustrated cards](https://github.com/intel/dokimi/blob/main/dokimi/docs/features/README.md)
+for the full capability map, including partial workflows and development directions.
 
 Evidence tracks:
 
@@ -49,7 +53,7 @@ the same material. Pick by what you are about to do with the answer:
 
 The first ships or cuts an artifact; the second edits one; the third measures a
 codebase instead of a document.
-[docs/README.md](https://github.com/intel/dokimi/blob/main/dokimi/docs/README.md) explains the three in full and
+[docs/what-it-measures.md](https://github.com/intel/dokimi/blob/main/dokimi/docs/what-it-measures.md) explains the three in full and
 [docs/architecture.md](https://github.com/intel/dokimi/blob/main/dokimi/docs/architecture.md) maps them to modules.
 
 The framework is content-agnostic: question sets, tasks, skills and products are
@@ -113,7 +117,7 @@ Run the test suite:
 python -m pytest -q
 ```
 
-For the LLM evaluation pipeline (answers, judging, dashboard), see [docs/quickstart.md](https://github.com/intel/dokimi/blob/main/dokimi/docs/quickstart.md). For benchmark invocation and comparison recipes, see [docs/benchmarking-and-comparison.md](https://github.com/intel/dokimi/blob/main/dokimi/docs/benchmarking-and-comparison.md). A high-level architecture map is in [docs/architecture.md](https://github.com/intel/dokimi/blob/main/dokimi/docs/architecture.md), and [docs/README.md](https://github.com/intel/dokimi/blob/main/dokimi/docs/README.md) is the documentation index.
+For the LLM evaluation pipeline (answers, judging, dashboard), see [docs/quickstart.md](https://github.com/intel/dokimi/blob/main/dokimi/docs/quickstart.md). For benchmark invocation and comparison recipes, see [docs/benchmarking-and-comparison.md](https://github.com/intel/dokimi/blob/main/dokimi/docs/benchmarking-and-comparison.md). A high-level architecture map is in [docs/architecture.md](https://github.com/intel/dokimi/blob/main/dokimi/docs/architecture.md), and [docs/documentation-index.md](https://github.com/intel/dokimi/blob/main/dokimi/docs/documentation-index.md) is the documentation index.
 
 ## Main CLI areas
 
@@ -279,8 +283,8 @@ data/                    Curated, version-controlled benchmark inputs (see data/
   skills/fastcontext/    The SKILL.md fixture the loader tests and examples use
   agent_profiles/        Agent persona (system) prompt fixtures
   explore_tasks/         Localization tasks with known-good references
-docs/                    Documentation (full index: docs/README.md)
-  README.md              Concept overview and documentation index
+docs/                    Documentation (full index: docs/documentation-index.md)
+  README.md              Start page: where to begin for each task
   quickstart.md          End-to-end LLM evaluation pipeline
   external-content.md    Pointing the framework at your own questions, tasks, skills
   evaluating-treatments.md