Template compliance

.github/CODE_OF_CONDUCT.md

@@ -0,0 +1,92 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+We as members, contributors, and leaders pledge to make participation in our
+community a harassment-free experience for everyone, regardless of age, body
+size, visible or invisible disability, ethnicity, sex characteristics, gender
+identity and expression, level of experience, education, socio-economic status,
+nationality, personal appearance, race, religion, or sexual identity and
+orientation.
+
+We pledge to act and interact in ways that contribute to an open, welcoming,
+diverse, inclusive, and healthy community.
+
+## Our Standards
+
+Examples of behavior that contributes to a positive environment for our
+community include:
+
+- Demonstrating empathy and kindness toward other people
+- Being respectful of differing opinions, viewpoints, and experiences
+- Giving and gracefully accepting constructive feedback
+- Accepting responsibility and apologizing to those affected by our mistakes,
+  and learning from the experience
+- Focusing on what is best not just for us as individuals, but for the overall
+  community
+
+Examples of unacceptable behavior include:
+
+- The use of sexualized language or imagery, and sexual attention or advances of
+  any kind
+- Trolling, insulting or derogatory comments, and personal or political attacks
+- Public or private harassment
+- Publishing others' private information, such as a physical or email address,
+  without their explicit permission
+- Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Enforcement Responsibilities
+
+Community leaders are responsible for clarifying and enforcing our standards of
+acceptable behavior and will take appropriate and fair corrective action in
+response to any behavior that they deem inappropriate, threatening, offensive,
+or harmful.
+
+Community leaders have the right and responsibility to remove, edit, or reject
+comments, commits, code, wiki edits, issues, and other contributions that are
+not aligned to this Code of Conduct, and will communicate reasons for moderation
+decisions when appropriate.
+
+## Scope
+
+This Code of Conduct applies within all community spaces, and also applies when
+an individual is officially representing the community in public spaces.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported to the community leaders responsible for enforcement at
+**aaron.tuor@pnnl.gov**. All complaints will be reviewed and investigated
+promptly and fairly.
+
+All community leaders are obligated to respect the privacy and security of the
+reporter of any incident.
+
+## Enforcement Guidelines
+
+Community leaders will follow these Community Impact Guidelines in determining
+the consequences for any action they deem in violation of this Code of Conduct:
+
+1. **Correction** — Community Impact: use of inappropriate language or other
+   behavior deemed unprofessional or unwelcome. Consequence: a private, written
+   warning, providing clarity around the nature of the violation and an
+   explanation of why the behavior was inappropriate. A public apology may be
+   requested.
+2. **Warning** — Community Impact: a violation through a single incident or
+   series of actions. Consequence: a warning with consequences for continued
+   behavior. No interaction with the people involved for a specified period.
+3. **Temporary Ban** — Community Impact: a serious violation of community
+   standards. Consequence: a temporary ban from any sort of interaction or
+   public communication with the community for a specified period.
+4. **Permanent Ban** — Community Impact: demonstrating a pattern of violation of
+   community standards. Consequence: a permanent ban from any sort of public
+   interaction within the community.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage],
+version 2.1, available at
+https://www.contributor-covenant.org/version/2/1/code_of_conduct.html.
+
+[homepage]: https://www.contributor-covenant.org

.github/CONTRIBUTING.md

@@ -0,0 +1,62 @@
+# Contributing to DSAgt
+
+Thanks for your interest in improving DSAgt. This guide covers the mechanics;
+for how the codebase is organized and why, see the
+[Developer Guide](https://ai-modcon.github.io/dsagt/developer/) (source:
+`docs/developer.md`).
+
+## Getting started
+
+DSAgt develops on [`uv`](https://github.com/astral-sh/uv) with Python 3.12 or
+3.13.
+
+```bash
+git clone https://github.com/AI-ModCon/dsagt.git
+cd dsagt
+uv sync --all-groups          # runtime + dev + docs dependencies
+source .venv/bin/activate      # so `dsagt` and the helpers are on PATH
+```
+
+Work on a feature branch off `main`; open a pull request when it's ready.
+
+## Tests
+
+Run the unit suite before opening a PR (`python -m pytest`, **not** bare
+`pytest` — the bare binary can resolve the wrong interpreter):
+
+```bash
+uv run --no-sync python -m pytest -m "not integration" -q
+```
+
+Run a single file while iterating:
+
+```bash
+uv run --no-sync python -m pytest tests/test_config.py -q
+```
+
+Integration tests (`-m integration`) hit real embedding/LLM providers and need
+`EMBEDDING_*` / `LLM_*` credentials in the environment; they're excluded from CI
+and the default local run.
+
+## Lint & format
+
+CI enforces both on `src/` and `tests/` (contributor scripts under
+`use_cases/` are exempt):
+
+```bash
+uv run ruff check src tests
+uv run black src tests          # drop the path to format everything you touched
+```
+
+## Pull requests
+
+- Keep PRs focused and reasonably small; describe the intent, not just the diff.
+- Update `docs/` and `CHANGELOG.md` when behavior changes.
+- Make sure `ruff`, `black --check`, and the non-integration tests pass.
+- This is pre-1.0, dev-stage code: prefer clean removal over back-compat shims.
+
+## Reporting issues
+
+Open a GitHub issue with steps to reproduce, expected vs actual behavior, your
+OS/Python version, and the agent platform involved. For security issues, follow
+[SECURITY.md](SECURITY.md) instead of opening a public issue.

.github/SECURITY.md

@@ -0,0 +1,34 @@
+# Security Policy
+
+## Reporting a vulnerability
+
+**Please do not report security vulnerabilities through public GitHub issues.**
+
+Report privately through GitHub's
+[private vulnerability reporting](https://github.com/AI-ModCon/dsagt/security/advisories/new)
+(the **Security** tab → *Report a vulnerability*). If that isn't available to
+you, email the maintainers at **aaron.tuor@pnnl.gov**.
+
+Please include:
+
+- a description of the issue and its impact,
+- steps to reproduce (or a proof of concept),
+- affected version / commit, and
+- any suggested remediation.
+
+The maintainers will acknowledge your report, keep you updated on progress, and
+credit you in the fix unless you prefer to remain anonymous.
+
+## Scope
+
+DSAgt executes CLI codes and installs skills that the agent registers, and it
+runs code from external skill catalogs the user chooses to sync. It does **not**
+sandbox that code — see the "Risks" section of [agent-card.md](../agent-card.md).
+Reports most relevant to this project include: path-traversal or arbitrary
+file write/delete from untrusted skill or code specs, injection via indexed
+knowledge-base documents, and unintended handling of credentials.
+
+## Supported versions
+
+DSAgt is pre-1.0 and dev-stage; security fixes are applied to the latest
+development line (currently the `0.2.x` series on `main`).

.github/workflows/ci.yml

@@ -0,0 +1,44 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+    branches: [main]
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+jobs:
+  lint:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: astral-sh/setup-uv@v5
+        with:
+          python-version: "3.12"
+      - name: Install dev tools
+        run: uv sync --group dev
+      - name: Ruff
+        run: uv run ruff check src tests
+      - name: Black
+        run: uv run black --check src tests
+
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.12"]
+    steps:
+      - uses: actions/checkout@v4
+      - uses: astral-sh/setup-uv@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+      - name: Install (all groups)
+        run: uv sync --all-groups
+      # `python -m pytest`, not bare `pytest`, per CLAUDE.md.  Integration tests
+      # need real EMBEDDING_*/LLM_* credentials and are excluded here.
+      - name: Tests (non-integration)
+        run: uv run --no-sync python -m pytest -m "not integration" -q

CLAUDE.md

@@ -57,7 +57,7 @@ The codebase separates **commands** (entry points with argparse, launched as CLI
 
 Entry points (`pyproject.toml` `[project.scripts]`): `dsagt` → `dsagt.commands.cli:main`, `dsagt-run` → `dsagt.commands.run_code:main`, `dsagt-server` → `dsagt.mcp.server:main`.
 
-**Bundled assets** (shipped as `package-data`):
+**Built-in assets** (declared as `package-data`):
 - `src/dsagt/codes/` — built-in codes as skill-standard dirs (`<name>/SKILL.md`), served from the package (never copied into projects).
 - `src/dsagt/skills/` — built-in skills (e.g., `skill-creator`) the agent discovers via `search_skills`.
 - `src/dsagt/dsagt_instructions.md` — agent-agnostic system instructions injected into per-agent files at init.
@@ -78,12 +78,16 @@ Distilled from working on this codebase; `knowledge.py` is the reference example
 
 **Comments state the real reason, at the point they explain it.** A lazy import is justified *at the import site* with its actual cause, not as an "this is absent" note in the import block citing a stale rationale. If the reason changes, fix the comment.
 
+**No change-narration in comments.** A comment describes what the code does and why it exists *now* — never how it used to work, what changed, or paradigms no longer in the tree. Ban breadcrumbs like "previously…", "was formerly…", "no longer uses…", "moved from…", "(not a `.get` default)", "instead of the old…". Git carries the history; the comment describes the present. State the hazard/intent directly ("session_id is null outside a minted session, and ChromaDB rejects null metadata") rather than contrasting with a prior version ("… beats coercing to `[]` like before").
+
 **Import hygiene on hot paths.** Modules on frequently-invoked paths (`dsagt-run` runs per tool call) must not transitively drag heavy modules in for *annotation-only* type hints. Use `from __future__ import annotations` + a `TYPE_CHECKING`-guarded import (verify the module doesn't introspect annotations at runtime first). Keep cold start lean; lazy-import the heavy leaf (llama_index) at its single use site.
 
 **Naming.** Prefer concise domain names (`APIEmbedder`/`LocalEmbedder`, not `…EmbeddingClient`).
 
 **Module docstrings (major modules).** Open with a title line + 3–5 sentences: what the module does, the capabilities it backs, the design motivations. Follow with an **ASCII-art UML class map** — one consistent notation throughout (`knowledge.py` uses `◇` holds · `◆` owns · `▷` inherits). Treat the class-map diagram as a deliverable of any **major module refactor** — refresh it whenever the class structure changes substantially.
 
+**Prose register (docs, comments, changelog, commit messages).** Plain, accurate, direct — no anthropomorphism, no code-jockey slang, no advertising gloss. Concretely: files/modules are *located in* / *defined in* / *stored in*, never "live in"; DSAgt *provides* / *includes* things, it does not "ship" or "provision" them; use *built-in*, not "bundled"; drop marketing gloss ("out of the box", "seamless", "with nothing to remember", "blazing"). State what a thing does, not how nice it is. Changelogs and commit messages record real behavior changes — pure renames and doc-only churn are noise, keep them out. This applies to this file too.
+
 ## BYOA artifacts
 
 `dsagt init --agent X --location <path>` writes, in the project dir:

README.md

@@ -237,4 +237,4 @@ Each launch gets a session id that every span carries, so you can filter the tra
 | `dsagt smoke-test [--agent claude\|goose\|codex\|opencode\|cline]` | End-to-end install verification |
 <!-- md-shared:cli:end -->
 
-For tests, troubleshooting, and other developer-facing material, see [developer.md](developer.md).
+For tests, troubleshooting, and other developer-facing material, see [docs/developer.md](docs/developer.md).

docs/developer.md

@@ -1,24 +1,74 @@
 # Developer Guide
 
-Material for contributors and developers.
+How the DSAgt codebase is set up and how to work in it. For contribution
+mechanics (branch/PR flow, commit style), see
+[CONTRIBUTING.md](https://github.com/AI-ModCon/dsagt/blob/main/.github/CONTRIBUTING.md).
+
+## Setup
+
+DSAgt develops on [uv](https://github.com/astral-sh/uv) with Python 3.12 or 3.13:
+
+```bash
+git clone https://github.com/AI-ModCon/dsagt.git
+cd dsagt
+uv sync --all-groups          # runtime + dev + docs dependencies
+source .venv/bin/activate      # so dsagt / dsagt-run / dsagt-server are on PATH
+```
 
 ## Tests
 
+Use `python -m pytest`, not bare `pytest` (the bare binary can resolve the wrong
+interpreter):
+
+```bash
+uv run --no-sync python -m pytest -m "not integration" -q   # unit suite (~640 tests)
+uv run --no-sync python -m pytest tests/test_config.py -q   # a single file
+uv run --no-sync python -m pytest -m integration -v         # integration (needs creds)
+```
+
+Integration tests hit real embedding/LLM providers and need `EMBEDDING_*` /
+`LLM_*` credentials in the environment; they're excluded from CI and the default
+local run.
+
+## Lint & format
+
+CI enforces both on `src/` and `tests/` (scientific scripts under `use_cases/`
+are exempt):
+
 ```bash
-uv run python -m pytest -m "not integration"     # unit tests, no creds required
-uv run python -m pytest -m integration -v        # integration tests (require real credentials / models)
+uv run ruff check src tests
+uv run black src tests          # omit the paths to format everything you touched
 ```
 
-For per-flow hand-tests (CLI, VS Code extensions), see the scripts under [`tests/manual_walkthroughs/`](https://github.com/AI-ModCon/dsagt/tree/main/tests/manual_walkthroughs/).
+## Docs
+
+The site is MkDocs (Material). `mkdocs.yml` at the repo root is the site config;
+`docs/` holds the pages. The `.github/workflows/docs.yml` workflow builds the
+site with `--strict` on every PR and deploys it to GitHub Pages from `main`.
+
+```bash
+uv run mkdocs serve             # live preview at http://127.0.0.1:8000
+uv run mkdocs build --strict    # what CI runs
+```
+
+## Codebase orientation
+
+The [Architecture](architecture.md) page is the map of the system — the
+capabilities, the single `dsagt-server` MCP layout, and the observability and
+memory design. `CLAUDE.md` at the repo root records the house coding and prose
+conventions (it doubles as instructions for AI coding agents working in the
+repo); read it before a substantial change.
 
 ## Troubleshooting
 
-**Agent command not found.** The agent CLI is not installed or is not on PATH. See the [supported agents table](index.md#supported-agents).
+**Agent command not found.** The agent CLI isn't installed or isn't on PATH —
+see the [supported agents](index.md#supported-agents).
 
-**MCP server not connecting.** Verify the server command resolves:
+**MCP server not connecting.** Confirm the entry point resolves:
 
 ```bash
 uv run which dsagt-server
 ```
 
-If missing, reinstall: `pip install --force-reinstall "git+https://github.com/AI-ModCon/dsagt.git"`.
+If it's missing, reinstall:
+`pip install --force-reinstall "git+https://github.com/AI-ModCon/dsagt.git"`.

docs/knowledge-base.md

@@ -2,6 +2,8 @@
 
 The knowledge base is DSAgt's catalog of **domain knowledge** — reference corpora and your own documents — that the agent searches to ground its work on scientific data-processing and AI-readiness evaluation.
 
+![DSAgt knowledge base](assets/knowledge-base.png)
+
 ## Domain-knowledge collections
 
 | Collection | Source | Populated by |

docs/memory.md

@@ -2,6 +2,8 @@
 
 DSAgt gives the agent two kinds of persistent memory backed by the project's vector store: **explicit memory** — facts the user confirms — and opt-in **episodic memory** — an automatic record of session turns. Both are retrievable by the agent with `kb_search` / `kb_get_memories` MCP tools.
 
+![DSAgt memory](assets/memory.png)
+
 ## Explicit memory
 
 Explicit memories are facts the user confirms during a session. The agent saves them via `kb_remember`, which writes to both the ChromaDB collection and `<project>/.dsagt/explicit_memories.yaml`. It fetches them via `kb_get_memories` on demand — typically when you ask it to recall something — so they are not auto-loaded at session start. 
@@ -12,6 +14,15 @@ When episodic memory is enabled, DSAgt reads the agent's transcript as the sessi
 
 Retrieval over `session_memory` filters first to a session, then by regex over the query's key terms, before a final **recency-weighted** semantic ranking: a newer turn edges out a stale one as a bounded boost, so a corrected fact wins by recency while a strongly-relevant old turn is never buried.
 
+## Comparison with platform-native memory
+
+Agent platforms provide their own memory: instruction files such as `CLAUDE.md` and `.goosehints`, and on some platforms notes the agent saves for itself. That memory is agent-curated (the model decides what is worth saving), stored as prose files loaded whole into context, and tied to one platform's format. DSAgt memory differs on each point:
+
+- **Mechanical capture.** Episodic memory records every completed turn from the transcript; nothing depends on the model choosing to save it.
+- **Retrieval on demand.** Memories are recalled by search with recency weighting, not loaded whole into context, so the record can grow without consuming the context window.
+- **One store across agents.** The same collections and YAML files serve all five supported platforms, so memory persists across a switch of agent.
+- **Auditable facts.** Explicit memories are user-confirmed and durable, with superseded entries kept in a history file.
+
 ## Try it
 
 ```bash
@@ -37,6 +48,7 @@ And **episodic** memory (captured automatically — no `remember` step):
 
 The agent recalls the decision from `session_memory` even though you never explicitly stored it. Confirm the collection materialized:
 
+
 ```bash
 ls ~/dsagt-projects/demo/kb_index/session_memory/
 ```