Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .codeguard/codeguard.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ checks:
security: false
prompts: false
ci: true
supply_chain: true
quality_rules:
max_file_lines: 400
max_function_lines: 80
Expand Down
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,7 @@ func main() {
## Docs

- [Getting started](/Users/alex/Documents/GitHub/codeguard/docs/getting-started.md:1)
- [Features](/Users/alex/Documents/GitHub/codeguard/docs/features.md:1)
- [AI-generated code quality](/Users/alex/Documents/GitHub/codeguard/docs/ai-quality.md:1)
- [Agent-native features](/Users/alex/Documents/GitHub/codeguard/docs/agent-native.md:1)
- [Integrations](/Users/alex/Documents/GitHub/codeguard/docs/integrations.md:1)
Expand All @@ -123,3 +124,4 @@ func main() {
- [Homebrew packaging](/Users/alex/Documents/GitHub/codeguard/docs/homebrew.md:1)
- [Checks reference](/Users/alex/Documents/GitHub/codeguard/docs/checks.md:1)
- [Architecture](/Users/alex/Documents/GitHub/codeguard/docs/architecture.md:1)
- [Competitive roadmap](/Users/alex/Documents/GitHub/codeguard/docs/competitive-roadmap.md:1)
2 changes: 2 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# CodeGuard Docs

- [Getting started](getting-started.md)
- [Features](features.md)
- [AI-generated code quality](ai-quality.md)
- [Agent-native features](agent-native.md)
- [Integrations](integrations.md)
Expand All @@ -10,3 +11,4 @@
- [Homebrew packaging](homebrew.md)
- [Architecture](architecture.md)
- [Checks](checks.md)
- [Competitive roadmap](competitive-roadmap.md)
57 changes: 55 additions & 2 deletions docs/ai-quality.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,16 +16,27 @@ This brief tracks the AI-generated-code quality features currently implemented i
- `quality.ai.provenance-policy`
- configurable through `checks.quality_rules.ai_provenance`
- supports `CODEGUARD_AI_ASSISTED`-style environment hints and commit trailers such as `AI-Assisted: true`
- Change-risk rollup
- `quality.ai.change-risk`
- `change_risk` report artifact
- configurable through `checks.quality_rules.ai_change_risk`
- aggregates slop-score signals, semantic findings, coverage gaps, diff breadth, and AI provenance into a review-priority score
- Consistency-with-codebase checks
- `quality.ai.local-idiom-drift`
- currently compares test framework choices against the dominant local repository idiom for Go and TypeScript/JavaScript targets
- Optional semantic review for AI-assisted diffs
- `quality.ai.semantic-doc-mismatch`
- `quality.ai.contract-drift`
- `quality.ai.semantic-error-message`
- `quality.ai.semantic-test-coverage`
- `quality.ai.semantic-test-adequacy`
- `quality.ai.semantic-runtime`
- request enrichment now adds framework metadata and contract hints for changed Express handlers and middleware, React components, and Next.js route/component files, so semantic runtimes can reason about handlers, props, route segments, request/response semantics, and middleware ordering with better local context
- runs for changed files from patch/diff input, or from a git diff against the scan base ref during full scans
- requires the semantic runtime to be explicitly enabled either through `ai.enabled` / `--ai` with a command-backed provider, or through `CODEGUARD_SEMANTIC_CHECKS=1`
- shells out to the command in `CODEGUARD_SEMANTIC_COMMAND`, sends a bounded JSON payload on stdin, and expects JSON verdicts on stdout
- uses the command from `ai.provider.type=command` plus `ai.provider.command`/`args` when configured; otherwise it falls back to `CODEGUARD_SEMANTIC_COMMAND`
- sends a bounded JSON payload on stdin and expects JSON verdicts on stdout
- emits `quality.ai.semantic-runtime` at `fail` level when semantic review is enabled but no command is configured, or when the command crashes or returns invalid JSON
- caches verdicts by request content hash in a sibling cache file next to the normal scan cache
- Hybrid AI triage for static findings
- optional provider-backed pass that tries to verify or dismiss existing findings conservatively
Expand Down Expand Up @@ -79,8 +90,50 @@ All HTTP providers (OpenAI-compatible and Anthropic, in triage and the shared ru
- Semantic review is opt-in:
- can be enabled through the normal AI runtime or through `CODEGUARD_SEMANTIC_CHECKS=1`
- scopes itself to changed files from diff or patch input, or from a git diff against the configured base ref during full scans, plus a small set of nearby test files
- `ai.semantic.function_contract`, `ai.semantic.misleading_error_messages`, and `ai.semantic.test_behavior_coverage` control which semantic prompts are sent
- includes per-file `frameworks` metadata in the JSON request when changed source snapshots match known framework patterns
- includes a structured `prompt` template in the JSON request so command-backed semantic runtimes receive explicit review instructions, response requirements, per-rule focus areas, and framework-specific reasoning guidance
- each framework entry can now carry low-level `signals` plus higher-level `hints` that summarize likely contracts such as `middleware-next-chain`, `component-props-contract`, `client-component`, `route-segment-component`, and `route-handler-contract`
- the prompt template now teaches `quality.ai.contract-drift` and `quality.ai.semantic-test-adequacy` to explicitly reason about props contracts, route `params` or `searchParams`, request or response contract shifts, and Express middleware sequencing
- current framework coverage is still intentionally narrow but broader than the first slice: Express route and middleware modules, React component files, and Next.js route/component conventions (`app/**/route.*`, `pages/api/**`, `app/**/page.*`, `app/**/layout.*`, `app/**/loading.*`, `app/**/error.*`, and `next/server` request/response patterns)
- `ai.semantic.function_contract`, `ai.semantic.contract_drift`, `ai.semantic.misleading_error_messages`, `ai.semantic.test_behavior_coverage`, and `ai.semantic.test_adequacy` control which semantic prompts are sent
- the external semantic command must read a JSON request from stdin and return `{"verdicts":[...]}` with `rule_id`, `path`, `line`, `level`, and `message`

### Reference semantic runner

The repo now ships a scaffold runner at `examples/semantic/reference_runner.py`.

Example wiring:

```bash
export CODEGUARD_SEMANTIC_CHECKS=1
export CODEGUARD_SEMANTIC_COMMAND="python3 examples/semantic/reference_runner.py"
```

What it does:
- reads the semantic request JSON from stdin
- renders the canonical prompt text from `prompt`, `frameworks`, `diff`, `source_files`, and `test_files`
- prints that prompt to stderr when `CODEGUARD_SEMANTIC_REFERENCE_PRINT_PROMPT=1`
- defaults to scaffold mode and returns `{"verdicts":[]}` so it is safe as a no-op

Backend modes:

- scaffold mode:
- `CODEGUARD_SEMANTIC_REFERENCE_MODE=scaffold`
- returns an empty `verdicts` array
- local command mode:
- `CODEGUARD_SEMANTIC_REFERENCE_MODE=command`
- `CODEGUARD_SEMANTIC_REFERENCE_LOCAL_COMMAND="python3 my-semantic-backend.py"`
- sends `{"request":<original request>,"prompt_text":"..."}` to the backend command on stdin
- expects `{"verdicts":[...]}` on stdout
- OpenAI-compatible mode:
- `CODEGUARD_SEMANTIC_REFERENCE_MODE=openai`
- `CODEGUARD_SEMANTIC_REFERENCE_OPENAI_BASE_URL=https://api.openai.com/v1`
- `CODEGUARD_SEMANTIC_REFERENCE_OPENAI_API_KEY=...`
- `CODEGUARD_SEMANTIC_REFERENCE_OPENAI_MODEL=gpt-5`
- posts the rendered prompt to `/chat/completions` and expects the model message content to be a JSON object with `verdicts`

This is intended as the canonical starting point for custom semantic commands. The prompt assembly stays fixed in one place, and you can swap only the backend transport.

- Verified auto-fix is fail-closed:
- fix generation requires an explicit AI provider plus `-ai`
- the proposed diff must apply cleanly, pass a diff-scoped `codeguard` rerun, and pass inferred or explicit verification tests
Expand Down
1 change: 1 addition & 0 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ The public SDK stays stable for consumers at `github.com/devr-tools/codeguard/pk
- `internal/codeguard/checks/security/` holds heuristic and vulnerability checks
- `internal/codeguard/checks/prompts/` holds prompt-safety checks
- `internal/codeguard/checks/ci/` holds repository and workflow policy checks
- `internal/codeguard/checks/supplychain/` holds dependency-policy and SBOM-oriented checks
- `internal/codeguard/checks/support/` holds the shared adapter surface used by the check packages

This split keeps a single implementation path while leaving room to add language-specific rules under each category as the scanner expands beyond Go.
Expand Down
92 changes: 90 additions & 2 deletions docs/checks.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,78 @@ This file documents the current check categories in `codeguard` and the config k
"design": true,
"security": true,
"prompts": true,
"ci": true
"ci": true,
"supply_chain": false
}
}
```

Each top-level boolean enables or disables an entire check family.

`supply_chain` is opt-in and currently covers normalized manifest parsing plus initial policy checks for missing lockfiles, content-based lockfile drift validation, unpinned dependencies, and dependency license policy resolved from local manifest and installed metadata where available.

For ecosystems where local metadata is not present, `supply_chain_rules.license_commands` can provide an opt-in per-ecosystem command that prints JSON license mappings for unresolved dependencies.

Each license command receives structured context through environment variables:
- `CODEGUARD_SUPPLY_CHAIN_ECOSYSTEM`
- `CODEGUARD_SUPPLY_CHAIN_MANIFEST_PATH`
- `CODEGUARD_SUPPLY_CHAIN_TARGET_NAME`
- `CODEGUARD_SUPPLY_CHAIN_TARGET_PATH`
- `CODEGUARD_SUPPLY_CHAIN_UNRESOLVED_NAMES`
- `CODEGUARD_SUPPLY_CHAIN_UNRESOLVED_COORDINATES`
- `CODEGUARD_SUPPLY_CHAIN_CONTEXT_FILE`

`CODEGUARD_SUPPLY_CHAIN_CONTEXT_FILE` points to a JSON payload containing the ecosystem, manifest path, target metadata, and unresolved dependency entries with:
- `coordinate`
- `name`
- `requirement`
- `version`
- `scope`
- `groups`
- `indirect`
- `pinned`
- `line`

License commands may return either `name`-keyed results for backward compatibility or `coordinate`-keyed results such as `left-pad@1.3.0` to disambiguate multiple versions of the same dependency.

Supported result shapes:

```json
[
{
"coordinate": "left-pad@1.3.0",
"license": "MIT",
"source": "license-command"
}
]
```

Or a richer candidate form:

```json
[
{
"coordinate": "left-pad@1.3.0",
"candidates": [
{
"license": "MIT",
"confidence": "high",
"provenance": "spdx-expression",
"source": "license-command"
},
{
"license": "GPL-3.0",
"confidence": "low",
"provenance": "heuristic-text-match",
"source": "license-command"
}
]
}
]
```

When multiple candidates are returned, CodeGuard prefers stronger evidence such as explicit SPDX provenance or high-confidence results. Heuristic-only matches still inform policy, but they are surfaced as weaker evidence rather than treated the same as definitive metadata.

## Exclusions

Purpose:
Expand Down Expand Up @@ -187,7 +252,13 @@ Current behavior:
- includes an AI-failure-mode pack for swallowed errors, narrative comments, hallucinated imports, plausible dead code, over-mocked tests, and codebase-idiom drift in Go, TypeScript, and JavaScript targets
- publishes a `slop_score` artifact in the report when AI-failure-mode signals are present so CI systems can trend the metric over time
- can apply a provenance-aware policy for AI-assisted changes through `quality_rules.ai_provenance` using environment hints or commit trailers
- can optionally run command-backed semantic review for changed files from diff/patch input, or from a git diff against the scan base ref during full scans, when a semantic runtime is enabled and `CODEGUARD_SEMANTIC_COMMAND` is set
- can publish a `change_risk` artifact and emit `quality.ai.change-risk` when AI-style and review-risk signals accumulate past configured thresholds
- can optionally run command-backed semantic review for changed files from diff/patch input, or from a git diff against the scan base ref during full scans, when a semantic runtime is enabled and a semantic command is configured either through `ai.provider.type=command` plus `ai.provider.command`/`args`, or through `CODEGUARD_SEMANTIC_COMMAND`
- if semantic review is enabled but no semantic command is configured, or the command crashes or returns invalid JSON, the scan emits `quality.ai.semantic-runtime` at `fail` level instead of silently skipping semantic coverage
- semantic review can also emit `quality.ai.contract-drift` when a changed function appears to silently drift from its prior behavior or nearby contract signals
- semantic review can also emit `quality.ai.semantic-test-adequacy` when nearby tests appear too weak, too happy-path, or otherwise inadequate for the changed behavior
- semantic review request payloads now include lightweight framework metadata plus contract hints for changed Express handlers and middleware, React components, and Next.js route/component files so external semantic runtimes can reason with handler-aware and component-aware context
- semantic review request payloads also include a structured `prompt` template with per-rule focus and framework-specific reasoning guidance, so command-backed runtimes do not have to invent their own contract-drift or test-adequacy instructions from scratch
- TypeScript and JavaScript quality built-ins use AST-derived function metrics and compiler-parsed syntax when the semantic runtime is available
- includes native maintainability heuristics for Python, TypeScript, JavaScript, Rust, Java, C#, and Ruby targets
- TypeScript and JavaScript targets also warn on `@ts-ignore`, `@ts-nocheck`, `@ts-expect-error`, explicit `any`, double assertions, non-null assertions, and committed `debugger` statements
Expand All @@ -212,6 +283,23 @@ AI provenance example:
}
```

AI change-risk example:

```json
{
"checks": {
"quality": true,
"quality_rules": {
"ai_change_risk": {
"enabled": true,
"warn_threshold": 30,
"fail_threshold": 60
}
}
}
}
```

Language command example:

```json
Expand Down
Loading
Loading