diff --git a/CHANGELOG.md b/CHANGELOG.md index 58d3fca..5adf434 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,24 @@ AgentExperience.NET is a **preview**. It is not production ready, and public API ## Unreleased +### Added + +- **`aexp init [--print] [--force] [--verify ]`** writes a commented `.aexp/config.json` from one command, + run in the project root, with no agent named: the verifier detected when one kind of project is at the root + (`dotnet test`, `npm test`, `pytest`, `go test ./...`; and now `make test`, for a `Makefile` with a `test:` target, + proposed only when none of those fits), + asked for at a terminal when none or several fit (a candidate's number, a JSON array, or Enter for the placeholder), + or given with `--verify` as a JSON array (never a shell line, never split on spaces). It adds `.aexp/` to + `.gitignore` when the directory is under git, never touches an existing config (`--force` replaces it, revoking its + trust like any edit), and prints the steps left to a person: `aexp trust`, then `aexp setup claude-code` or + `claude mcp add`. It approves nothing. `aexp setup claude-code` writes its config and `.gitignore` entry through the + same generator (`ProjectInit`), so there is one. See [CLI and MCP](docs/guide/cli-and-mcp.md#write-it-aexp-init). + +### Changed + +- The config `aexp setup claude-code` writes carries a `//` comment on each choice, as `aexp init`'s does; the loader + has always skipped comments. Its values are unchanged. + ## 0.1.0-preview.19 This preview adds an experimental, off-by-default reflective execution loop. The same tag releases the eight NuGet packages, the Python distribution and the Go modules at this version, as before. diff --git a/README.md b/README.md index da95cde..8578dbb 100644 --- a/README.md +++ b/README.md @@ -67,10 +67,12 @@ preview, `0.1.0-preview.19`; until then, [build it from this repository](dogfood ```bash dotnet tool install --global AgentExperience.Cli --prerelease +aexp init # in the project root: detects the test command, writes .aexp/config.json +aexp trust # after you reviewed the config: a person's step claude mcp add aexp -- aexp mcp # or any MCP client: the server is `aexp mcp`, run in the project ``` -A `.aexp/config.json` in the project names the scope and the one command `aexp` may run, the verifier +`.aexp/config.json` names the scope and the one command `aexp` may run, the verifier (`"verify": { "command": ["dotnet", "test"] }`, exit code 0 = pass), which runs only once a person has reviewed the config and approved it with `aexp trust` (an edit revokes the approval). The MCP tools `experience_recall`, `experience_failure_hint`, `experience_record_call`, `experience_finish` and `experience_lessons` (or the commands diff --git a/docs/guide/claude-code.md b/docs/guide/claude-code.md index f9210af..cec826e 100644 --- a/docs/guide/claude-code.md +++ b/docs/guide/claude-code.md @@ -26,12 +26,14 @@ aexp trust # after you rev - **adds the hooks to `.claude/settings.json`**, merged with what is there: every existing hook, matcher and setting is kept, and an event that already has an `aexp hook` command gets no second one. A settings file that is not plain JSON (with comments, say) is left alone, and setup tells you to add the block by hand. Running setup again changes nothing. -- **writes `.aexp/config.json`** if there is none (an existing one is never touched), with the identity named after the - directory, the holdout at 20% so the benefit can be measured, `file_path` as the only argument kept from tool calls, and - the verifier detected from the project when it is obvious: `dotnet test` (a solution or project file at the root), - `npm test` (a `package.json` with a test script), `pytest` (`pyproject.toml`, `pytest.ini`, `conftest.py`, `tox.ini`) - or `go test ./...` (`go.mod`). When none or several match, it writes a placeholder that refuses to run until you set - `verify.command`. +- **writes `.aexp/config.json`** if there is none (an existing one is never touched), through the same generator as + [`aexp init`](cli-and-mcp.md#write-it-aexp-init): the identity named after the directory with `"agent": "claude-code"`, + the holdout at 20% so the benefit can be measured, `file_path` as the only argument kept from tool calls, a comment on + each choice, and the verifier detected from the project when it is obvious: `dotnet test` (a solution or project file + at the root), `npm test` (a `package.json` with a test script), `pytest` (`pyproject.toml`, `pytest.ini`, + `conftest.py`, `tox.ini`) or `go test ./...` (`go.mod`), and `make test` (a `Makefile` with a `test:` target) only when + none of those fits. When none or several match, it writes a placeholder that refuses to run until you set `verify.command` (setup asks nothing; run + `aexp init` first to choose from the candidates or pass `--verify`). - **adds `.aexp/` to `.gitignore`**: the store, open runs, sessions, the hook log and the config stay local. - **prints the next steps**, including the permission rules below. It never approves the config. diff --git a/docs/guide/cli-and-mcp.md b/docs/guide/cli-and-mcp.md index db442ab..dc61afe 100644 --- a/docs/guide/cli-and-mcp.md +++ b/docs/guide/cli-and-mcp.md @@ -14,6 +14,7 @@ the [injection holdout](injection.md), the [lessons view](deployment.md#seeing-w ```bash dotnet tool install --global AgentExperience.Cli --prerelease aexp --version +aexp init # in the project root: writes .aexp/config.json and the .gitignore entry ``` It needs the .NET 10 runtime. The SQLite file is created (owner-only) on first use. @@ -26,6 +27,30 @@ file `--config ` or the `AEXP_CONFIG` environment variable names. Relative root, the directory that holds `.aexp/`. The tool refuses to start without a config, without an identity, or without a verifier, and on Unix it refuses a config (or config directory) another user owns. +### Write it: `aexp init` + +`aexp init [--print] [--force] [--verify ]`, run in the project root, writes a commented `.aexp/config.json` +with the defaults below (the SQLite store in `.aexp/`, the identity named after the directory, a 20% holdout, `file_path` +as the only argument kept from tool calls, recall on, a 540 s timeout under the Claude Code Stop hook's 600 s, check +`tests`), adds `.aexp/` to `.gitignore` in that directory when it is under git (a `.gitignore` there, or a `.git` there or +in a directory above it, up to your home directory), and prints the steps left to you: review it, `aexp trust`, then +`aexp setup claude-code` or `claude mcp add`. It never approves the config. The verifier is: + +- **detected** when exactly one kind of project is at the root: `dotnet test` (a `.sln`, `.slnx`, `.csproj` or + `.fsproj`), `npm test` (a `package.json` whose `scripts.test` is set), `pytest` (`pyproject.toml`, `pytest.ini`, + `conftest.py` or `tox.ini`) or `go test ./...` (`go.mod`); when none of those fits, `make test` (a `Makefile` with a + `test:` target); +- **asked for** when none or several match and you are at a terminal: a number from the list of candidates, a JSON + array, or Enter for a placeholder to set later. A wrong answer is asked again once; then the placeholder. When standard + input is not a terminal, the placeholder is written and the candidates are printed; +- **given** with `--verify '["npm","test","--","--ci"]'`: a JSON array of non-empty strings, never a shell line and + never split on spaces (anything else is a usage error, exit 2). + +The placeholder refuses to run until `verify.command` is set. An existing config is never touched (exit 2, naming +`--force`); `--force` replaces it whole, which revokes its `aexp trust` approval like any other edit. `--print` prints the +config and the `.gitignore` line and writes nothing. The file is written atomically, `.aexp/` owner-only (`0700`) and the +config owner read and write. Comments (`//`) are allowed in the config; the loader skips them. + ### Approve it: `aexp trust` A config names a command, so **a config from an untrusted checkout is code execution**. Like direnv, `aexp` runs the @@ -138,7 +163,8 @@ aexp finalize --run "$run" # runs the verifier; | `aexp mcp` | Serves the MCP tools over stdin and stdout. | | `aexp trust` | Prints what the config would run, and approves the config (see above). | | `aexp runs prune --older-than ` | Deletes open runs (never ended) started longer ago than `7d`, `12h` or `30m`, unless a command is using them, and Claude Code session files not touched for as long. | -| `aexp setup claude-code [--print]` | Adds the Claude Code hooks to `.claude/settings.json`, a default config and the `.gitignore` entry ([Claude Code](claude-code.md)). | +| `aexp init [--print] [--force] [--verify ]` | Writes a commented `.aexp/config.json` with the verifier detected, asked for or given, and the `.gitignore` entry; prints the next steps. Never trusts the config ([above](#write-it-aexp-init)). | +| `aexp setup claude-code [--print]` | Adds the Claude Code hooks to `.claude/settings.json`, the same default config when there is none, and the `.gitignore` entry ([Claude Code](claude-code.md)). | | `aexp hook [--timeout ]` | A Claude Code hook: reads the event's JSON on standard input, prints the hook output; always exits 0 ([Claude Code](claude-code.md)). | Exit codes: 0 done, 1 failed, 2 a usage error (including a run that is not open), 3 a configuration the tool refuses diff --git a/src/AgentExperience.Cli/AexpCli.cs b/src/AgentExperience.Cli/AexpCli.cs index e8e4a99..9d407aa 100644 --- a/src/AgentExperience.Cli/AexpCli.cs +++ b/src/AgentExperience.Cli/AexpCli.cs @@ -10,7 +10,10 @@ namespace AgentExperience.Cli; -/// What one invocation of the tool runs with: its streams, working directory, environment and clock. +/// +/// What one invocation of the tool runs with: its streams, working directory, environment and clock. ReadLine +/// reads one line a person typed; means standard input is not a terminal, so nothing asks. +/// internal sealed record AexpConsole( TextWriter Out, TextWriter Error, @@ -18,7 +21,8 @@ internal sealed record AexpConsole( Func Environment, TimeProvider Clock, Stream? StandardInput = null, - Stream? StandardOutput = null); + Stream? StandardOutput = null, + Func? ReadLine = null); /// /// The aexp command line. Plain parsing, no framework: each command takes named options only, and every value @@ -50,6 +54,9 @@ Export lessons measured as helping (to a file, or aexp mcp Serve the MCP tools over stdio. aexp trust Show the config's verifier, and approve the config. aexp runs prune --older-than Delete open runs older than e.g. 7d, 12h, 30m. + aexp init [--print] [--force] [--verify ] + Write .aexp/config.json (verifier detected, asked or + given) and the .gitignore entry; never trusts it. aexp setup claude-code [--print] Add the Claude Code hooks, a config and .gitignore entry. aexp hook [--timeout ] [--claude-md [file]] [--agents-md [file]] A Claude Code hook (reads the event JSON on stdin). @@ -89,18 +96,26 @@ public static async Task RunAsync(string[] args, AexpConsole console, Cance return parsed.Command.Count == 0 && !parsed.Flag("help") ? Usage : Ok; } - // Setup runs before there is a config. - if (parsed.Command is ["setup", ..]) + // Init and setup run before there is a config. + if (parsed.Command is ["init", ..] or ["setup", ..]) { try { - return await ClaudeCodeSetup.RunAsync(parsed, console, cancellationToken).ConfigureAwait(false); + return parsed.Command[0] == "init" + ? await ProjectInit.RunAsync(parsed, console, cancellationToken).ConfigureAwait(false) + : await ClaudeCodeSetup.RunAsync(parsed, console, cancellationToken).ConfigureAwait(false); } catch (AexpUsageException ex) { await console.Error.WriteLineAsync("aexp: " + ex.Message + (ex.Detail is null ? string.Empty : " (" + ex.Detail + ")")).ConfigureAwait(false); return Usage; } + catch (Exception ex) when (ex is not OperationCanceledException) + { + // A write that failed (permissions, a full disk): the type and message only, never a stack trace. + await console.Error.WriteLineAsync($"aexp: failed ({ex.GetType().Name}): {ex.Message}").ConfigureAwait(false); + return Failed; + } } AexpSettings settings; @@ -332,7 +347,7 @@ internal static TimeSpan ParseAge(string text) /// Command words, then --name value options and bare --flags. internal sealed class Arguments { - private static readonly HashSet Flags = new(StringComparer.Ordinal) { "failure", "all", "help", "version", "print" }; + private static readonly HashSet Flags = new(StringComparer.Ordinal) { "failure", "all", "help", "version", "print", "force" }; /// Options whose value may be left out (the next argument is theirs unless it is another option). private static readonly HashSet OptionalValues = new(StringComparer.Ordinal) { "claude-md", "agents-md" }; diff --git a/src/AgentExperience.Cli/ClaudeCode/ClaudeCodeSetup.cs b/src/AgentExperience.Cli/ClaudeCode/ClaudeCodeSetup.cs index 7e2f8d6..93a8f22 100644 --- a/src/AgentExperience.Cli/ClaudeCode/ClaudeCodeSetup.cs +++ b/src/AgentExperience.Cli/ClaudeCode/ClaudeCodeSetup.cs @@ -9,8 +9,8 @@ namespace AgentExperience.Cli.ClaudeCode; /// /// aexp setup claude-code [--print], run in the project root: adds aexp's hooks to the project's /// .claude/settings.json (merged: every existing hook and setting stays, and a hook already there is not added -/// twice), writes a default .aexp/config.json when there is none (never over one), and adds .aexp/ to -/// .gitignore. Idempotent. --print writes nothing and prints what it would add. It never trusts the +/// twice), writes a default .aexp/config.json when there is none (never over one; the same generator as +/// aexp init, see ), and adds .aexp/ to .gitignore. Idempotent. --print writes nothing and prints what it would add. It never trusts the /// config: approving the verifier stays a person's step. /// internal static class ClaudeCodeSetup @@ -79,6 +79,11 @@ public static async Task RunAsync(AexpCli.Arguments args, AexpConsole conso } args.Only("print"); + if (args.Value("config") is not null) + { + throw new AexpUsageException("'setup' writes ./.aexp/config.json in the working directory and takes no --config."); + } + var print = args.Flag("print"); var root = Path.GetFullPath(console.WorkingDirectory); var output = console.Out; @@ -124,54 +129,34 @@ public static async Task RunAsync(AexpCli.Arguments args, AexpConsole conso } } - // 2. The config, only when there is none. + // 2. The config, only when there is none: the same generator as 'aexp init', with the agent named. var configPath = Path.Combine(root, AexpConfigLoader.DefaultRelativePath); - var verifier = DetectVerifier(root); if (File.Exists(configPath)) { await output.WriteLineAsync(".aexp/config.json: exists; left as it is.").ConfigureAwait(false); } - else if (print) - { - await output.WriteLineAsync(".aexp/config.json would be written:").ConfigureAwait(false); - await output.WriteLineAsync(DefaultConfig(root, verifier)).ConfigureAwait(false); - } else { - var directory = Path.GetDirectoryName(configPath)!; - if (OperatingSystem.IsWindows()) + var candidates = ProjectInit.Detect(root); + var verifier = candidates.Count == 1 ? candidates[0].Command : null; + var config = ProjectInit.Render(root, verifier, agent: "claude-code"); + if (print) { - Directory.CreateDirectory(directory); + await output.WriteLineAsync(".aexp/config.json would be written:").ConfigureAwait(false); + await output.WriteLineAsync(config).ConfigureAwait(false); } else { - Directory.CreateDirectory(directory, UnixFileMode.UserRead | UnixFileMode.UserWrite | UnixFileMode.UserExecute); + ProjectInit.WriteConfig(configPath, config, overwrite: false); + await output.WriteLineAsync(verifier is null + ? ".aexp/config.json: written. No test command was obvious, so verify.command is a placeholder that refuses to run: set it" + + (candidates.Count == 0 ? "." : $" (candidates: {string.Join("; ", candidates)}).") + : $".aexp/config.json: written, verifier {JsonSerializer.Serialize(verifier)} (detected; check it).").ConfigureAwait(false); } - - AtomicFile.Write(configPath, Encoding.UTF8.GetBytes(DefaultConfig(root, verifier)), overwrite: false); - await output.WriteLineAsync(verifier is null - ? ".aexp/config.json: written. No test command was obvious, so verify.command is a placeholder that refuses to run: set it." - : $".aexp/config.json: written, verifier {JsonSerializer.Serialize(verifier)} (detected; check it).").ConfigureAwait(false); } // 3. Keep the store, runs, sessions and config out of version control. - var gitignore = Path.Combine(root, ".gitignore"); - var ignoreText = File.Exists(gitignore) ? await File.ReadAllTextAsync(gitignore, cancellationToken).ConfigureAwait(false) : string.Empty; - if (Ignores(ignoreText)) - { - await output.WriteLineAsync(".gitignore: already ignores .aexp/.").ConfigureAwait(false); - } - else if (print) - { - await output.WriteLineAsync(".gitignore would get the line: .aexp/").ConfigureAwait(false); - } - else - { - var newline = ignoreText.Contains("\r\n", StringComparison.Ordinal) ? "\r\n" : "\n"; - var prefix = ignoreText.Length == 0 || ignoreText.EndsWith('\n') ? string.Empty : newline; - await File.AppendAllTextAsync(gitignore, prefix + "# aexp: the local lesson store, open runs and the config" + newline + ".aexp/" + newline, cancellationToken).ConfigureAwait(false); - await output.WriteLineAsync(".gitignore: added .aexp/.").ConfigureAwait(false); - } + await output.WriteLineAsync(await ProjectInit.EnsureGitignoreAsync(root, print, onlyUnderGit: false, console.Environment, cancellationToken).ConfigureAwait(false)).ConfigureAwait(false); await output.WriteLineAsync(NextSteps(console.Environment)).ConfigureAwait(false); return AexpCli.Ok; @@ -274,84 +259,6 @@ private static bool HasAexpHook(JsonObject group, string hookEvent) => && value.TryGetValue(out var command) && (command.Contains($"aexp hook {hookEvent} ", StringComparison.Ordinal) || command.TrimEnd().EndsWith($"aexp hook {hookEvent}", StringComparison.Ordinal))); - /// The project's test command when it is obvious (exactly one kind of project at the root); otherwise . - internal static IReadOnlyList? DetectVerifier(string root) - { - var found = new List(); - if (Directory.EnumerateFiles(root, "*.sln").Any() || Directory.EnumerateFiles(root, "*.slnx").Any() - || Directory.EnumerateFiles(root, "*.csproj").Any() || Directory.EnumerateFiles(root, "*.fsproj").Any()) - { - found.Add(["dotnet", "test"]); - } - - if (File.Exists(Path.Combine(root, "package.json")) && HasTestScript(Path.Combine(root, "package.json"))) - { - found.Add(["npm", "test"]); - } - - if (new[] { "pytest.ini", "pyproject.toml", "conftest.py", "tox.ini" }.Any(name => File.Exists(Path.Combine(root, name)))) - { - found.Add(["pytest"]); - } - - if (File.Exists(Path.Combine(root, "go.mod"))) - { - found.Add(["go", "test", "./..."]); - } - - return found.Count == 1 ? found[0] : null; - } - - private static bool HasTestScript(string packageJson) - { - try - { - return JsonNode.Parse(File.ReadAllText(packageJson)) is JsonObject package - && package["scripts"] is JsonObject scripts - && scripts["test"] is JsonValue test - && test.TryGetValue(out var script) - && !script.Contains("no test specified", StringComparison.Ordinal); - } - catch (Exception ex) when (ex is JsonException or IOException) - { - return false; - } - } - - internal static string DefaultConfig(string root, IReadOnlyList? verifier) - { - var name = new string([.. Path.GetFileName(Path.TrimEndingDirectorySeparator(root)).Where(c => !char.IsControl(c))]).Trim(); - if (name.Length == 0) - { - name = "project"; - } - else if (name.Length > 64) - { - name = name[..64]; - } - - var config = new JsonObject - { - ["store"] = AexpConfigLoader.DefaultStore, - ["identity"] = new JsonObject { ["tenant"] = "local", ["application"] = name, ["project"] = name, ["agent"] = "claude-code" }, - ["verify"] = new JsonObject - { - ["command"] = new JsonArray([.. (verifier ?? [VerifierRunner.Placeholder]).Select(part => (JsonNode?)JsonValue.Create(part))]), - // Below the Stop hook's timeout, so the hook can still end and finalize around it. - ["timeoutSeconds"] = 540, - ["checkId"] = "tests", - }, - ["sanitization"] = new JsonObject { ["allowArguments"] = new JsonArray("file_path") }, - ["injection"] = new JsonObject { ["holdoutRate"] = 0.2 }, - ["recall"] = new JsonObject { ["enabled"] = true }, - }; - return config.ToJsonString(Indented) + "\n"; - } - - private static bool Ignores(string gitignore) => - gitignore.Replace("\r\n", "\n", StringComparison.Ordinal).Split('\n').Select(line => line.Trim()) - .Any(line => line is ".aexp" or ".aexp/" or "/.aexp" or "/.aexp/" or ".aexp/*" or "/.aexp/*" or ".aexp/**" or "/.aexp/**"); - private static string NextSteps(Func environment) { var deny = new JsonObject { ["permissions"] = new JsonObject { ["deny"] = new JsonArray([.. RecommendedDenyRules(environment).Select(rule => (JsonNode?)JsonValue.Create(rule))]) } }; diff --git a/src/AgentExperience.Cli/Configuration/ProjectInit.cs b/src/AgentExperience.Cli/Configuration/ProjectInit.cs new file mode 100644 index 0000000..be40eb5 --- /dev/null +++ b/src/AgentExperience.Cli/Configuration/ProjectInit.cs @@ -0,0 +1,396 @@ +using System.Globalization; +using System.Text; +using System.Text.Json; +using System.Text.Json.Nodes; +using System.Text.RegularExpressions; +using AgentExperience.Cli.Engine; + +namespace AgentExperience.Cli.Configuration; + +/// +/// aexp init [--print] [--force] [--verify <json array>], run in the project root: detects the project's kind +/// and proposes its test command as the verifier, writes a commented .aexp/config.json, adds .aexp/ to +/// .gitignore when the project is under git, and prints the steps left to a person. It is the one config generator: +/// aexp setup claude-code writes its config and .gitignore entry through the same methods. It never trusts +/// the config: approving the verifier stays a person's step. +/// +internal static class ProjectInit +{ + /// A test command the project's files suggest, and the file that suggested it. + internal sealed record Candidate(string Evidence, IReadOnlyList Command) + { + public override string ToString() => $"{JsonSerializer.Serialize(Command)} ({Evidence})"; + } + + /// The line .gitignore gets, with the comment above it. + internal const string GitignoreEntry = ".aexp/"; + + private const string GitignoreComment = "# aexp: the local lesson store, open runs and the config"; + + // A target line ('test:' or 'test::'), not a variable ('test :=', 'test ::='). + private static readonly Regex MakeTestTarget = new(@"^test\s*:(?!:*=)", RegexOptions.Multiline | RegexOptions.CultureInvariant); + + public static async Task RunAsync(AexpCli.Arguments args, AexpConsole console, CancellationToken cancellationToken) + { + if (args.Command is not ["init"]) + { + throw new AexpUsageException("Usage: aexp init [--print] [--force] [--verify ] (run in the project root)."); + } + + args.Only("print", "force", "verify"); + if (args.Value("config") is not null) + { + throw new AexpUsageException("'init' writes ./.aexp/config.json in the working directory and takes no --config."); + } + + var print = args.Flag("print"); + var force = args.Flag("force"); + var root = Path.GetFullPath(console.WorkingDirectory); + var configPath = Path.Combine(root, AexpConfigLoader.DefaultRelativePath); + var output = console.Out; + + if (File.Exists(configPath) && !print && !force) + { + await console.Error.WriteLineAsync( + $"aexp: {AexpConfigLoader.DefaultRelativePath} exists; nothing written. Run 'aexp init --force' to replace it " + + "(the replacement revokes any 'aexp trust' approval of the old one), or edit it in place.").ConfigureAwait(false); + return AexpCli.Usage; + } + + // 1. The verifier: given, detected, asked for, or the placeholder. + IReadOnlyList? verifier; + if (args.Value("verify") is { } given) + { + verifier = ParseVerifier(given) + ?? throw new AexpUsageException("""--verify is a JSON array of non-empty strings, the program then its arguments: --verify '["npm","test"]'. It is never split on spaces."""); + } + else + { + var candidates = Detect(root); + if (candidates.Count == 1) + { + verifier = candidates[0].Command; + await output.WriteLineAsync($"Detected verifier: {candidates[0]}.").ConfigureAwait(false); + } + else + { + await output.WriteLineAsync(candidates.Count == 0 + ? "No test command was detected at the root (looked for a .NET solution or project, package.json with a test script, pytest files, go.mod, a Makefile with a test target)." + : "Several test commands fit this root:").ConfigureAwait(false); + for (var i = 0; i < candidates.Count; i++) + { + await output.WriteLineAsync($" {(i + 1).ToString(CultureInfo.InvariantCulture)}. {candidates[i]}").ConfigureAwait(false); + } + + verifier = console.ReadLine is null + ? null + : await AskAsync(candidates, console).ConfigureAwait(false); + if (verifier is null) + { + await output.WriteLineAsync( + "verify.command is a placeholder that refuses to run: set it in .aexp/config.json to this project's test command " + + """(a JSON array, e.g. ["npm","test"]), or run 'aexp init --force --verify '["npm","test"]''.""").ConfigureAwait(false); + } + } + } + + var json = Render(root, verifier, agent: null); + + // 2. The config, and the .gitignore entry. + if (print) + { + await output.WriteLineAsync(File.Exists(configPath) + ? $"{AexpConfigLoader.DefaultRelativePath} exists; this is what 'aexp init --force' would replace it with (nothing written):" + : $"{AexpConfigLoader.DefaultRelativePath} would be written (nothing written):").ConfigureAwait(false); + await output.WriteAsync(json).ConfigureAwait(false); + await output.WriteLineAsync(await EnsureGitignoreAsync(root, print: true, onlyUnderGit: true, console.Environment, cancellationToken).ConfigureAwait(false)).ConfigureAwait(false); + return AexpCli.Ok; + } + + var existed = File.Exists(configPath); + WriteConfig(configPath, json, overwrite: force); + await output.WriteLineAsync(existed + ? $"{AexpConfigLoader.DefaultRelativePath}: replaced. Any 'aexp trust' approval of the old file no longer applies: the bytes changed." + : $"{AexpConfigLoader.DefaultRelativePath}: written.").ConfigureAwait(false); + await output.WriteLineAsync(await EnsureGitignoreAsync(root, print: false, onlyUnderGit: true, console.Environment, cancellationToken).ConfigureAwait(false)).ConfigureAwait(false); + await output.WriteLineAsync(NextSteps).ConfigureAwait(false); + return AexpCli.Ok; + } + + private const string NextSteps = """ + + Next (a person's steps, not the agent's; init wrote a config and approved nothing): + 1. Review .aexp/config.json: verify.command is the one command aexp runs to check the agent's work. + 2. Run 'aexp trust' yourself. Until then nothing is verified and no lesson is made; any edit to the config revokes + the approval. + 3. Connect your agent: 'aexp setup claude-code' adds the Claude Code hooks, or register the MCP server with + 'claude mcp add aexp -- aexp mcp' (any MCP client: the server is 'aexp mcp', run in this directory). + """; + + /// + /// Asks for the verifier on the terminal: a candidate's number, a JSON array, or nothing for the placeholder. An + /// answer that is neither is asked again once; then the placeholder. + /// + private static async Task?> AskAsync(IReadOnlyList candidates, AexpConsole console) + { + var prompt = (candidates.Count > 0 ? "Verifier: a number from the list, " : "Verifier: ") + + """a JSON array such as ["npm","test"], or Enter for a placeholder to set later: """; + for (var attempt = 0; attempt < 2; attempt++) + { + await console.Out.WriteAsync(prompt).ConfigureAwait(false); + await console.Out.FlushAsync().ConfigureAwait(false); + var answer = console.ReadLine!()?.Trim(); + if (string.IsNullOrEmpty(answer)) + { + return null; + } + + if (int.TryParse(answer, NumberStyles.None, CultureInfo.InvariantCulture, out var number) && number >= 1 && number <= candidates.Count) + { + return candidates[number - 1].Command; + } + + if (ParseVerifier(answer) is { } command) + { + return command; + } + + await console.Out.WriteLineAsync( + "Not a listed number or a JSON array of non-empty strings" + (attempt == 0 ? "; once more." : "; the placeholder it is.")).ConfigureAwait(false); + } + + return null; + } + + /// A JSON array of non-empty strings, or for anything else (text is never split on spaces). + internal static IReadOnlyList? ParseVerifier(string text) + { + try + { + if (JsonNode.Parse(text) is not JsonArray array || array.Count == 0) + { + return null; + } + + var parts = new List(array.Count); + foreach (var item in array) + { + if (item is not JsonValue value || !value.TryGetValue(out var part) || string.IsNullOrWhiteSpace(part)) + { + return null; + } + + parts.Add(part); + } + + return parts; + } + catch (JsonException) + { + return null; + } + } + + /// Every test command the files at the root suggest, in a fixed order; one candidate is an obvious verifier. + internal static IReadOnlyList Detect(string root) + { + var found = new List(); + var dotnet = new[] { "*.sln", "*.slnx", "*.csproj", "*.fsproj" }.SelectMany(pattern => Directory.EnumerateFiles(root, pattern)).Select(Path.GetFileName).FirstOrDefault(); + if (dotnet is not null) + { + found.Add(new Candidate(dotnet, ["dotnet", "test"])); + } + + if (File.Exists(Path.Combine(root, "package.json")) && HasTestScript(Path.Combine(root, "package.json"))) + { + found.Add(new Candidate("package.json, scripts.test", ["npm", "test"])); + } + + if (new[] { "pytest.ini", "pyproject.toml", "conftest.py", "tox.ini" }.FirstOrDefault(name => File.Exists(Path.Combine(root, name))) is { } python) + { + found.Add(new Candidate(python, ["pytest"])); + } + + if (File.Exists(Path.Combine(root, "go.mod"))) + { + found.Add(new Candidate("go.mod", ["go", "test", "./..."])); + } + + // A Makefile usually wraps one of the above, so 'make test' is proposed only when nothing else fits. + if (found.Count == 0 + && new[] { "Makefile", "makefile", "GNUmakefile" }.FirstOrDefault(name => File.Exists(Path.Combine(root, name))) is { } makefile + && HasMakeTestTarget(Path.Combine(root, makefile))) + { + found.Add(new Candidate($"{makefile}, target test", ["make", "test"])); + } + + return found; + } + + private static bool HasTestScript(string packageJson) + { + try + { + return JsonNode.Parse(File.ReadAllText(packageJson)) is JsonObject package + && package["scripts"] is JsonObject scripts + && scripts["test"] is JsonValue test + && test.TryGetValue(out var script) + && !script.Contains("no test specified", StringComparison.Ordinal); + } + catch (Exception ex) when (ex is JsonException or IOException or UnauthorizedAccessException) + { + return false; + } + } + + private static bool HasMakeTestTarget(string makefile) + { + try + { + return MakeTestTarget.IsMatch(File.ReadAllText(makefile).Replace("\r\n", "\n", StringComparison.Ordinal)); + } + catch (Exception ex) when (ex is IOException or UnauthorizedAccessException) + { + return false; + } + } + + /// + /// The config as written: today's defaults, one // comment per choice (the loader skips comments), the + /// placeholder when is , and identity.agent only when + /// names one. + /// + internal static string Render(string root, IReadOnlyList? verifier, string? agent) + { + var name = new string([.. Path.GetFileName(Path.TrimEndingDirectorySeparator(root)).Where(c => !char.IsControl(c))]).Trim(); + if (name.Length == 0) + { + name = "project"; + } + else if (name.Length > 64) + { + name = name[..64]; + } + + static string Json(object value) => JsonSerializer.Serialize(value, Encoded); + + var b = new StringBuilder(); + b.Append("// aexp: the scope the runs are kept under and the one command aexp may run. Written by 'aexp init'; edit it freely.\n"); + b.Append("// Every edit revokes 'aexp trust' until a person approves the file again. Reference: docs/guide/cli-and-mcp.md.\n"); + b.Append("{\n"); + b.Append(" // The store, open runs, sessions and the hook log stay in .aexp/, which .gitignore excludes. Or \"postgres:\".\n"); + b.Append(" \"store\": ").Append(Json(AexpConfigLoader.DefaultStore)).Append(",\n"); + b.Append(" // The scope every run and lesson is kept under. \"agent\" (optional) names the agent the runs belong to.\n"); + b.Append(" \"identity\": { \"tenant\": \"local\", \"application\": ").Append(Json(name)).Append(", \"project\": ").Append(Json(name)); + if (agent is not null) + { + b.Append(", \"agent\": ").Append(Json(agent)); + } + + b.Append(" },\n"); + b.Append(" \"verify\": {\n"); + b.Append(verifier is null + ? " // No test command was obvious, so this placeholder refuses to run: set it to the project's test command.\n" + : " // Detected from the project's files: check it.\n"); + b.Append(" // The only command aexp runs: an argument vector, never a shell line. Exit code 0 is a pass.\n"); + b.Append(" \"command\": ").Append(Json(verifier ?? [VerifierRunner.Placeholder])).Append(",\n"); + b.Append(" // Below the Claude Code Stop hook's 600 s, so the hook can still end and finalize around it.\n"); + b.Append(" \"timeoutSeconds\": 540,\n"); + b.Append(" // The check each lesson names as what verified it.\n"); + b.Append(" \"checkId\": \"tests\"\n"); + b.Append(" },\n"); + b.Append(" // Tool argument names capture keeps. Everything else is dropped; secret-looking names and values are redacted anyway.\n"); + b.Append(" \"sanitization\": { \"allowArguments\": [\"file_path\"] },\n"); + b.Append(" // One run in five is given no lessons, so 'aexp benefit' can measure whether they help.\n"); + b.Append(" \"injection\": { \"holdoutRate\": 0.2 },\n"); + b.Append(" // After a tool call fails, the agent may ask what worked after the same failure before.\n"); + b.Append(" \"recall\": { \"enabled\": true }\n"); + b.Append("}\n"); + return b.ToString(); + } + + private static readonly JsonSerializerOptions Encoded = new() + { + // Read by people, not embedded in HTML: '<', '>', '&' and non-ASCII stay as written. + Encoder = System.Text.Encodings.Web.JavaScriptEncoder.UnsafeRelaxedJsonEscaping, + }; + + /// Writes the config whole or not at all: .aexp/ owner-only (0700), the file owner read and write. + internal static void WriteConfig(string configPath, string json, bool overwrite) + { + var directory = Path.GetDirectoryName(configPath)!; + if (OperatingSystem.IsWindows()) + { + Directory.CreateDirectory(directory); + } + else + { + const UnixFileMode OwnerOnly = UnixFileMode.UserRead | UnixFileMode.UserWrite | UnixFileMode.UserExecute; + Directory.CreateDirectory(directory, OwnerOnly); + // An .aexp/ that was there already (made by hand, say) is tightened to the same mode. + if (File.GetUnixFileMode(directory) != OwnerOnly) + { + File.SetUnixFileMode(directory, OwnerOnly); + } + } + + AtomicFile.Write(configPath, Encoding.UTF8.GetBytes(json), overwrite); + } + + /// + /// Adds .aexp/ to the root's .gitignore unless it is ignored already, and returns the one line to print. + /// With , a root with no .gitignore of its own and no .git (directory or + /// file) in it or above it, up to the home directory, gets no file. + /// + internal static async Task EnsureGitignoreAsync(string root, bool print, bool onlyUnderGit, Func environment, CancellationToken cancellationToken) + { + var gitignore = Path.Combine(root, ".gitignore"); + if (onlyUnderGit && !File.Exists(gitignore) && !UnderGit(root, environment)) + { + return ".gitignore: not under git here (no .git or .gitignore), so none was created; keep .aexp/ out of version control yourself."; + } + + var ignoreText = File.Exists(gitignore) ? await File.ReadAllTextAsync(gitignore, cancellationToken).ConfigureAwait(false) : string.Empty; + if (Ignores(ignoreText)) + { + return ".gitignore: already ignores .aexp/."; + } + + if (print) + { + return ".gitignore would get the line: " + GitignoreEntry; + } + + var newline = ignoreText.Contains("\r\n", StringComparison.Ordinal) ? "\r\n" : "\n"; + var prefix = ignoreText.Length == 0 || ignoreText.EndsWith('\n') ? string.Empty : newline; + await File.AppendAllTextAsync(gitignore, prefix + GitignoreComment + newline + GitignoreEntry + newline, cancellationToken).ConfigureAwait(false); + return ".gitignore: added .aexp/."; + } + + /// Whether or a directory above it holds .git, looking no further than the home directory (as the config lookup does). + private static bool UnderGit(string root, Func environment) + { + var home = environment(OperatingSystem.IsWindows() ? "USERPROFILE" : "HOME"); + home = string.IsNullOrWhiteSpace(home) ? null : Path.TrimEndingDirectorySeparator(Path.GetFullPath(home)); + for (var directory = new DirectoryInfo(Path.GetFullPath(root)); directory is not null; directory = directory.Parent) + { + var here = Path.TrimEndingDirectorySeparator(directory.FullName); + var git = Path.Combine(here, ".git"); + if (Directory.Exists(git) || File.Exists(git)) + { + return true; + } + + if (home is not null && string.Equals(here, home, OperatingSystem.IsLinux() ? StringComparison.Ordinal : StringComparison.OrdinalIgnoreCase)) + { + return false; + } + } + + return false; + } + + internal static bool Ignores(string gitignore) => + gitignore.Replace("\r\n", "\n", StringComparison.Ordinal).Split('\n').Select(line => line.Trim()) + .Any(line => line is ".aexp" or ".aexp/" or "/.aexp" or "/.aexp/" or ".aexp/*" or "/.aexp/*" or ".aexp/**" or "/.aexp/**"); +} diff --git a/src/AgentExperience.Cli/Engine/VerifierRunner.cs b/src/AgentExperience.Cli/Engine/VerifierRunner.cs index faf229d..77b8a97 100644 --- a/src/AgentExperience.Cli/Engine/VerifierRunner.cs +++ b/src/AgentExperience.Cli/Engine/VerifierRunner.cs @@ -18,8 +18,8 @@ internal sealed record VerifierResult(int ExitCode, bool TimedOut); internal static class VerifierRunner { /// - /// What aexp setup writes as the verifier when it cannot tell the project's test command: never run, so the - /// verifier stays refused until a person sets it. + /// What aexp init and aexp setup write as the verifier when they cannot tell the project's test command: + /// never run, so the verifier stays refused until a person sets it. /// public const string Placeholder = ""; @@ -30,7 +30,7 @@ public static string ResolveProgram(VerifierSetting verifier, string projectRoot if (program.StartsWith('<') && program.EndsWith('>')) { throw new AexpUsageException( - "verify.command is still the placeholder 'aexp setup' wrote: set it to this project's test command, then a person " + "verify.command is still the placeholder 'aexp init' (or 'aexp setup') wrote: set it to this project's test command, then a person " + "reviews the config and runs 'aexp trust'."); } diff --git a/src/AgentExperience.Cli/Program.cs b/src/AgentExperience.Cli/Program.cs index 830fbb8..617b0a1 100644 --- a/src/AgentExperience.Cli/Program.cs +++ b/src/AgentExperience.Cli/Program.cs @@ -9,5 +9,12 @@ return await AexpCli.RunAsync( args, - new AexpConsole(Console.Out, Console.Error, Environment.CurrentDirectory, Environment.GetEnvironmentVariable, TimeProvider.System), + new AexpConsole( + Console.Out, + Console.Error, + Environment.CurrentDirectory, + Environment.GetEnvironmentVariable, + TimeProvider.System, + // A question is asked only of a person at a terminal, never of a pipe. + ReadLine: Console.IsInputRedirected ? null : Console.ReadLine), cancellation.Token); diff --git a/src/AgentExperience.Cli/README.md b/src/AgentExperience.Cli/README.md index b5e0192..dba2876 100644 --- a/src/AgentExperience.Cli/README.md +++ b/src/AgentExperience.Cli/README.md @@ -14,12 +14,20 @@ lessons, the failure hints and the measurement are the library's own. dotnet tool install --global AgentExperience.Cli --prerelease ``` -Then put a `.aexp/config.json` in your project: +Then, in your project root: + +```bash +aexp init # writes a commented .aexp/config.json and adds .aexp/ to .gitignore; approves nothing +``` + +It detects the verifier when one kind of project is at the root (`dotnet test`, `npm test`, `pytest`, `go test ./...`; +`make test` only when none of those fits), asks when none or several fit (or pass `--verify '["npm","test"]'`, a JSON array, never a shell line), +and otherwise writes a placeholder that refuses to run until you set it. The config it writes is, in short: ```json { - "identity": { "tenant": "local", "application": "my-service", "project": "my-repo" }, - "verify": { "command": ["dotnet", "test"], "timeoutSeconds": 900, "checkId": "tests" } + "identity": { "tenant": "local", "application": "my-service", "project": "my-service" }, + "verify": { "command": ["dotnet", "test"], "timeoutSeconds": 540, "checkId": "tests" } } ``` diff --git a/tests/AgentExperience.Cli.Tests/ClaudeCodeSetupTests.cs b/tests/AgentExperience.Cli.Tests/ClaudeCodeSetupTests.cs index a00750a..6844275 100644 --- a/tests/AgentExperience.Cli.Tests/ClaudeCodeSetupTests.cs +++ b/tests/AgentExperience.Cli.Tests/ClaudeCodeSetupTests.cs @@ -1,4 +1,5 @@ using System.Text; +using System.Text.Json; using System.Text.Json.Nodes; using AgentExperience.Abstractions; using AgentExperience.Cli.ClaudeCode; @@ -87,9 +88,11 @@ public async Task Setup_merges_the_hooks_into_existing_settings_without_losing_a Assert.Contains("/path/to/lint-check.sh", Commands(settings, "PostToolUse")); Assert.Contains("/path/to/notify.sh", Commands(settings, "Stop")); - // The config: the detected test command, never trusted by setup. - var config = JsonNode.Parse(File.ReadAllText(project.Config))!.AsObject(); + // The config: commented (the loader skips comments), the detected test command, the agent named, never trusted by setup. + var config = JsonNode.Parse(File.ReadAllText(project.Config), documentOptions: new JsonDocumentOptions { CommentHandling = JsonCommentHandling.Skip })!.AsObject(); Assert.Equal("""["dotnet","test"]""", config["verify"]!["command"]!.ToJsonString()); + Assert.Equal("claude-code", (string?)config["identity"]!["agent"]); + Assert.Contains("// ", File.ReadAllText(project.Config)); Assert.Equal("bin/\nobj/\n# aexp: the local lesson store, open runs and the config\n.aexp/\n", File.ReadAllText(Path.Combine(project.Root, ".gitignore"))); Assert.Contains("aexp trust", first.Output); Assert.Contains("Edit(//**/.aexp/**)", first.Output); diff --git a/tests/AgentExperience.Cli.Tests/InitTests.cs b/tests/AgentExperience.Cli.Tests/InitTests.cs new file mode 100644 index 0000000..bb0cf13 --- /dev/null +++ b/tests/AgentExperience.Cli.Tests/InitTests.cs @@ -0,0 +1,401 @@ +using System.Text.Json; +using System.Text.Json.Nodes; +using AgentExperience.Cli.Configuration; +using AgentExperience.Cli.Engine; + +namespace AgentExperience.Cli.Tests; + +/// Story 29.1: aexp init, one command that writes the config, and the one generator it shares with setup. +public sealed class InitTests +{ + /// + /// A project directory (a repository root unless git is false) with its own user config directory. Lines in + /// are what a person at the terminal types; means standard input is a pipe. + /// + private sealed class Scratch : IDisposable + { + public Scratch(bool git = true) + { + Directory.CreateDirectory(Root); + if (git) + { + Directory.CreateDirectory(Path.Combine(Root, ".git")); + } + + Directory.CreateDirectory(UserConfig); + } + + public string Root { get; } = Path.Combine(Path.GetTempPath(), $"aexp-init-{Guid.NewGuid():N}"); + + public string UserConfig { get; } = Path.Combine(Path.GetTempPath(), $"aexp-user-{Guid.NewGuid():N}"); + + public string Config => Path.Combine(Root, ".aexp", "config.json"); + + public string Gitignore => Path.Combine(Root, ".gitignore"); + + public Queue? Answers { get; set; } + + public int Asked { get; private set; } + + public void Touch(string name, string content = "") => File.WriteAllText(Path.Combine(Root, name), content); + + public JsonObject ReadConfig() => + JsonNode.Parse(File.ReadAllText(Config), documentOptions: new JsonDocumentOptions { CommentHandling = JsonCommentHandling.Skip })!.AsObject(); + + public string Verifier() => ReadConfig()["verify"]!["command"]!.ToJsonString(); + + public Task RunAsync(params string[] args) => RunInAsync(Root, args); + + /// Runs the tool in (a subdirectory of the root, say). + public async Task RunInAsync(string workingDirectory, params string[] args) + { + var output = new StringWriter(); + var error = new StringWriter(); + var exit = await AexpCli.RunAsync( + args, + new AexpConsole( + output, + error, + workingDirectory, + name => name switch + { + "XDG_CONFIG_HOME" => UserConfig, + "CLAUDE_PROJECT_DIR" => null, + _ => Environment.GetEnvironmentVariable(name), + }, + TimeProvider.System, + ReadLine: Answers is null ? null : () => + { + Asked++; + return Answers.Count > 0 ? Answers.Dequeue() : null; + })); + return new CliResult(exit, output.ToString(), error.ToString()); + } + + public async Task OkAsync(params string[] args) + { + var result = await RunAsync(args); + Assert.True(result.Exit == 0, $"aexp {string.Join(' ', args)} exited {result.Exit}: {result.Error}"); + return result.Output; + } + + public void Dispose() + { + try + { + Microsoft.Data.Sqlite.SqliteConnection.ClearAllPools(); + Directory.Delete(Root, recursive: true); + Directory.Delete(UserConfig, recursive: true); + } + catch (IOException) + { + } + } + } + + [Fact] + public async Task One_kind_detected_writes_that_verifier_the_gitignore_entry_and_the_next_steps() + { + using var project = new Scratch(); + project.Touch("go.mod", "module example.com/x\n"); + + var output = await project.OkAsync("init"); + + Assert.Equal("""["go","test","./..."]""", project.Verifier()); + Assert.Contains("""["go","test","./..."]""", output); + Assert.Contains("go.mod", output); + Assert.Contains("\n.aexp/\n", File.ReadAllText(project.Gitignore)); + Assert.Contains("aexp trust", output); + Assert.Contains("aexp setup claude-code", output); + Assert.Contains("aexp mcp", output); + Assert.Contains("approved nothing", output); + Assert.Equal(0, project.Asked); + } + + [Fact] + public async Task Acceptance_a_csproj_under_git_gives_dotnet_test_which_trust_then_prints() + { + using var project = new Scratch(); + project.Touch("Payments.csproj", ""); + + var init = await project.OkAsync("init"); + Assert.Contains("aexp trust", init); + Assert.Contains("aexp setup claude-code", init); + Assert.Contains("approved nothing", init); + Assert.Contains(".aexp/", File.ReadAllText(project.Gitignore)); + + // The written config loads, and 'aexp trust' (a person's step, run here as the test's person) shows the chosen verifier. + var trust = await project.OkAsync("trust"); + Assert.Contains("""Verifier: ["dotnet","test"]""", trust); + Assert.Contains("Trusted:", trust); + Assert.DoesNotContain("agent", project.ReadConfig()["identity"]!.AsObject().Select(p => p.Key)); // init ties the config to no agent + + if (!OperatingSystem.IsWindows()) + { + Assert.Equal(UnixFileMode.UserRead | UnixFileMode.UserWrite | UnixFileMode.UserExecute, File.GetUnixFileMode(Path.GetDirectoryName(project.Config)!)); + Assert.Equal(UnixFileMode.UserRead | UnixFileMode.UserWrite, File.GetUnixFileMode(project.Config)); + + // An .aexp/ made by hand with a looser mode is tightened when the config is written into it. + using var loose = new Scratch(); + var directory = Path.Combine(loose.Root, ".aexp"); + Directory.CreateDirectory(directory, UnixFileMode.UserRead | UnixFileMode.UserWrite | UnixFileMode.UserExecute | UnixFileMode.GroupRead | UnixFileMode.GroupExecute | UnixFileMode.OtherRead | UnixFileMode.OtherExecute); + await loose.OkAsync("init"); + Assert.Equal(UnixFileMode.UserRead | UnixFileMode.UserWrite | UnixFileMode.UserExecute, File.GetUnixFileMode(directory)); + } + } + + [Fact] + public void A_makefile_with_a_test_target_is_make_test_only_when_nothing_else_fits() + { + using var project = new Scratch(); + project.Touch("Makefile", "build:\n\tgo build\n\ntest := nope\ntest ::= nope\n"); + Assert.Empty(ProjectInit.Detect(project.Root)); + + project.Touch("Makefile", "build:\n\tgo build\n\ntest: build\n\tgo test ./...\n"); + var only = Assert.Single(ProjectInit.Detect(project.Root)); + Assert.Equal(["make", "test"], only.Command); + Assert.Contains("Makefile", only.Evidence); + + // A Makefile beside go.mod wraps go: go test is the one candidate, as it was before make was detected. + project.Touch("go.mod", "module example.com/x\n"); + Assert.Equal(["go", "test", "./..."], Assert.Single(ProjectInit.Detect(project.Root)).Command); + } + + [Fact] + public async Task Init_takes_no_config_option() + { + using var project = new Scratch(); + var result = await project.RunAsync("init", "--config", "elsewhere.json"); + Assert.Equal(AexpCli.Usage, result.Exit); + Assert.Contains("--config", result.Error); + Assert.False(File.Exists(project.Config)); + } + + [Fact] + public async Task Several_detected_and_no_terminal_writes_the_placeholder_and_lists_the_candidates() + { + using var project = new Scratch(); + project.Touch("package.json", """{ "scripts": { "test": "vitest run" } }"""); + project.Touch("pyproject.toml", "[tool.pytest.ini_options]\n"); + + var output = await project.OkAsync("init"); + + Assert.Equal(JsonSerializer.Serialize(new[] { VerifierRunner.Placeholder }), project.Verifier()); + Assert.Contains("""1. ["npm","test"]""", output); + Assert.Contains("""2. ["pytest"]""", output); + Assert.Contains("verify.command", output); + Assert.Equal(0, project.Asked); + } + + [Fact] + public async Task None_detected_and_no_terminal_writes_the_placeholder_and_says_what_it_looked_for() + { + using var project = new Scratch(); + var output = await project.OkAsync("init"); + Assert.Equal(JsonSerializer.Serialize(new[] { VerifierRunner.Placeholder }), project.Verifier()); + Assert.Contains("No test command was detected", output); + Assert.Contains("verify.command", output); + } + + [Theory] + [InlineData("2", """["pytest"]""", 1)] + [InlineData(" 1 ", """["npm","test"]""", 1)] + [InlineData("""["make", "check"]""", """["make","check"]""", 1)] + [InlineData("", "placeholder", 1)] + public async Task At_a_terminal_the_answer_is_a_number_a_json_array_or_empty_for_the_placeholder(string answer, string expected, int asked) + { + using var project = new Scratch(); + project.Touch("package.json", """{ "scripts": { "test": "vitest run" } }"""); + project.Touch("pyproject.toml", "[tool.pytest.ini_options]\n"); + project.Answers = new Queue([answer]); + + var output = await project.OkAsync("init"); + + Assert.Equal(expected == "placeholder" ? JsonSerializer.Serialize(new[] { VerifierRunner.Placeholder }) : expected, project.Verifier()); + Assert.Equal(asked, project.Asked); + Assert.Contains("Verifier:", output); + } + + [Fact] + public async Task A_bad_answer_is_asked_again_once_then_the_placeholder() + { + using var project = new Scratch(); + project.Touch("package.json", """{ "scripts": { "test": "vitest run" } }"""); + project.Touch("pyproject.toml", "[tool.pytest.ini_options]\n"); + + project.Answers = new Queue(["7", "npm test", "2"]); + await project.OkAsync("init"); + Assert.Equal(JsonSerializer.Serialize(new[] { VerifierRunner.Placeholder }), project.Verifier()); + Assert.Equal(2, project.Asked); // the third answer is never read + + project.Answers = new Queue(["""["x", ""]""", "1"]); + await project.OkAsync("init", "--force"); + Assert.Equal("""["npm","test"]""", project.Verifier()); + + // End of input at the prompt is an empty answer. + project.Answers = new Queue(); + await project.OkAsync("init", "--force"); + Assert.Equal(JsonSerializer.Serialize(new[] { VerifierRunner.Placeholder }), project.Verifier()); + } + + [Fact] + public async Task Verify_given_is_used_as_is_with_no_detection_and_no_prompt() + { + using var project = new Scratch(); + project.Touch("go.mod", "module example.com/x\n"); + project.Touch("package.json", """{ "scripts": { "test": "vitest run" } }"""); + project.Answers = new Queue(["1"]); + + var output = await project.OkAsync("init", "--verify", """["npm", "test", "--", "--ci"]"""); + + Assert.Equal("""["npm","test","--","--ci"]""", project.Verifier()); + Assert.Equal(0, project.Asked); + Assert.DoesNotContain("Detected", output); + Assert.DoesNotContain("go.mod", output); + } + + [Theory] + [InlineData("npm test")] + [InlineData("\"npm test\"")] + [InlineData("[]")] + [InlineData("""["npm", ""]""")] + [InlineData("""["npm", 1]""")] + [InlineData("""{"command": ["npm"]}""")] + public async Task Verify_that_is_not_a_json_array_of_non_empty_strings_is_a_usage_error(string verify) + { + using var project = new Scratch(); + var result = await project.RunAsync("init", "--verify", verify); + Assert.Equal(AexpCli.Usage, result.Exit); + Assert.Contains("JSON array", result.Error); + Assert.False(File.Exists(project.Config)); + } + + [Fact] + public async Task An_existing_config_is_kept_and_force_replaces_it_atomically_noting_that_trust_is_revoked() + { + using var project = new Scratch(); + project.Touch("go.mod", "module example.com/x\n"); + Directory.CreateDirectory(Path.GetDirectoryName(project.Config)!); + File.WriteAllText(project.Config, "{ \"mine\": true }"); + + var kept = await project.RunAsync("init"); + Assert.Equal(AexpCli.Usage, kept.Exit); + Assert.Contains("--force", kept.Error); + Assert.Equal("{ \"mine\": true }", File.ReadAllText(project.Config)); + + var replaced = await project.OkAsync("init", "--force"); + Assert.Contains("replaced", replaced); + Assert.Contains("aexp trust", replaced); + Assert.Contains("no longer applies", replaced); + Assert.Equal("""["go","test","./..."]""", project.Verifier()); + Assert.Empty(Directory.EnumerateFiles(Path.GetDirectoryName(project.Config)!, "*.tmp")); + } + + [Fact] + public async Task Print_shows_the_config_and_the_gitignore_line_and_writes_nothing() + { + using var project = new Scratch(); + project.Touch("go.mod", "module example.com/x\n"); + + var output = await project.OkAsync("init", "--print"); + + Assert.Contains("""["go","test","./..."]""", output); + Assert.Contains("\"holdoutRate\": 0.2", output); + Assert.Contains("// ", output); + Assert.Contains(".gitignore would get the line: .aexp/", output); + Assert.False(Directory.Exists(Path.GetDirectoryName(project.Config)!)); + Assert.False(File.Exists(project.Gitignore)); + + // With a config in place, --print still prints and writes nothing. + await project.OkAsync("init"); + var bytes = File.ReadAllBytes(project.Config); + var again = await project.OkAsync("init", "--print"); + Assert.Contains("exists", again); + Assert.Equal(bytes, File.ReadAllBytes(project.Config)); + } + + [Fact] + public async Task Not_under_git_writes_the_config_and_no_gitignore_and_says_so() + { + using var project = new Scratch(git: false); + project.Touch("go.mod", "module example.com/x\n"); + + var output = await project.OkAsync("init"); + + Assert.True(File.Exists(project.Config)); + Assert.False(File.Exists(project.Gitignore)); + Assert.Contains("not under git", output); + + // A subdirectory of a repository is under git: the .gitignore is created in the init directory itself. + using var repo = new Scratch(); + var sub = Path.Combine(repo.Root, "services", "payments"); + Directory.CreateDirectory(sub); + File.WriteAllText(Path.Combine(sub, "go.mod"), "module example.com/x\n"); + var inSub = await repo.RunInAsync(sub, "init"); + Assert.True(inSub.Exit == 0, inSub.Error); + Assert.Contains(".gitignore: added .aexp/.", inSub.Output); + Assert.Equal("# aexp: the local lesson store, open runs and the config\n.aexp/\n", File.ReadAllText(Path.Combine(sub, ".gitignore"))); + Assert.True(File.Exists(Path.Combine(sub, ".aexp", "config.json"))); + Assert.False(File.Exists(repo.Gitignore)); + + // An existing .gitignore without .git is enough to add the line to, as is a .git file (a worktree). + using var withIgnore = new Scratch(git: false); + withIgnore.Touch(".gitignore", "bin/"); + withIgnore.Touch("go.mod", "module example.com/x\n"); + await withIgnore.OkAsync("init"); + Assert.Equal("bin/\n# aexp: the local lesson store, open runs and the config\n.aexp/\n", File.ReadAllText(withIgnore.Gitignore)); + + using var worktree = new Scratch(git: false); + worktree.Touch(".git", "gitdir: /elsewhere\n"); + await worktree.OkAsync("init"); + Assert.Contains(".aexp/", File.ReadAllText(worktree.Gitignore)); + Assert.Contains("already ignores", await worktree.OkAsync("init", "--force")); + } + + [Fact] + public async Task Setup_after_init_keeps_the_config_byte_for_byte() + { + using var project = new Scratch(); + project.Touch("Payments.csproj", ""); + await project.OkAsync("init"); + var bytes = File.ReadAllBytes(project.Config); + + var setup = await project.OkAsync("setup", "claude-code"); + + Assert.Contains(".aexp/config.json: exists; left as it is.", setup); + Assert.Equal(bytes, File.ReadAllBytes(project.Config)); + Assert.Contains("already ignores", setup); + Assert.Equal(1, File.ReadAllText(project.Gitignore).Split('\n').Count(line => line == ".aexp/")); + } + + [Theory] + [InlineData(null, null)] + [InlineData(null, "claude-code")] + [InlineData("""["dotnet","test"]""", "claude-code")] + [InlineData("""["npm","test","--","--ci"]""", null)] + [InlineData("""["","&","olá"]""", null)] + public void Every_rendered_config_loads_with_todays_defaults_and_no_unknown_key(string? verifier, string? agent) + { + using var project = new Scratch(); + var command = verifier is null ? null : JsonSerializer.Deserialize(verifier); + var json = ProjectInit.Render(project.Root, command, agent); + ProjectInit.WriteConfig(project.Config, json, overwrite: false); + + var settings = AexpConfigLoader.Load(null, project.Root, Environment.GetEnvironmentVariable); + + Assert.Equal(verifier is null ? [VerifierRunner.Placeholder] : command!, settings.Verifier.Command); + Assert.Equal(TimeSpan.FromSeconds(540), settings.Verifier.Timeout); + Assert.Equal("tests", settings.Verifier.CheckId); + Assert.Equal(0.2, settings.HoldoutRate); + Assert.Equal(["file_path"], settings.AllowArguments); + Assert.True(settings.RecallEnabled); + Assert.False(settings.Verifier.AcceptExitCode); + Assert.Equal(Path.Combine(Path.GetFullPath(project.Root), ".aexp", "experience.db"), Assert.IsType(settings.Store).Path); + Assert.Equal("local", settings.Scope.TenantId); + Assert.Equal(Path.GetFileName(project.Root), settings.Scope.ApplicationId); + Assert.Equal(Path.GetFileName(project.Root), settings.Scope.ProjectId); + Assert.Equal(agent, settings.Scope.AgentId); + Assert.Equal(agent is null ? "aexp:local" : "aexp:" + agent, settings.Authorization(DateTimeOffset.UnixEpoch).PrincipalId); + Assert.DoesNotContain("\\u", json); // non-ASCII and '<', '>', '&' as written, for a person reading it + } +}