diff --git a/README.md b/README.md index 87c7c4e..0847bcb 100644 --- a/README.md +++ b/README.md @@ -26,8 +26,8 @@ verdict when it can't prove one (see [docs/postgres-online-ddl-reference.md](docs/postgres-online-ddl-reference.md)). **Status: Phases 1 and 2.1–2.5.** The parse boundary, declarative diff, -classifier, router seam, versioned dry-run plan report, and offline linter -are implemented. `pg-sprite migrate --alter '…'` runs a bounded optimistic +classifier, router seam, versioned dry-run plan report, offline linter, and +advisory `suggest` command are implemented. `pg-sprite migrate --alter '…'` runs a bounded optimistic native attempt; routed execution beyond that attempt lands in Phase 3. Changes without an available backend get a structured refusal (exit code 2). The design docs and the phased diff --git a/SAFETY.md b/SAFETY.md index 8cb3556..1e0d8e0 100644 --- a/SAFETY.md +++ b/SAFETY.md @@ -27,10 +27,10 @@ The invariant registry (invariant IDs referenced below) lives in | `pkg/checkpoint` — durable resume state | ✅ core | planned (Phase 8) | ST-1, ST-2 | | slot lifecycle (in `pkg/decode`) — create, reap, lag ceiling | ✅ core | planned (Phase 8) | ST-3 | | `pkg/schemachange` — orchestrator, **cutover swap + fidelity gate** | ✅ core | planned (Phase 7) | LK-2, LK-4, ST-5 | -| `pkg/statement`, `pkg/planner`, `pkg/schemadiff`, `pkg/router`, `pkg/plan`, `pkg/lint` — classify/diff/route/report | ❌ periphery¹ | `pkg/statement` (parse boundary), `pkg/schemadiff` (introspect/diff via scratch execute-and-introspect), `pkg/planner` (classifier), `pkg/router` (backend assignment + availability policy), `pkg/plan` (versioned dry-run plan report), and `pkg/lint` (offline typed findings) exist (Phases 2.1–2.5) | (CO-7 holds at the parse boundary) | +| `pkg/statement`, `pkg/planner`, `pkg/schemadiff`, `pkg/router`, `pkg/plan`, `pkg/lint`, `pkg/suggest` — classify/diff/route/report | ❌ periphery¹ | `pkg/statement` (parse boundary), `pkg/schemadiff` (introspect/diff via scratch execute-and-introspect), `pkg/planner` (classifier), `pkg/router` (backend assignment + availability policy), `pkg/plan` (versioned dry-run plan report), `pkg/lint` (offline typed findings), and `pkg/suggest` (advisory rewrites with typed caveats) exist (Phases 2.1–2.5) | (CO-7 holds at the parse boundary) | | `pkg/verdict` — structured outcome contract, rendering, exit codes | ❌ periphery | exists (Phase 1) | — | | `pkg/diffplan` — desired schema → routed convergence plan, the declarative front door as a library (the CLI `diff` and embedding orchestrators share it) | ❌ periphery | exists | — | -| `internal/cli` — CLI, flags, help, prompts | ❌ periphery | `migrate`, `status`, `diff`, `fmt`, and `lint` exist | — | +| `internal/cli` — CLI, flags, help, prompts | ❌ periphery | `migrate`, `status`, `diff`, `fmt`, `lint`, and `suggest` exist | — | | status / progress / advisory rendering, metrics | ❌ periphery | planned | — | | orchestrator adapter | ❌ periphery | planned (Phase 11) | OC-* hold *at* the boundary | | `internal/testutil` | ❌ test-only | exists | — | diff --git a/docs/README.md b/docs/README.md index 83adbc1..5384f3d 100644 --- a/docs/README.md +++ b/docs/README.md @@ -34,6 +34,7 @@ combination is the product is [vision.md](vision.md); start there. | [tcb-model.md](tcb-model.md) | The **TCB model** — the trusted-computing-base partition of the engine: which components are the small trusted core that enforces the invariant registry vs the untrusted periphery, the never-trust-callers rule, domain types that make illegal states unrepresentable, the in-TCB engineering rules (from TigerBeetle TIGER_STYLE, s2n-tls, qmail, bitcoin-core), the verification ladder, and the per-side AI-assisted development policy. | | [plan-report.md](plan-report.md) | The **plan report contract** — the versioned JSON shape both front doors emit for dry-run plans: fields, closed vocabularies, the fingerprint identity, required consumer behavior for unknown versions/values, and one generated example per source (pinned by test). | | [lint-report.md](lint-report.md) | The **lint report contract** — the versioned JSON shape `pg-sprite lint` emits for offline CI gating: finding fields (verbatim SQL, line/column), the codes table, severities and exit behavior, the offline-conservatism rules, and how the contract versions relative to the plan report. | +| [suggest-report.md](suggest-report.md) | The **suggest report contract** — the versioned JSON shape `pg-sprite suggest` emits for offline advice: the typed caveat vocabulary (what changes about how you must run a safer form, and what a failed step leaves behind), the typed guidance codes for rewrites the planner cannot construct, and the operation → safer form → caveats table (pinned by test). | | [testing.md](testing.md) | The **test-suite guide** — how to run the suite (unit, per-major, all supported majors, compose database), current coverage, the remaining executor-phase test obligations, and the vanilla-PostgreSQL-matrix vs real-Aurora validation boundary. | | [schemabot-integration.md](schemabot-integration.md) | The **single home for orchestrator integration** — how SchemaBot (the reference orchestrator) drives the engine: the pluggable-engine overview, the verb mappings, the concrete adapter contract, and the design constraints (OC-* invariants) the integration imposes on the core. | diff --git a/docs/lint-report.md b/docs/lint-report.md index be62a70..63ab874 100644 --- a/docs/lint-report.md +++ b/docs/lint-report.md @@ -67,7 +67,7 @@ codes make the conservatism visible instead of burying it: | `code` | string | always | The typed finding kind (see Codes). Automation branches on this, never on prose. | | `severity` | string | always | What the engine would do about it (see Severities). | | `reason` | string | classifier findings | The planner's typed cause, drawn from the plan report's Reasons vocabulary. Absent for destructive findings, which are a property of the operation, not a routing decision. | -| `suggestion` | array | when constructible | The ordered safer SQL, present only for `blocking-idiom` findings where the planner constructed the rewrite. A safer form of the submitted statement, not a semantic equivalent — running it by hand forgoes the engine's execution-time guards (invalid-index detection after a concurrent build). Its absence still means the submitted form blocks — the planner does not construct rewrites for multi-operation statements or for operations that need catalog knowledge (ATTACH PARTITION's proving CHECK). | +| `suggestion` | array | when constructible | The ordered safer SQL, present only for `blocking-idiom` findings where the planner constructed the rewrite. A safer form of the submitted statement, not a semantic equivalent — running it by hand forgoes the engine's execution-time guards (invalid-index detection after a concurrent build). Its absence still means the submitted form blocks — the planner does not construct rewrites for multi-operation statements or for operations that need catalog knowledge (ATTACH PARTITION's proving CHECK); the [suggest report](suggest-report.md) emits typed guidance naming the manual path for exactly those cases. | | `suggestion_execution` | string | with `suggestion` | The typed execution contract for `suggestion`, drawn from the plan report's Execution contracts vocabulary (`autocommit-each-step`: each step in its own implicit transaction, never inside an enclosing transaction block; a failed step leaves partial state the runner must detect and recover). A consumer that runs the suggestion branches on this. Present exactly when `suggestion` is. | ## Codes (`code`) diff --git a/docs/low-level-design.md b/docs/low-level-design.md index 0350c9c..07ebefa 100644 --- a/docs/low-level-design.md +++ b/docs/low-level-design.md @@ -303,7 +303,9 @@ request, not a permission. The [advisory behaviour](high-level-design.md#advisory-mode-suggest-the-safe-rewrite-dont-silently-run-the-risky-one) is a property of the **planner's classifier output**, not a separate code path. Every current `planner.Decision` carries the operation, route, typed reason, and safer SQL where applicable. -The CLI renders that output in `diff` and `migrate --dry-run`. +The CLI renders that output in `diff` and `migrate --dry-run`, and the offline `suggest` +command (`pkg/suggest`) emits it as a standalone advisory report — original → recommended +with typed reason and caveat metadata, never executing anything. ### What the classifier emits per operation diff --git a/docs/suggest-report.md b/docs/suggest-report.md new file mode 100644 index 0000000..b5eb91d --- /dev/null +++ b/docs/suggest-report.md @@ -0,0 +1,157 @@ +# The suggest report contract + +The suggest report is the machine-readable result of `pg-sprite suggest` — the offline +advisory surface that maps DDL that is risky as written to the safer native form the engine +would run instead. It runs the same parse-and-classify pipeline as the front doors, with +zero live facts and no database, and it never gates: `suggest` always exits zero on a valid +script; `pg-sprite lint` owns the gate. This document is the contract: the fields, the +closed vocabularies, and the behavior required of a consumer. The Go source of truth is +`pkg/suggest`; tests in `pkg/suggest` and `internal/cli` pin everything documented here. + +## Versioning: `format_version` + +Every report carries `format_version`. A consumer that does not recognize the version must +**reject the report** — never guess at field semantics. The version covers the field shape +and the closed vocabularies below (caveats, guidance codes, and the embedded planner +reasons and execution contracts): adding a value to any of them is a contract change and +bumps `format_version`, even if no field is added or renamed. + +### Relationship to the plan and lint reports + +The suggest report, the [lint report](lint-report.md), and the +[plan report](plan-report.md) are **versioned independently** — each carries its own +`format_version`, and they move separately. They share vocabularies: a suggestion's +`reason` draws from the plan report's Reasons set and its `execution` from the plan +report's Execution contracts, and the suggest `format_version` pins the generation of +those sets a suggest consumer must understand, exactly as the plan and lint versions do +for their consumers. All three reports impose the same rule on unknown values, stated in +the next section. + +## Consumer behavior for unknown values + +Every enum field draws from a closed vocabulary listed here. A consumer that meets a value +it does not recognize must **fail closed** — treat the suggestion as unknown, refuse to run +its sequence, and surface it rather than ignore it. A caveat you cannot interpret may be the +one that says the sequence must not run inside your transaction. + +## Report fields + +| Field | Type | Presence | Meaning | +|---|---|---|---| +| `format_version` | int | always | Contract version; reject unknown versions. | +| `suggestions` | array | always | The advisory results in statement order, one per safer-idiom decision; `[]` (never `null`) means every statement is already in its safest known form or is outside the advisory surface (refusals, table rewrites, and destructive drops are lint findings). | + +## Suggestion fields + +Every suggestion carries **exactly one** of `recommended` and `guidance`: a constructed +rewrite, or the typed manual path when the planner cannot construct one. Silence is not an +outcome — every statement `lint` flags `blocking-idiom` appears here. + +| Field | Type | Presence | Meaning | +|---|---|---|---| +| `statement` | int | always | 1-based index of the statement in the script. | +| `line` | int | always | 1-based source line of the statement's first token, so a consumer can annotate the advice onto the file it came from. | +| `column` | int | always | 1-based source column of the statement's first token. | +| `original` | string | always | The statement's **verbatim source text** (without the trailing semicolon), so it can be found in the source by exact match. | +| `operation` | string | always | Operator-facing label of the risky operation. Display only — never branch on it. | +| `reason` | string | always | The classifier's typed cause, drawn from the plan report's Reasons vocabulary; always `safer-idiom` under this version. | +| `recommended` | array | when constructible | The ordered safer SQL to run instead. A safer form of the submitted statement, not a semantic equivalent — it converges on the same declared end state with different locking, transactionality, and failure modes, which `caveats` names. | +| `execution` | string | with `recommended` | The typed execution contract for `recommended`, drawn from the plan report's Execution contracts vocabulary (`autocommit-each-step`: each step in its own implicit transaction, never inside an enclosing transaction block; a failed step leaves partial state the runner must detect and recover). Present exactly when `recommended` is. | +| `caveats` | array | with `recommended` | The typed conditions under which the recommendation differs from the original (see Caveats). Never empty when present — a rewrite with no trade would be the same statement. | +| `guidance` | string | when not constructible | The typed manual path (see Guidance). The submitted form still blocks; this names what to do about it. | + +## Caveats (`caveats`) + +The caveats are **independent — no caveat implies another**; a sequence carries every +caveat that applies to it. (`non-transactional` does not imply `separate-transactions`: +the first says a step refuses a transaction block, the second says the steps must not +share one.) + +| Value | Meaning | +|---|---| +| `non-transactional` | The sequence contains a CONCURRENTLY statement, which cannot run inside a transaction block. | +| `separate-transactions` | The steps must commit separately — the weaker locks the sequence exists for are held to commit, so one enclosing transaction reproduces the blocking the rewrite avoids. | +| `invalid-index-on-failure` | A failed or cancelled concurrent build leaves an INVALID index that must be detected (`pg_index.indisvalid`) and dropped or rebuilt before retrying. | +| `detach-finalize-on-failure` | An interrupted concurrent detach leaves the partition half-detached; it must be finished with `DETACH PARTITION FINALIZE`. | +| `validation-scan` | The VALIDATE step still scans every row — the rewrite trades the lock strength, not the scan. | +| `scaffold-constraint-on-failure` | A failed VALIDATE leaves the NOT VALID constraint the sequence added on the live table, and replaying the sequence then fails at the ADD CONSTRAINT step (`duplicate_object`). The runner must detect the leftover constraint (`pg_constraint`) and resume from the VALIDATE step, or drop it and restart. | + +## Guidance (`guidance`) + +| Value | Emitted for | Manual path | +|---|---|---| +| `split-statement` | Any risky operation inside a multi-operation statement | Rewrites are constructed only for single-operation statements — a partial rewrite of a compound ALTER would be misleading. Split the statement into one operation per statement and advise again. | +| `add-column-then-constraint` | `ADD COLUMN` with an inline UNIQUE / PRIMARY KEY / FOREIGN KEY / CHECK | The inline constraint builds or validates under the ADD COLUMN's ACCESS EXCLUSIVE lock. Add the plain column first, then build the constraint with its online pattern. | +| `pre-add-validated-check` | `ATTACH PARTITION` | The attach scans the child under the parent's lock unless a validated CHECK matching the partition bound already exists on the child. Pre-add that CHECK (NOT VALID, then VALIDATE), attach, then drop it. The bound-matching CHECK cannot be constructed from the statement alone. | +| `not-null-scaffold` | `ADD CONSTRAINT … NOT NULL` | Prove the invariant with a NOT VALID CHECK (`col IS NOT NULL`) plus an online VALIDATE, then the NOT NULL constraint is a catalog flip — the same scaffold sequence the `SET NOT NULL` form gets constructed. | + +## The rewrite table: operation → safer form → caveats + +This is the specification of the advice — which safer native idiom each risky-as-written +operation gets, and what changes about how you must run it. It is useful even without +running pg-sprite: it is the "which safer idioms leave what behind when they fail" table +for PostgreSQL online DDL (background in +[postgres-online-ddl-reference.md](postgres-online-ddl-reference.md)). + +| Operation as written | Recommended safer form | Caveats | +|---|---|---| +| `CREATE INDEX` / `DROP INDEX` / `REINDEX` (non-concurrent) | The `CONCURRENTLY` form of the same statement | `non-transactional`, `invalid-index-on-failure` | +| `ALTER TABLE … DETACH PARTITION` (non-concurrent) | `DETACH PARTITION … CONCURRENTLY` | `non-transactional`, `detach-finalize-on-failure` | +| `ALTER TABLE … ALTER COLUMN … SET NOT NULL` | `ADD CONSTRAINT … CHECK (col IS NOT NULL) NOT VALID` → `VALIDATE CONSTRAINT` → `SET NOT NULL` → `DROP CONSTRAINT` (scaffold) | `separate-transactions`, `validation-scan`, `scaffold-constraint-on-failure` | +| `ALTER TABLE … ADD PRIMARY KEY` / `ADD UNIQUE` | `CREATE UNIQUE INDEX CONCURRENTLY` → `ADD CONSTRAINT … USING INDEX` | `non-transactional`, `separate-transactions`, `invalid-index-on-failure` | +| `ALTER TABLE … ADD CHECK` / `ADD FOREIGN KEY` | `ADD CONSTRAINT … NOT VALID` → `VALIDATE CONSTRAINT` | `separate-transactions`, `validation-scan`, `scaffold-constraint-on-failure` | + +An operation with a constructed rewrite that this table does not cover is a contract +violation: `pkg/suggest` fails closed rather than emitting caveat-less advice, and a test +walks every safer-idiom path the planner produces so the gap is caught in CI, not in a +consumer's report. + +## Examples + +A constructed rewrite (`pg-sprite suggest --json` over +`CREATE INDEX t_c_idx ON t (c)`): + +```json +{ + "format_version": 1, + "suggestions": [ + { + "statement": 1, + "line": 1, + "column": 1, + "original": "CREATE INDEX t_c_idx ON t (c)", + "operation": "CREATE INDEX t_c_idx", + "reason": "safer-idiom", + "recommended": ["CREATE INDEX CONCURRENTLY t_c_idx ON t USING btree (c)"], + "execution": "autocommit-each-step", + "caveats": ["non-transactional", "invalid-index-on-failure"] + } + ] +} +``` + +A guidance suggestion (`ALTER TABLE t ALTER COLUMN c SET NOT NULL, ADD COLUMN d int`): + +```json +{ + "format_version": 1, + "suggestions": [ + { + "statement": 1, + "line": 1, + "column": 1, + "original": "ALTER TABLE t ALTER COLUMN c SET NOT NULL, ADD COLUMN d int", + "operation": "ALTER COLUMN c SET NOT NULL", + "reason": "safer-idiom", + "guidance": "split-statement" + } + ] +} +``` + +## Text output + +Without `--json`, each suggestion renders as the statement header +(`statement N: operation — reason`) followed by either the safer sequence with its caveat +list or the guidance code with its manual path. The text form is for humans; automation +consumes the JSON report. A clean script prints nothing and always exits zero. diff --git a/internal/cli/cli.go b/internal/cli/cli.go index 0e918b5..710a346 100644 --- a/internal/cli/cli.go +++ b/internal/cli/cli.go @@ -1,6 +1,6 @@ // Package cli defines the pg-sprite command tree (Kong): migrate and // status (the optimistic front door), diff and fmt (the declarative front -// door), and lint (the offline checker). +// door), and lint and suggest (the offline checker and advisor). package cli import ( @@ -23,6 +23,7 @@ type CLI struct { Diff DiffCmd `cmd:"" help:"Diff a desired-state schema file against the live schema."` Fmt FmtCmd `cmd:"" help:"Canonicalize a schema file."` Lint LintCmd `cmd:"" help:"Lint DDL for unsafe patterns."` + Suggest SuggestCmd `cmd:"" help:"Recommend safer native forms for risky DDL."` Status StatusCmd `cmd:"" help:"Report the status of a running schema change."` } @@ -121,6 +122,17 @@ type LintCmd struct { // Run implements the lint subcommand. func (c *LintCmd) Run() error { return c.runLint(os.Stdin, os.Stdout) } +// SuggestCmd maps risky-as-written DDL to the safer native form the engine +// would run instead, with typed caveats. It is offline and advisory — no +// database flags, nothing executes, and it always exits zero. +type SuggestCmd struct { + Path string `arg:"" optional:"" help:"DDL file to advise on; stdin when omitted." type:"existingfile"` + JSON bool `help:"Emit the suggestions report as JSON."` +} + +// Run implements the suggest subcommand. +func (c *SuggestCmd) Run() error { return c.runSuggest(os.Stdin, os.Stdout) } + // StatusCmd reports schema-change progress. type StatusCmd struct { DBFlags `embed:""` diff --git a/internal/cli/suggest.go b/internal/cli/suggest.go new file mode 100644 index 0000000..aa1942f --- /dev/null +++ b/internal/cli/suggest.go @@ -0,0 +1,97 @@ +package cli + +import ( + "encoding/json" + "fmt" + "io" + "os" + "strings" + + "github.com/block/pg-sprite/pkg/suggest" +) + +// runSuggest maps a DDL script to its advisory rewrites: parse every +// statement through the PostgreSQL grammar, classify it with zero live +// facts, and report the safer native form for anything risky as written. +// Offline — no database, nothing executes. A script with no rewrites +// prints nothing and the command always exits zero: suggest advises, lint +// gates. +func (c *SuggestCmd) runSuggest(in io.Reader, out io.Writer) error { + var src []byte + var err error + if c.Path == "" { + if src, err = io.ReadAll(in); err != nil { + return fmt.Errorf("read DDL from stdin: %w", err) + } + } else if src, err = os.ReadFile(c.Path); err != nil { + return fmt.Errorf("read DDL file: %w", err) + } + report, err := suggest.Advise(string(src)) + if err != nil { + return err + } + if c.JSON { + enc := json.NewEncoder(out) + enc.SetIndent("", " ") + if err := enc.Encode(report); err != nil { + return fmt.Errorf("write suggest report: %w", err) + } + return nil + } + return writeSuggestText(out, report) +} + +// writeSuggestText renders each suggestion as the original, then either +// the safer sequence with its caveats or the guidance naming the manual +// path. A report with no suggestions prints nothing. +func writeSuggestText(out io.Writer, report suggest.Report) error { + for _, s := range report.Suggestions { + if _, err := fmt.Fprintf(out, "statement %d: %s — %s\n", s.Statement, s.Operation, s.Reason); err != nil { + return fmt.Errorf("write suggest report: %w", err) + } + if len(s.Recommended) == 0 { + if _, err := fmt.Fprintf(out, " guidance: %s — %s\n", s.Guidance, guidanceText(s.Guidance)); err != nil { + return fmt.Errorf("write suggest report: %w", err) + } + continue + } + if _, err := fmt.Fprintf(out, " safer form (not equivalent — see docs/postgres-online-ddl-reference.md):\n"); err != nil { + return fmt.Errorf("write suggest report: %w", err) + } + for _, sql := range s.Recommended { + if _, err := fmt.Fprintf(out, " %s;\n", sql); err != nil { + return fmt.Errorf("write suggest report: %w", err) + } + } + if _, err := fmt.Fprintf(out, " caveats: %s\n", joinCaveats(s.Caveats)); err != nil { + return fmt.Errorf("write suggest report: %w", err) + } + } + return nil +} + +// guidanceText renders the manual path a guidance code names. An unknown +// code renders as itself: the typed value is the contract, the prose is +// display only. +func guidanceText(g suggest.Guidance) string { + switch g { + case suggest.GuidanceSplitStatement: + return "split the statement into one operation per statement, then advise again" + case suggest.GuidanceAddColumnThenConstraint: + return "add the plain column first, then build the constraint with its online pattern" + case suggest.GuidancePrevalidatedCheck: + return "pre-add a validated CHECK matching the partition bound on the child, attach, then drop it" + case suggest.GuidanceNotNullScaffold: + return "prove the invariant with a NOT VALID CHECK plus an online VALIDATE, then SET NOT NULL is a catalog flip" + } + return string(g) +} + +// joinCaveats renders the typed caveats as a comma-separated list. +func joinCaveats(caveats []suggest.Caveat) string { + names := make([]string, len(caveats)) + for i, c := range caveats { + names[i] = string(c) + } + return strings.Join(names, ", ") +} diff --git a/internal/cli/suggest_test.go b/internal/cli/suggest_test.go new file mode 100644 index 0000000..92007fb --- /dev/null +++ b/internal/cli/suggest_test.go @@ -0,0 +1,92 @@ +package cli + +import ( + "encoding/json" + "os" + "path/filepath" + "strings" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "github.com/block/pg-sprite/pkg/suggest" +) + +func TestSuggestCleanScriptPrintsNothing(t *testing.T) { + var out strings.Builder + cmd := SuggestCmd{} + err := cmd.runSuggest(strings.NewReader("CREATE TABLE t (id int)"), &out) + require.NoError(t, err) + assert.Empty(t, out.String()) +} + +// Suggest is advisory: a script full of findings still exits zero — lint +// owns the gate. +func TestSuggestAlwaysExitsZeroOnValidScripts(t *testing.T) { + var out strings.Builder + cmd := SuggestCmd{JSON: true} + err := cmd.runSuggest(strings.NewReader(` + CREATE INDEX t_c_idx ON t (c); + ALTER TABLE t ADD CONSTRAINT no_overlap EXCLUDE USING gist (room WITH =); + `), &out) + require.NoError(t, err) + + var report suggest.Report + require.NoError(t, json.Unmarshal([]byte(out.String()), &report)) + assert.Equal(t, suggest.FormatVersion, report.FormatVersion) + require.Len(t, report.Suggestions, 1, "the refused statement yields no advice") + assert.Equal(t, 1, report.Suggestions[0].Statement) +} + +func TestSuggestReadsFromFile(t *testing.T) { + path := filepath.Join(t.TempDir(), "change.sql") + require.NoError(t, os.WriteFile(path, + []byte("ALTER TABLE t ALTER COLUMN c SET NOT NULL"), 0o600)) + + var out strings.Builder + cmd := SuggestCmd{Path: path, JSON: true} + require.NoError(t, cmd.runSuggest(strings.NewReader(""), &out)) + + var report suggest.Report + require.NoError(t, json.Unmarshal([]byte(out.String()), &report)) + require.Len(t, report.Suggestions, 1) + assert.Equal(t, + []suggest.Caveat{suggest.CaveatSeparateTransactions, suggest.CaveatValidationScan, + suggest.CaveatScaffoldConstraintOnFailure}, + report.Suggestions[0].Caveats) +} + +func TestSuggestParseFailureIsError(t *testing.T) { + var out strings.Builder + cmd := SuggestCmd{} + err := cmd.runSuggest(strings.NewReader("CREATE TABEL t (id int)"), &out) + require.Error(t, err) +} + +// The text rendering is this renderer's own unit test: each suggestion +// shows the safer sequence and its caveats. +func TestSuggestTextRendering(t *testing.T) { + var out strings.Builder + cmd := SuggestCmd{} + err := cmd.runSuggest(strings.NewReader("CREATE INDEX t_c_idx ON t (c)"), &out) + require.NoError(t, err) + text := out.String() + assert.Contains(t, text, "statement 1:") + assert.Contains(t, text, "CONCURRENTLY") + assert.Contains(t, text, string(suggest.CaveatNonTransactional)) + assert.Contains(t, text, string(suggest.CaveatInvalidIndexOnFailure)) +} + +// A suggestion without a constructible rewrite renders its guidance code — +// the manual path — instead of an empty safer-form block. +func TestSuggestTextRendersGuidance(t *testing.T) { + var out strings.Builder + cmd := SuggestCmd{} + err := cmd.runSuggest(strings.NewReader( + "ALTER TABLE t ALTER COLUMN c SET NOT NULL, ADD COLUMN d int"), &out) + require.NoError(t, err) + text := out.String() + assert.Contains(t, text, string(suggest.GuidanceSplitStatement)) + assert.NotContains(t, text, "safer form") +} diff --git a/pkg/suggest/docs_test.go b/pkg/suggest/docs_test.go new file mode 100644 index 0000000..8e35651 --- /dev/null +++ b/pkg/suggest/docs_test.go @@ -0,0 +1,65 @@ +package suggest_test + +import ( + "encoding/json" + "fmt" + "os" + "regexp" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "github.com/block/pg-sprite/pkg/suggest" +) + +// suggestReportDoc is the human-facing contract page these tests keep +// honest. +const suggestReportDoc = "../../docs/suggest-report.md" + +func readDoc(t *testing.T) string { + t.Helper() + raw, err := os.ReadFile(suggestReportDoc) + require.NoError(t, err) + return string(raw) +} + +// The doc's example reports are generated output, not prose: rebuilding +// them through the real advisory pipeline must reproduce the published +// JSON byte for byte (up to JSON equivalence). If this fails, regenerate +// the examples in docs/suggest-report.md. +func TestDocExamplesMatchPipelineOutput(t *testing.T) { + doc := readDoc(t) + blocks := regexp.MustCompile("(?s)```json\n(.*?)```").FindAllStringSubmatch(doc, -1) + require.Len(t, blocks, 2, "the doc publishes one rewrite example and one guidance example") + + for i, sql := range []string{ + "CREATE INDEX t_c_idx ON t (c)", + "ALTER TABLE t ALTER COLUMN c SET NOT NULL, ADD COLUMN d int", + } { + report, err := suggest.Advise(sql) + require.NoError(t, err) + raw, err := json.Marshal(report) + require.NoError(t, err) + assert.JSONEq(t, blocks[i][1], string(raw), + "docs/suggest-report.md example %d drifted from pipeline output", i+1) + } +} + +// Every vocabulary value the contract closes over must be documented: a +// constant added to the code without a row in docs/suggest-report.md (and +// a format_version decision) fails here. +func TestDocListsEveryVocabularyValue(t *testing.T) { + doc := readDoc(t) + var values []string + for _, c := range suggest.Caveats() { + values = append(values, string(c)) + } + for _, g := range suggest.Guidances() { + values = append(values, string(g)) + } + for _, v := range values { + assert.Contains(t, doc, fmt.Sprintf("| `%s` |", v), + "docs/suggest-report.md is missing a vocabulary row for %q", v) + } +} diff --git a/pkg/suggest/residue_integration_test.go b/pkg/suggest/residue_integration_test.go new file mode 100644 index 0000000..c78c0ea --- /dev/null +++ b/pkg/suggest/residue_integration_test.go @@ -0,0 +1,98 @@ +package suggest_test + +import ( + "errors" + "fmt" + "testing" + + "github.com/jackc/pgx/v5/pgconn" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "github.com/block/pg-sprite/internal/testutil" + "github.com/block/pg-sprite/pkg/dbconn" + "github.com/block/pg-sprite/pkg/suggest" +) + +// The SQLSTATEs the scaffold-residue contract is defined by. +const ( + sqlstateCheckViolation = "23514" + sqlstateDuplicateObject = "42710" +) + +// pgCode extracts the SQLSTATE from a PostgreSQL error; empty when the +// error is not a server error. +func pgCode(err error) string { + var pgErr *pgconn.PgError + if errors.As(err, &pgErr) { + return pgErr.Code + } + return "" +} + +// The scaffold-constraint-on-failure caveat describes real behavior: a +// failed VALIDATE leaves the NOT VALID scaffold on the live table, and +// replaying the recommended sequence then fails at the ADD CONSTRAINT step +// with duplicate_object. This pins the failure mode the caveat promises so +// the advice and the server can never drift apart. +func TestSetNotNullSequenceResidueMatchesCaveat(t *testing.T) { + pool, err := dbconn.NewPool(t.Context(), dbconn.Config{URL: testutil.StartPostgres(t)}) + require.NoError(t, err) + t.Cleanup(pool.Close) + schema := testutil.NewSchema(t, pool) + + _, err = pool.Exec(t.Context(), fmt.Sprintf( + "CREATE TABLE %s.orders (id bigint PRIMARY KEY, paid_at timestamptz)", schema)) + require.NoError(t, err) + // One NULL row — the ordinary case a VALIDATE discovers. + _, err = pool.Exec(t.Context(), fmt.Sprintf("INSERT INTO %s.orders VALUES (1, NULL)", schema)) + require.NoError(t, err) + + report, err := suggest.Advise(fmt.Sprintf( + "ALTER TABLE %s.orders ALTER COLUMN paid_at SET NOT NULL", schema)) + require.NoError(t, err) + require.Len(t, report.Suggestions, 1) + s := report.Suggestions[0] + assert.Contains(t, s.Caveats, suggest.CaveatScaffoldConstraintOnFailure, + "the sequence whose residue this test proves must carry the caveat") + require.Len(t, s.Recommended, 4) + + // Step 1 (ADD ... NOT VALID) succeeds; step 2 (VALIDATE) fails on the + // NULL row. + _, err = pool.Exec(t.Context(), s.Recommended[0]) + require.NoError(t, err) + _, err = pool.Exec(t.Context(), s.Recommended[1]) + require.Equal(t, sqlstateCheckViolation, pgCode(err), + "VALIDATE fails as check_violation on the NULL row") + + // The residue the caveat names: the scaffold constraint is still on + // the live table. + var scaffolds int + require.NoError(t, pool.QueryRow(t.Context(), + `SELECT count(*) FROM pg_constraint + WHERE connamespace = $1::regnamespace AND contype = 'c' AND NOT convalidated`, + schema).Scan(&scaffolds)) + assert.Equal(t, 1, scaffolds, "a failed VALIDATE leaves the NOT VALID scaffold behind") + + // A naive replay of the sequence fails at step 1 with + // duplicate_object — the retry behavior the caveat warns about. + _, err = pool.Exec(t.Context(), fmt.Sprintf( + "UPDATE %s.orders SET paid_at = now() WHERE paid_at IS NULL", schema)) + require.NoError(t, err) + _, err = pool.Exec(t.Context(), s.Recommended[0]) + require.Equal(t, sqlstateDuplicateObject, pgCode(err), + "replaying the sequence trips over the leftover scaffold") + + // The recovery the caveat prescribes: resume from the VALIDATE step; + // the rest of the sequence completes. + for _, step := range s.Recommended[1:] { + _, err = pool.Exec(t.Context(), step) + require.NoError(t, err) + } + var notNull bool + require.NoError(t, pool.QueryRow(t.Context(), + `SELECT attnotnull FROM pg_attribute + WHERE attrelid = ($1 || '.orders')::regclass AND attname = 'paid_at'`, + schema).Scan(¬Null)) + assert.True(t, notNull, "resuming from VALIDATE converges on the declared end state") +} diff --git a/pkg/suggest/suggest.go b/pkg/suggest/suggest.go new file mode 100644 index 0000000..3e1d226 --- /dev/null +++ b/pkg/suggest/suggest.go @@ -0,0 +1,286 @@ +// Package suggest is the advisory surface: it maps DDL that is risky as +// written to the safer native form the engine would run instead, offline +// and without executing anything. Every safer-idiom decision yields a +// suggestion — a constructed rewrite carries the safer sequence with typed +// caveats, because a safer form is not a semantic equivalent: it reaches +// the same end state with different locking, transactionality, and failure +// modes; an operation whose rewrite the planner cannot construct carries +// typed guidance naming the manual path instead of staying silent. +// Refusals, table rewrites, and destructive drops are pkg/lint's job. +package suggest + +import ( + "fmt" + + "github.com/block/pg-sprite/pkg/planner" + "github.com/block/pg-sprite/pkg/statement" +) + +// FormatVersion identifies the report contract +// (docs/suggest-report.md). A consumer must reject a report whose version +// it does not understand instead of guessing at the field semantics. +const FormatVersion = 1 + +// Caveat is a typed condition attached to a recommendation; automation +// branches on it, never on prose. The caveats are independent — no caveat +// implies another; a sequence carries every caveat that applies to it. +type Caveat string + +// The caveats a recommendation can carry. +const ( + // CaveatNonTransactional: the recommended sequence contains a + // CONCURRENTLY statement, which cannot run inside a transaction + // block. + CaveatNonTransactional Caveat = "non-transactional" + // CaveatSeparateTransactions: the steps must commit separately — the + // weaker locks the sequence exists for are held to commit, so one + // enclosing transaction reproduces the blocking the rewrite avoids. + CaveatSeparateTransactions Caveat = "separate-transactions" + // CaveatInvalidIndexOnFailure: a failed or cancelled concurrent build + // leaves an INVALID index that must be detected (pg_index.indisvalid) + // and dropped or rebuilt; the engine's executor owns that check when + // it runs the sequence. + CaveatInvalidIndexOnFailure Caveat = "invalid-index-on-failure" + // CaveatDetachFinalizeOnFailure: an interrupted concurrent detach + // leaves the partition half-detached; it must be finished with + // DETACH PARTITION FINALIZE. + CaveatDetachFinalizeOnFailure Caveat = "detach-finalize-on-failure" + // CaveatValidationScan: the VALIDATE step still scans every row — the + // rewrite trades the lock strength, not the scan. + CaveatValidationScan Caveat = "validation-scan" + // CaveatScaffoldConstraintOnFailure: a failed VALIDATE leaves the + // NOT VALID constraint the sequence added on the live table, and + // replaying the sequence then fails at the ADD CONSTRAINT step + // (duplicate_object) — the runner must detect the leftover constraint + // (pg_constraint) and resume from the VALIDATE step, or drop it and + // restart. + CaveatScaffoldConstraintOnFailure Caveat = "scaffold-constraint-on-failure" +) + +// Caveats returns the closed set of Caveat values. It is part of the +// suggest-report contract (docs/suggest-report.md): the set changes only +// with a format_version bump, and a consumer that meets an unrecognized +// value must treat the recommendation as unknown and refuse to run it. +func Caveats() []Caveat { + return []Caveat{ + CaveatNonTransactional, + CaveatSeparateTransactions, + CaveatInvalidIndexOnFailure, + CaveatDetachFinalizeOnFailure, + CaveatValidationScan, + CaveatScaffoldConstraintOnFailure, + } +} + +// Guidance is the typed manual path for a risky operation whose safer form +// the planner cannot construct; automation branches on it, never on prose. +// It is what keeps the advisory surface aligned with pkg/lint: every +// statement lint flags blocking-idiom gets advice here — a constructed +// rewrite or, failing that, guidance. +type Guidance string + +// The guidance codes a suggestion can carry. +const ( + // GuidanceSplitStatement: rewrites are constructed only for + // single-operation statements — a partial rewrite of a compound ALTER + // would be misleading. Split the statement into one operation per + // statement and advise again. + GuidanceSplitStatement Guidance = "split-statement" + // GuidanceAddColumnThenConstraint: an inline UNIQUE / PRIMARY KEY / + // FOREIGN KEY / CHECK on ADD COLUMN builds or validates under the ADD + // COLUMN's ACCESS EXCLUSIVE lock. Add the plain column first, then + // build the constraint with its online pattern. + GuidanceAddColumnThenConstraint Guidance = "add-column-then-constraint" + // GuidancePrevalidatedCheck: ATTACH PARTITION scans the child under + // the parent's lock unless a validated CHECK matching the partition + // bound already exists on the child. Pre-add that CHECK (NOT VALID, + // then VALIDATE), attach, then drop it. The planner cannot construct + // the bound-matching CHECK from the statement alone. + GuidancePrevalidatedCheck Guidance = "pre-add-validated-check" + // GuidanceNotNullScaffold: prove the invariant with a NOT VALID CHECK + // (col IS NOT NULL) plus an online VALIDATE, then the NOT NULL + // constraint is a catalog flip — the same scaffold sequence the + // SET NOT NULL form gets constructed. + GuidanceNotNullScaffold Guidance = "not-null-scaffold" +) + +// Guidances returns the closed set of Guidance values. It is part of the +// suggest-report contract (docs/suggest-report.md): the set changes only +// with a format_version bump, and a consumer that meets an unrecognized +// value must surface the suggestion as unknown rather than ignore it. +func Guidances() []Guidance { + return []Guidance{ + GuidanceSplitStatement, + GuidanceAddColumnThenConstraint, + GuidancePrevalidatedCheck, + GuidanceNotNullScaffold, + } +} + +// Suggestion is one advisory result: the statement as written and either +// the safer native sequence with its typed metadata, or typed guidance +// naming the manual path when no sequence could be constructed. +type Suggestion struct { + // Statement is the 1-based index of the statement in the script. + Statement int `json:"statement"` + // Line is the 1-based source line of the statement's first token, so + // a consumer can annotate the advice onto the file it came from. + Line int `json:"line"` + // Column is the 1-based source column of the statement's first token. + Column int `json:"column"` + // Original is the statement's verbatim source text (without the + // trailing semicolon), so it can be found in the source by exact + // match. + Original string `json:"original"` + // Operation is the operator-facing label of the risky operation + // (display only). + Operation string `json:"operation"` + // Reason is the classifier's typed cause for preferring the rewrite. + Reason planner.Reason `json:"reason"` + // Recommended is the ordered safer SQL to run instead, present exactly + // when the planner constructed the rewrite. Absent, Guidance names the + // manual path. + Recommended []string `json:"recommended,omitempty"` + // Execution is the typed execution contract for Recommended + // (planner.Execution), present exactly when Recommended is. A consumer + // that runs the sequence branches on it instead of prose — it is what + // says the steps must never be wrapped in one transaction block. + Execution planner.Execution `json:"execution,omitempty"` + // Caveats are the typed conditions under which the recommendation + // differs from the original, present exactly when Recommended is and + // never empty — a rewrite with no trade would be the same statement. + Caveats []Caveat `json:"caveats,omitempty"` + // Guidance is the typed manual path, present exactly when Recommended + // is absent: the submitted form still blocks, and this names what to + // do about it. + Guidance Guidance `json:"guidance,omitempty"` +} + +// Report is the advisory result for one script. +type Report struct { + // FormatVersion is the report contract version; always FormatVersion. + FormatVersion int `json:"format_version"` + // Suggestions are the advisory results in statement order, one per + // safer-idiom decision; empty means every statement is already in its + // safest known form or is outside the advisory surface (refusals and + // rewrites are lint findings). + Suggestions []Suggestion `json:"suggestions"` +} + +// Advise maps a DDL script to its advisory results: every statement is +// parsed with the PostgreSQL grammar and classified with zero live facts, +// and each risky-as-written operation yields a Suggestion — the safer +// sequence when the planner could construct it, typed guidance when it +// could not. Nothing is executed and no database is touched. A parse +// failure is an error. +func Advise(sql string) (Report, error) { + stmts, err := statement.Split(sql) + if err != nil { + return Report{}, err + } + report := Report{FormatVersion: FormatVersion, Suggestions: []Suggestion{}} + for i, stmt := range stmts { + suggestions, err := adviseStatement(i+1, stmt) + if err != nil { + return Report{}, fmt.Errorf("statement %d: %w", i+1, err) + } + report.Suggestions = append(report.Suggestions, suggestions...) + } + return report, nil +} + +// adviseStatement produces the suggestions for one statement: one per +// safer-idiom decision. The operation list and decision list are +// index-aligned by the planner's contract (one decision per operation, in +// order); a mismatch is a contract violation and fails closed. +func adviseStatement(index int, stmt statement.SourceStatement) ([]Suggestion, error) { + plan, err := planner.Classify(stmt.SQL, planner.Facts{}) + if err != nil { + return nil, err + } + ops, err := statement.ParseOps(stmt.SQL) + if err != nil { + return nil, err + } + if len(ops) != len(plan.Decisions) { + return nil, fmt.Errorf("planner produced %d decisions for %d operations", len(plan.Decisions), len(ops)) + } + var suggestions []Suggestion + for i, d := range plan.Decisions { + if d.Reason != planner.ReasonSaferIdiom { + continue + } + s := Suggestion{ + Statement: index, + Line: stmt.Line, + Column: stmt.Column, + Original: stmt.SQL, + Operation: d.Operation, + Reason: d.Reason, + } + if len(d.SaferSQL) > 0 { + caveats, err := rewriteCaveats(ops[i]) + if err != nil { + return nil, err + } + s.Recommended, s.Execution, s.Caveats = d.SaferSQL, d.SaferSQLExecution, caveats + } else { + guidance, err := manualGuidance(ops[i], len(ops) > 1) + if err != nil { + return nil, err + } + s.Guidance = guidance + } + suggestions = append(suggestions, s) + } + return suggestions, nil +} + +// rewriteCaveats maps an operation to the typed caveats of its safer +// rewrite. An operation with a rewrite this table does not know is a +// contract violation — when the planner learns a new rewrite, its caveats +// must be recorded here before the advice ships — so it fails closed +// rather than emitting caveat-less advice. +func rewriteCaveats(op statement.Op) ([]Caveat, error) { + switch op.Kind { + case statement.OpCreateIndex, statement.OpDropIndex, statement.OpReindex: + return []Caveat{CaveatNonTransactional, CaveatInvalidIndexOnFailure}, nil + case statement.OpDetachPartition: + return []Caveat{CaveatNonTransactional, CaveatDetachFinalizeOnFailure}, nil + case statement.OpSetNotNull: + return []Caveat{CaveatSeparateTransactions, CaveatValidationScan, CaveatScaffoldConstraintOnFailure}, nil + case statement.OpAddConstraint: + switch op.Constraint { + case statement.ConstraintPrimaryKey, statement.ConstraintUnique: + // The concurrent index build and the USING INDEX attach must + // also commit separately — non-transactional does not imply it. + return []Caveat{CaveatNonTransactional, CaveatSeparateTransactions, CaveatInvalidIndexOnFailure}, nil + case statement.ConstraintCheck, statement.ConstraintForeignKey: + return []Caveat{CaveatSeparateTransactions, CaveatValidationScan, CaveatScaffoldConstraintOnFailure}, nil + } + } + return nil, fmt.Errorf("no caveat mapping for rewritten operation %q", op.Describe()) +} + +// manualGuidance maps a safer-idiom operation without a constructed +// rewrite to the typed manual path. A safer-idiom decision this table does +// not know is a contract violation — when the planner learns a new +// non-constructible pattern, its guidance must be recorded here before the +// advice ships — so it fails closed rather than staying silent about a +// statement lint flags. +func manualGuidance(op statement.Op, multi bool) (Guidance, error) { + if multi { + return GuidanceSplitStatement, nil + } + switch op.Kind { + case statement.OpAddColumn: + return GuidanceAddColumnThenConstraint, nil + case statement.OpAttachPartition: + return GuidancePrevalidatedCheck, nil + case statement.OpAddConstraint: + if op.Constraint == statement.ConstraintNotNull { + return GuidanceNotNullScaffold, nil + } + } + return "", fmt.Errorf("no guidance mapping for non-constructible operation %q", op.Describe()) +} diff --git a/pkg/suggest/suggest_test.go b/pkg/suggest/suggest_test.go new file mode 100644 index 0000000..a5ab19d --- /dev/null +++ b/pkg/suggest/suggest_test.go @@ -0,0 +1,298 @@ +package suggest_test + +import ( + "encoding/json" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "github.com/block/pg-sprite/pkg/lint" + "github.com/block/pg-sprite/pkg/planner" + "github.com/block/pg-sprite/pkg/suggest" +) + +func TestAdviseCleanScriptHasNoSuggestions(t *testing.T) { + report, err := suggest.Advise(` + CREATE TABLE t (id bigint PRIMARY KEY); + ALTER TABLE t ADD COLUMN age int DEFAULT 0; + CREATE INDEX CONCURRENTLY t_age_idx ON t (age); + `) + require.NoError(t, err) + assert.Equal(t, suggest.FormatVersion, report.FormatVersion) + assert.Empty(t, report.Suggestions) +} + +func TestAdviseEmptyScriptIsClean(t *testing.T) { + report, err := suggest.Advise("") + require.NoError(t, err) + assert.Empty(t, report.Suggestions) +} + +func TestAdviseParseFailureIsError(t *testing.T) { + _, err := suggest.Advise("ALTER TABEL t ADD COLUMN c int") + require.Error(t, err) +} + +func TestAdviseCreateIndexGetsConcurrentRewrite(t *testing.T) { + report, err := suggest.Advise("CREATE INDEX t_c_idx ON t (c)") + require.NoError(t, err) + require.Len(t, report.Suggestions, 1) + s := report.Suggestions[0] + assert.Equal(t, 1, s.Statement) + assert.Equal(t, planner.ReasonSaferIdiom, s.Reason) + require.Len(t, s.Recommended, 1) + assert.NotEqual(t, s.Original, s.Recommended[0], "the recommendation is the concurrent rewrite") + assert.Equal(t, planner.ExecutionAutocommit, s.Execution) + assert.Equal(t, + []suggest.Caveat{suggest.CaveatNonTransactional, suggest.CaveatInvalidIndexOnFailure}, + s.Caveats) + assert.Empty(t, s.Guidance, "a constructed rewrite carries no guidance") +} + +func TestAdviseAddCheckGetsNotValidValidateSequence(t *testing.T) { + report, err := suggest.Advise("ALTER TABLE t ADD CONSTRAINT t_age_pos CHECK (age > 0)") + require.NoError(t, err) + require.Len(t, report.Suggestions, 1) + s := report.Suggestions[0] + require.Len(t, s.Recommended, 2, "NOT VALID then VALIDATE") + assert.Equal(t, planner.ExecutionAutocommit, s.Execution) + assert.Equal(t, + []suggest.Caveat{suggest.CaveatSeparateTransactions, suggest.CaveatValidationScan, + suggest.CaveatScaffoldConstraintOnFailure}, + s.Caveats) +} + +func TestAdviseAddPrimaryKeyGetsUsingIndexSequence(t *testing.T) { + report, err := suggest.Advise("ALTER TABLE t ADD PRIMARY KEY (id)") + require.NoError(t, err) + require.Len(t, report.Suggestions, 1) + s := report.Suggestions[0] + require.Len(t, s.Recommended, 2, "concurrent unique index build then USING INDEX attach") + assert.Equal(t, planner.ExecutionAutocommit, s.Execution) + assert.Equal(t, + []suggest.Caveat{suggest.CaveatNonTransactional, suggest.CaveatSeparateTransactions, + suggest.CaveatInvalidIndexOnFailure}, + s.Caveats) +} + +func TestAdviseSetNotNullGetsConstraintSequence(t *testing.T) { + report, err := suggest.Advise("ALTER TABLE t ALTER COLUMN c SET NOT NULL") + require.NoError(t, err) + require.Len(t, report.Suggestions, 1) + s := report.Suggestions[0] + require.Len(t, s.Recommended, 4, "add NOT VALID, validate, set not null, drop scaffold") + assert.Equal(t, planner.ExecutionAutocommit, s.Execution) + assert.Equal(t, + []suggest.Caveat{suggest.CaveatSeparateTransactions, suggest.CaveatValidationScan, + suggest.CaveatScaffoldConstraintOnFailure}, + s.Caveats) +} + +// Statements outside the advisory surface — refusals, table rewrites, +// destructive drops, and forms already safe as written — produce no +// suggestions; they are lint findings, not advice. +func TestAdviseSkipsNonRewritableStatements(t *testing.T) { + report, err := suggest.Advise(` + ALTER TABLE t ADD CONSTRAINT no_overlap EXCLUDE USING gist (room WITH =); + ALTER TABLE t ALTER COLUMN id TYPE bigint; + ALTER TABLE t DROP COLUMN legacy; + ALTER TABLE t ADD CONSTRAINT t_fk FOREIGN KEY (o) REFERENCES orders (id) NOT VALID; + `) + require.NoError(t, err) + assert.Empty(t, report.Suggestions) +} + +// A multi-operation statement gets no partial rewrite — a rewrite of one +// subcommand of a compound ALTER would be misleading — but it is not +// silent: each risky operation carries split-statement guidance. +func TestAdviseMultiOperationStatementsGetSplitGuidance(t *testing.T) { + report, err := suggest.Advise( + "ALTER TABLE t ALTER COLUMN c SET NOT NULL, ADD COLUMN d int") + require.NoError(t, err) + require.Len(t, report.Suggestions, 1, "only the risky operation gets advice") + s := report.Suggestions[0] + assert.Equal(t, planner.ReasonSaferIdiom, s.Reason) + assert.Empty(t, s.Recommended) + assert.Empty(t, s.Execution) + assert.Empty(t, s.Caveats) + assert.Equal(t, suggest.GuidanceSplitStatement, s.Guidance) +} + +// ATTACH PARTITION has a safer native pattern the planner cannot construct +// (the proving CHECK depends on the partition bound); the suggestion names +// the manual path instead of staying silent. +func TestAdviseAttachPartitionGetsPrevalidatedCheckGuidance(t *testing.T) { + report, err := suggest.Advise( + "ALTER TABLE orders ATTACH PARTITION orders_2026 FOR VALUES FROM ('2026-01-01') TO ('2027-01-01')") + require.NoError(t, err) + require.Len(t, report.Suggestions, 1) + s := report.Suggestions[0] + assert.Empty(t, s.Recommended) + assert.Equal(t, suggest.GuidancePrevalidatedCheck, s.Guidance) +} + +// An inline constraint on ADD COLUMN builds under the ADD COLUMN's ACCESS +// EXCLUSIVE lock; the advice is to split the column addition from an +// online constraint build. +func TestAdviseInlineConstraintGetsAddColumnThenConstraintGuidance(t *testing.T) { + report, err := suggest.Advise("ALTER TABLE t ADD COLUMN email text UNIQUE") + require.NoError(t, err) + require.Len(t, report.Suggestions, 1) + s := report.Suggestions[0] + assert.Empty(t, s.Recommended) + assert.Equal(t, suggest.GuidanceAddColumnThenConstraint, s.Guidance) +} + +// lint and suggest agree about the same script: every blocking-idiom +// finding has a suggestion for the same statement — a constructed rewrite +// or typed guidance, never silence. This is the workflow contract: lint +// says what is risky, suggest says what to do about it. +func TestAdviseCoversEveryBlockingIdiomLintFinding(t *testing.T) { + script := `ALTER TABLE orders ALTER COLUMN paid_at SET NOT NULL, ALTER COLUMN shipped_at SET NOT NULL; +ALTER TABLE orders ATTACH PARTITION orders_2026 FOR VALUES FROM ('2026-01-01') TO ('2027-01-01'); +CREATE INDEX orders_ref_idx ON orders (reference);` + + lintReport, err := lint.Check(script) + require.NoError(t, err) + var flagged []int + for _, f := range lintReport.Findings { + if f.Code == lint.CodeBlockingIdiom { + flagged = append(flagged, f.Statement) + } + } + require.Len(t, flagged, 4, "the script exercises the compound, attach, and index paths") + + report, err := suggest.Advise(script) + require.NoError(t, err) + var advised []int + for _, s := range report.Suggestions { + advised = append(advised, s.Statement) + hasRewrite := len(s.Recommended) > 0 + hasGuidance := s.Guidance != "" + assert.True(t, hasRewrite != hasGuidance, + "statement %d: exactly one of recommended and guidance is present", s.Statement) + } + assert.Equal(t, flagged, advised, + "one suggestion per blocking-idiom finding, in the same statement order") +} + +// Every suggestion is complete: a constructed rewrite carries caveats and +// its execution contract, a non-constructible one carries guidance. The +// statements cover every safer-idiom path the planner produces, so an +// unmapped caveat or guidance entry fails here, in CI, not in a consumer's +// report. +func TestAdviseEverySaferIdiomPathIsMapped(t *testing.T) { + for _, sql := range []string{ + "CREATE INDEX t_c_idx ON t (c)", + "DROP INDEX t_c_idx", + "REINDEX INDEX t_c_idx", + "ALTER TABLE t DETACH PARTITION p", + "ALTER TABLE t ALTER COLUMN c SET NOT NULL", + "ALTER TABLE t ADD PRIMARY KEY (id)", + "ALTER TABLE t ADD CONSTRAINT t_c_key UNIQUE (c)", + "ALTER TABLE t ADD CONSTRAINT t_age_pos CHECK (age > 0)", + "ALTER TABLE t ADD CONSTRAINT t_fk FOREIGN KEY (o) REFERENCES orders (id)", + "ALTER TABLE t ATTACH PARTITION p FOR VALUES FROM (1) TO (10)", + "ALTER TABLE t ADD COLUMN email text UNIQUE", + "ALTER TABLE t ALTER COLUMN c SET NOT NULL, ADD COLUMN d int", + } { + report, err := suggest.Advise(sql) + require.NoError(t, err, "statement: %s", sql) + require.NotEmpty(t, report.Suggestions, "statement: %s", sql) + for _, s := range report.Suggestions { + if len(s.Recommended) > 0 { + assert.NotEmpty(t, s.Caveats, "constructed rewrite without caveats: %s", sql) + assert.Equal(t, planner.ExecutionAutocommit, s.Execution, + "constructed rewrite without its execution contract: %s", sql) + assert.Empty(t, s.Guidance, "rewrite and guidance are exclusive: %s", sql) + } else { + assert.NotEmpty(t, s.Guidance, "non-constructible advice without guidance: %s", sql) + assert.Empty(t, s.Caveats, "caveats describe a recommendation: %s", sql) + assert.Empty(t, s.Execution, "execution describes a recommendation: %s", sql) + } + } + } +} + +// Suggestion indexes track the statement position in the script, not the +// suggestion count. +func TestAdviseMultiStatementIndexes(t *testing.T) { + report, err := suggest.Advise(` + CREATE TABLE t (id bigint PRIMARY KEY); + CREATE INDEX t_c_idx ON t (c); + DROP INDEX t_c_idx; + `) + require.NoError(t, err) + require.Len(t, report.Suggestions, 2) + assert.Equal(t, 2, report.Suggestions[0].Statement) + assert.Equal(t, 3, report.Suggestions[1].Statement) +} + +// The JSON shape is the automation-facing contract: exact keys, exact +// omissions, suggestions as [] when clean. +func TestReportJSONShape(t *testing.T) { + report, err := suggest.Advise("CREATE INDEX t_c_idx ON t (c)") + require.NoError(t, err) + raw, err := json.Marshal(report) + require.NoError(t, err) + require.Len(t, report.Suggestions, 1) + recommended, err := json.Marshal(report.Suggestions[0].Recommended) + require.NoError(t, err) + assert.JSONEq(t, `{ + "format_version": 1, + "suggestions": [ + { + "statement": 1, + "line": 1, + "column": 1, + "original": "CREATE INDEX t_c_idx ON t (c)", + "operation": "CREATE INDEX t_c_idx", + "reason": "safer-idiom", + "recommended": `+string(recommended)+`, + "execution": "autocommit-each-step", + "caveats": ["non-transactional", "invalid-index-on-failure"] + } + ] + }`, string(raw)) +} + +// A guidance suggestion omits the rewrite-only keys entirely — recommended, +// execution, and caveats are absent, not null or empty. +func TestReportJSONGuidanceShape(t *testing.T) { + report, err := suggest.Advise( + "ALTER TABLE orders ATTACH PARTITION orders_2026 FOR VALUES FROM ('2026-01-01') TO ('2027-01-01')") + require.NoError(t, err) + raw, err := json.Marshal(report) + require.NoError(t, err) + require.Len(t, report.Suggestions, 1) + original, err := json.Marshal(report.Suggestions[0].Original) + require.NoError(t, err) + operation, err := json.Marshal(report.Suggestions[0].Operation) + require.NoError(t, err) + assert.JSONEq(t, `{ + "format_version": 1, + "suggestions": [ + { + "statement": 1, + "line": 1, + "column": 1, + "original": `+string(original)+`, + "operation": `+string(operation)+`, + "reason": "safer-idiom", + "guidance": "pre-add-validated-check" + } + ] + }`, string(raw)) +} + +func TestReportJSONCleanSuggestionsAreEmptyArray(t *testing.T) { + report, err := suggest.Advise("CREATE TABLE t (id int)") + require.NoError(t, err) + raw, err := json.Marshal(report) + require.NoError(t, err) + assert.JSONEq(t, `{ + "format_version": 1, + "suggestions": [] + }`, string(raw)) +}