Fix/penalidade generica nao corta

CHANGELOG.md

@@ -8,7 +8,12 @@ Versions follow [semantic versioning](https://semver.org/spec/v2.0.0.html). Whil
 the major version is `0`, the command-line surface and the JSON contract may
 change between minor versions; both are versioned and documented when they do.
 
-## [Unreleased]
+## [0.2.0] — 2026-08-18
+
+Minor rather than patch: `audit` gained a class of finding it did not have, and
+discovery raises and confirms relationships this tool used to drop. A release
+that changes what comes back from a run should not arrive numbered as if
+nothing had.
 
 ### Added
 
@@ -37,12 +42,70 @@ change between minor versions; both are versioned and documented when they do.
   print it. A schema whose only detected convention was that one got a section
   announcing conventions and naming none.
 
+### Fixed
+
+- **The most ordinary foreign keys in an English schema are no longer discarded
+  unseen.** A domain table — `categories`, `statuses`, `types` — carries a
+  penalty so that dozens of uninteresting-but-real relationships do not bury the
+  findings that justify the tool. That penalty was reaching the score threshold,
+  which is a different thing entirely: `category_id → categories` scored 0.65,
+  took the -0.30, landed at 0.35 under a threshold of 0.50, and was dropped
+  before validation ever ran. The relationship the data would have confirmed at
+  100% containment was never looked at, and the report named a count with no
+  relation beside it.
+
+  The penalty now ranks and never cuts on its own: the threshold sees the score
+  without it, and the reported score still carries it, so the ordering it exists
+  for is unchanged. Every other negative signal still cuts — an ambiguous target
+  means the tool does not know which table is meant, which is exactly what a
+  confidence threshold is for. A candidate already below the threshold on its
+  other signals is still discarded, with its reason.
+
+  It bit hardest where the table name is plural and the name match is therefore
+  normalized rather than exact, which is every Rails and Django schema. Measured
+  against a real server, an English schema goes from 4 confirmed to 5. The cost
+  is one validation query per real domain key. The public corpus cannot measure
+  this either way: it is DDL without a single row, so the penalty, which needs a
+  row estimate, is never emitted there at all.
+
+- **The benchmark report says which corpus schemas it could not measure.** The
+  manifest promises that an absent local dump is stated rather than omitted; the
+  statement went to the test log, which the published document does not carry.
+  Regenerating `docs/benchmark/recall.md` on a machine without the optional dump
+  silently deleted that schema's section, and the result read as a complete
+  corpus rather than a partly measured one.
+
+- **The integration suite no longer fails on Windows.** One assertion required
+  every pipeline stage to report a strictly positive duration. Naming detection
+  is in-memory work over a catalog already read — the published cost table
+  prints it as `0s` even on Linux — and Go's monotonic clock on Windows is
+  coarser than that, so it measured exactly zero and the suite went red on a
+  clean clone. Durations are now allowed to be zero and rejected only if
+  negative; a stage that drops out is caught by the exhaustive stage list, which
+  is what was actually guarding against it.
+
 ### Changed
 
 - The lexical fallback no longer extracts a table's trigrams once per column in
   the database. On a 1,000-table schema the generation stage drops from 17.4s to
   2.5s and from 16 GB of allocation to 11 MB.
 
+### Documentation
+
+- The README said the binary carried four dependencies. It carries six direct
+  ones and links twenty-five modules at 11 MB — the interactive `setup` guide
+  arrived in between, and the sentence did not. The cost is now stated with the
+  reason it was accepted, which is the argument the reader opening `go.mod` is
+  actually looking for.
+- The README now says that the shipped profile default is `pt-br`, and what
+  passing `--profile en` buys an English schema. Every run already printed the
+  profile it used; nothing said which one you would get.
+- The status badge tracked a hand-written "pre-release" through three releases.
+  It now reads the latest release from GitHub, so it cannot go stale again.
+- "Weak and rejected are reported too" was true of weak and half-true of
+  rejected: discards are counted on every run, and named by `--include-rejected`.
+  The sentence now says so.
+
 ## [0.1.2] — 2026-08-13
 
 ### Fixed
@@ -137,6 +200,7 @@ These held from the first release and are covered by tests:
 - **Silence is never absence.** Every report states what was not analysed, so
   "nothing found" can never be confused with "nothing was looked at".
 
+[0.2.0]: https://github.com/lvcas-dotcom/pgfathom/releases/tag/v0.2.0
 [0.1.2]: https://github.com/lvcas-dotcom/pgfathom/releases/tag/v0.1.2
 [0.1.1]: https://github.com/lvcas-dotcom/pgfathom/releases/tag/v0.1.1
 [0.1.0]: https://github.com/lvcas-dotcom/pgfathom/releases/tag/v0.1.0

README.md

@@ -7,7 +7,7 @@
 </p>
 
 <p align="center">
-  <img src="https://img.shields.io/badge/status-pre--release-E04A3F" alt="status: pre-release">
+  <img src="https://img.shields.io/github/v/release/lvcas-dotcom/pgfathom?color=E04A3F&label=release" alt="latest release">
   <img src="https://img.shields.io/badge/go-1.25%2B-00ADD8?logo=go&logoColor=white" alt="Go 1.25+">
   <img src="https://img.shields.io/badge/postgres-13%2B-336791?logo=postgresql&logoColor=white" alt="PostgreSQL 13+">
   <img src="https://img.shields.io/badge/mode-read--only-2E7D32" alt="read-only">
@@ -134,8 +134,9 @@ to be resolved before any constraint can be added.
 plus the `CREATE INDEX CONCURRENTLY` on the child column when it's missing, because an
 unindexed FK child is a classic delete trap.
 
-**Weak** and **rejected** are reported too, so you never wonder why an obvious-looking
-column was ignored.
+**Weak** is reported beside them. **Rejected** candidates are counted on every run and named
+by `--include-rejected`, each with the score and the reason it fell — so you never wonder why
+an obvious-looking column was ignored.
 
 Note the first row: `os_servico.resp_tecnico → funcionario.id`. No name-matching heuristic
 in the world finds that one — `resp_tecnico` looks nothing like `funcionario`. `pgfathom`
@@ -303,9 +304,12 @@ privileges, candidates that timed out, schemas not covered — all of it appears
 coverage block on every run. A clean report means "I looked and it's clean", never "I
 couldn't look".
 
-**Small dependency tree, on purpose.** Four dependencies in the binary. No cgo. The person
-who has to approve running this against production will open `go.mod` first, and we intend
-that to be a short read.
+**Small dependency tree, on purpose.** Six direct dependencies, twenty-five modules in the
+linked binary, 11 MB, no cgo. Four of the six do the work — the driver, the CLI framework,
+the TOML reader and `x/sync`. The other two draw the interactive `setup` guide and are
+reachable from nothing else; that cost was argued before it was paid, in
+[`openspec/project.md`](openspec/project.md). The person who has to approve running this
+against production will open `go.mod` first, and we intend that to be a short read.
 
 ## How it works
 
@@ -342,6 +346,21 @@ catalog  →  usage evidence  →  candidates  →  scoring  →  stats prefilte
 
 Most schema tools assume English. The databases that need this tool most often aren't.
 
+**The shipped default is `pt-br`, and every run prints which profile it used in its first
+line.** That default is not a claim about your database — it is where this tool was built and
+what it was built to read. If your schema is English, say so:
+
+```console
+$ pgfathom discover --profile en
+```
+
+What it costs you if you don't is the plural rules. `categories → category` and `boxes → box`
+are in `en` and in no other profile, so under `pt-br` those columns fall through to the
+lexical fallback — still found, at roughly half the name weight (0.08 against 0.15 for
+`category_id` here), which means they now need the other signals to carry them over the
+threshold rather than clearing it on their own. Naming detection, on by default, reads the
+affix convention off your schema either way; what it reads is affixes, not grammar.
+
 Affix and plural rules live in TOML, not in Go, so teaching `pgfathom` a new convention is
 a config file rather than a patch:
 

internal/bench/corpus_test.go

@@ -60,13 +60,19 @@ func TestCorpus(t *testing.T) {
 
 	started := time.Now()
 	var results []bench.EntryResult
+	var skipped []bench.Skipped
 
 	for _, e := range manifest.Schemas {
 		if os.Getenv("PGFATHOM_BENCH_DSN_"+strings.ToUpper(strings.ReplaceAll(e.Name, "-", "_"))) == "" && !bench.Ready(e) {
 			if e.Kind == bench.FromLocal {
 				// Optional by design: a real dump cannot live in a public
-				// manifest. Its absence is stated rather than omitted.
-				t.Logf("SKIP %s: optional local schema absent from this machine", e.Name)
+				// manifest. Its absence is stated rather than omitted — in the
+				// published report, not only here, or regenerating the file on
+				// a machine without the dump would delete the row and read as
+				// if it had never been measured at all.
+				const reason = "optional local dump, absent from the machine that ran this"
+				t.Logf("SKIP %s: %s", e.Name, reason)
+				skipped = append(skipped, bench.Skipped{Name: e.Name, Reason: reason})
 				continue
 			}
 			t.Fatalf("%s is not in the cache; run: make corpus", e.Name)
@@ -79,7 +85,7 @@ func TestCorpus(t *testing.T) {
 	if len(results) == 0 {
 		t.Fatal("no schema was measured")
 	}
-	if err := bench.WriteReports(results, time.Since(started)); err != nil {
+	if err := bench.WriteReports(results, skipped, time.Since(started)); err != nil {
 		t.Fatalf("writing reports: %v", err)
 	}
 }

internal/bench/report.go

@@ -26,15 +26,33 @@ const (
 	FileCost   = "cost.md"
 )
 
+// Skipped is a manifest entry this run could not measure, and why.
+//
+// It exists because the alternative is worse than it looks. A schema the
+// harness cannot reach simply produces no section, and the regenerated file
+// reads as complete — the reader has no way to tell a corpus of two from a
+// corpus of three measured on a machine that held two. Whoever runs
+// `make benchmark` without the optional local dump would republish the file
+// with a row silently deleted, and the deletion is invisible in review unless
+// somebody diffs against a copy they happened to remember.
+//
+// The manifest already promised this: "presente na máquina, é medido; ausente,
+// o relatório diz que não foi" (bench/corpus.toml). It was said to the test
+// log, which the published document does not carry.
+type Skipped struct {
+	Name   string
+	Reason string
+}
+
 // WriteReports renders both files under docs/benchmark.
-func WriteReports(results []EntryResult, elapsed time.Duration) error {
+func WriteReports(results []EntryResult, skipped []Skipped, elapsed time.Duration) error {
 	dir := filepath.Join(repoRoot(), "docs", "benchmark")
 	if err := os.MkdirAll(dir, 0o755); err != nil {
 		return fmt.Errorf("creating report directory: %w", err)
 	}
 
 	for name, content := range map[string]string{
-		FileRecall: renderRecall(results),
+		FileRecall: renderRecall(results, skipped),
 		FileCost:   renderCost(results, elapsed),
 	} {
 		path := filepath.Join(dir, name)
@@ -45,7 +63,7 @@ func WriteReports(results []EntryResult, elapsed time.Duration) error {
 	return nil
 }
 
-func renderRecall(results []EntryResult) string {
+func renderRecall(results []EntryResult, skipped []Skipped) string {
 	var b strings.Builder
 
 	b.WriteString("# Recovery rate on the corpus\n\n")
@@ -77,6 +95,17 @@ func renderRecall(results []EntryResult) string {
 
 	fmt.Fprintf(&b, "Tool `%s`.\n\n", version())
 
+	if len(skipped) > 0 {
+		b.WriteString("**Not measured on this run.** The manifest names schemas this machine\n")
+		b.WriteString("could not reach, so the sections below are the corpus minus these. A file\n")
+		b.WriteString("regenerated without them is not a smaller corpus; it is the same corpus,\n")
+		b.WriteString("partly measured.\n\n")
+		for _, s := range skipped {
+			fmt.Fprintf(&b, "- `%s` — %s\n", s.Name, s.Reason)
+		}
+		b.WriteString("\n")
+	}
+
 	for _, r := range results {
 		fmt.Fprintf(&b, "## %s\n\n", r.Entry.Name)
 

internal/discovery/discovery_integration_test.go

@@ -232,10 +232,23 @@ func TestStagesAccountForTheRun(t *testing.T) {
 		t.Fatal("a finished run must report where its time went")
 	}
 
+	// Negative, not zero. A stage that quietly dropped out contributes no
+	// entry at all (see timeline.mark), so what catches it is the exhaustive
+	// stage list at the end of this test, not a duration.
+	//
+	// Requiring every stage to be strictly positive tested the clock instead
+	// of the code, and failed on Windows for a real reason: Go's monotonic
+	// clock there is coarser than the fastest stage. Naming detection is pure
+	// in-memory work over the catalog already read — it measures under a
+	// millisecond even on Linux, where the published cost table prints it as
+	// `0s` — so on Windows it measures exactly zero, and the suite went red on
+	// a clean clone for whoever contributes from that platform. Zero is a
+	// truthful measurement of work faster than the clock can see. Negative
+	// would mean the timer itself went backwards.
 	var sum time.Duration
 	for _, s := range res.Stages {
-		if s.Duration <= 0 {
-			t.Errorf("stage %q reports %s", s.Stage, s.Duration)
+		if s.Duration < 0 {
+			t.Errorf("stage %q reports %s: durations cannot run backwards", s.Stage, s.Duration)
 		}
 		sum += s.Duration
 	}

internal/infer/generate.go

@@ -530,7 +530,9 @@ func finalize(res *Result, minScore float64) {
 	survivors := res.Candidates[:0:0]
 
 	for _, c := range res.Candidates {
-		if c.MetaScore >= minScore {
+		// cutScore, not MetaScore: the generic-name penalty ranks and never
+		// cuts on its own. See its doc comment for why the two differ.
+		if cutScore(c) >= minScore {
 			survivors = append(survivors, c)
 			continue
 		}