Skip to content

test(top-module): reproduce over-rebuild on unreferenced lib's .mli change#14476

Merged
ElectreAAS merged 8 commits into
ocaml:mainfrom
robinbb:robinbb-test-topmod-over-invalidation
May 19, 2026
Merged

test(top-module): reproduce over-rebuild on unreferenced lib's .mli change#14476
ElectreAAS merged 8 commits into
ocaml:mainfrom
robinbb:robinbb-test-topmod-over-invalidation

Conversation

@robinbb
Copy link
Copy Markdown
Collaborator

@robinbb robinbb commented May 10, 2026

Summary

Adds a regression test under test/blackbox-tests/test-cases/top-module/ that reproduces a dune ocaml top-module over-rebuild: editing the interface of a library referenced only from .mli (never from .ml) rebuilds the top-module artefact, even though top-module drops the .mli. The test passes today while the bug is present.

Related: #14477.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new blackbox cram test that documents and pins the current dune ocaml top-module rebuild behavior when a library dependency’s interface changes, intended as an observational baseline for future dependency-precision improvements.

Changes:

  • Added cctx-wide-cmi-deps.t to reproduce/pin over-invalidation of the top-module compile rule when an “unreferenced” library’s .cmi changes.
  • Encoded expected rebuild outcomes (REBUILT) for both a real dependency change and the over-invalidation probe case.

Comment thread test/blackbox-tests/test-cases/top-module/cctx-wide-cmi-deps.t Outdated
Comment thread test/blackbox-tests/test-cases/top-module/cctx-wide-cmi-deps.t Outdated
robinbb added a commit to robinbb/dune that referenced this pull request May 10, 2026
Match the four sibling tests in this directory
([load-from-{exe,lib}.t], [load-with-{pp,ppx}.t]) which all use
[make_dune_project] for [dune-project] generation. Pinned to
[3.24] to track current dev behaviour.

Addresses Copilot review at
ocaml#14476 (comment).

Signed-off-by: Robin Bate Boerop <me@robinbb.com>
robinbb added a commit to robinbb/dune that referenced this pull request May 10, 2026
The pipeline's exit status was [head]'s (always zero with input),
laundering away [dune build]'s exit status. Failure modes:

- [dune build @check] fails noisily: error text appears in pipe
  output, cram diff catches the unexpected output. Not masked.
- [dune build @check] fails silently: pipe output empty, matches
  expected empty output. Masked.

The cap added no value on success (output already empty); on
failure it acts as a masking surface. Drop the pipe — the bare
[$ dune build @check] catches both modes via the diff.

Addresses Copilot review at
ocaml#14476 (comment).

Signed-off-by: Robin Bate Boerop <me@robinbb.com>
robinbb added a commit to robinbb/dune that referenced this pull request May 10, 2026
Previously [m.ml] contained [let tag = Dep_for_intf.Tag], a real
textual reference to [Dep_for_intf]. The topmod rule's rebuild on
[dep_for_intf.cmi] change was therefore *legitimate*, not the
cctx-wide-glob over-invalidation under test. The prose claimed
[m.ml] never mentioned [Dep_for_intf]; the code contradicted it.

Fix: change [Dep_for_intf.t] from a constructor variant
([type t = Tag]) to a primitive alias ([type t = int]). [m.ml]
now constructs [let tag = 42] with no textual reference to
[Dep_for_intf]:
- ocamldep on [m.ml] reports no [Dep_for_intf] entry.
- The topmod compile of [m.ml] (with [.mli] dropped) infers
  [tag : int] without needing [dep_for_intf.cmi] for type-checking.
- The cctx-wide glob still pulls [dep_for_intf.cmi] into the
  topmod rule's deps — that is the over-invalidation.

[Dep_for_intf]'s [.mli]-only Perturbation 2 edit also moves to a
primitive shape (adds [val zero : t]), still mutating the cmi.

Addresses Copilot review at
ocaml#14476 (comment).

Signed-off-by: Robin Bate Boerop <me@robinbb.com>
@robinbb robinbb requested a review from Copilot May 10, 2026 20:07
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 2 comments.

Comment thread test/blackbox-tests/test-cases/top-module/cctx-wide-cmi-deps.t Outdated
robinbb added a commit to robinbb/dune that referenced this pull request May 10, 2026
…ertions

[stat -c '%y'] is GNU coreutils — not portable to macOS/BSD
[stat] (which uses [-f]). Replace the mtime-comparison harness
with trace-based assertions that test the property directly:
"did the topmod compile rule run during the second invocation?"

Each [dune ocaml top-module] invocation now writes to a
phase-specific trace file ([_build/trace-impl.csexp],
[_build/trace-intf.csexp]). After each perturbation, the test
queries the trace for any action whose target matches
[\\.topmod/mylib/m\\.ml/mylib__M\\.cmo$]. A non-empty array means
the rule re-evaluated; an empty array would mean it stayed cached.

Drops [stat -c] entirely and asserts the property under
observation rather than a file-system side-effect.

Addresses Copilot review at
ocaml#14476 (comment).

Signed-off-by: Robin Bate Boerop <me@robinbb.com>
@robinbb robinbb requested a review from Copilot May 10, 2026 20:19
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 2 comments.

@robinbb robinbb requested a review from ElectreAAS May 11, 2026 16:25
@robinbb robinbb marked this pull request as ready for review May 11, 2026 16:25
@ElectreAAS
Copy link
Copy Markdown
Collaborator

The title, PR text, test title, and test paragraphs sound way too verbose to me, it's hard to follow.
What do you mean by "observational baseline"? The system performs the same whether under observation or not.

I've left a few comments, sorry if they sound harsh but AI-generated PRs like this need adversarial reviews

Comment thread test/blackbox-tests/test-cases/top-module/cctx-wide-cmi-deps.t Outdated
Comment thread test/blackbox-tests/test-cases/top-module/cctx-wide-cmi-deps.t Outdated
Comment thread test/blackbox-tests/test-cases/top-module/cctx-wide-cmi-deps.t Outdated
Comment thread test/blackbox-tests/test-cases/top-module/cctx-wide-cmi-deps.t Outdated
Comment thread test/blackbox-tests/test-cases/top-module/cctx-wide-cmi-deps.t Outdated
robinbb added a commit to robinbb/dune that referenced this pull request May 11, 2026
Replace "Observational baseline: [dune ocaml top-module]
over-invalidates the top-module compile when an unreferenced
library's interface changes." with "Reproduction: `dune ocaml
top-module` over-rebuilds when an unreferenced library's interface
changes." — adopting the term, code formatting, and verb from
ElectreAAS's suggestion, while keeping the original phrase about
which library is involved.

Addresses review at
ocaml#14476 (comment).

Signed-off-by: Robin Bate Boerop <me@robinbb.com>
robinbb added a commit to robinbb/dune that referenced this pull request May 11, 2026
Per ElectreAAS's review of ocaml#14476:

- r3221893397: drop the "This test pins..." paragraph and
  fold its substance into the `Reproduction:` opener
  ("The test passes while the bug is present; promote when
  fixed."). Preserves the polarity signal and maintainer
  hint in fewer words.

- r3221942293: reword Control and Probe prose to lead with
  the `.mli` change. Both sections edit both `.ml` and
  `.mli`; the `.ml` change is scaffolding so the package
  builds, while the `.mli` change is what alters the `.cmi`
  and triggers the rebuild under test.

- r3221908837: simplify `mylib/m.ml` from two values to one
  (`let tag = Dep_for_impl.value`). Same cmi-deps; `tag` is
  now meaningful instead of throwaway.

- r3221888985: trim dune-internals from the cctx paragraph
  (`Compilation_context.set_obj_dir`,
  `Module_compilation.build_module`, `Module.set_source`,
  "cctx"). Keep the dropped-`.mli` precondition and the
  cmi-deps-from-full-closure fact. Add a labelled
  "Code under test: `src/dune_rules/top_module.ml`" pointer.

Also: convert remaining `[identifier]` code-spans to
backticks (follow-up from the r3221878493 thread; backticks
are the ~82% majority convention in this repo).

Signed-off-by: Robin Bate Boerop <me@robinbb.com>
@robinbb
Copy link
Copy Markdown
Collaborator Author

robinbb commented May 11, 2026

What do you mean by "observational baseline"? The system performs the same whether under observation or not.

It is a "baseline" meaning that it what happens currently, and we expect a change in future (as it is a bug). It is "observed" to emphasise that it is the current behaviour but not the desired behaviour. It's a term I've been using for a whole bunch of tests recently. A better industry standard term is "characterisation tests" (a term coined by Michael Feathers). Really, I want a term that says "this is a regression test that passes, but it is not documenting desired behaviour".

@robinbb
Copy link
Copy Markdown
Collaborator Author

robinbb commented May 11, 2026

I've left a few comments, sorry if they sound harsh but AI-generated PRs like this need adversarial reviews

None sound harsh. Good comments. I have made improvements. See what you think, now, @ElectreAAS.

@robinbb robinbb changed the title test: observational baseline for top-module cctx-wide cmi-dep over-invalidation test(top-module): reproduce over-rebuild on unreferenced lib's .mli change May 11, 2026
@ElectreAAS
Copy link
Copy Markdown
Collaborator

ElectreAAS commented May 12, 2026

Really, I want a term that says "this is a regression test that passes, but it is not documenting desired behaviour".

This is what I've been calling a reproduction case 🤷‍♀️

Bar my comment about modifying the .ml file (and maybe the title of the test cctx-wide-cmi-deps.t being verbosely unclear), this is good to go

robinbb added a commit to robinbb/dune that referenced this pull request May 12, 2026
Reviewers asked why the edit steps rewrote both the `.ml` and the
`.mli` when the test prose framed the scenario as an interface
change (PR ocaml#14476, r3221942293, r3226815617). Switch to the
"pre-implement, then expose" pattern: the `.ml` files carry the
full set of definitions from the start; the initial `.mli` exposes
a subset; the edit step adds a `val …` to the `.mli` alone.

Signed-off-by: Robin Bate Boerop <me@robinbb.com>
@robinbb
Copy link
Copy Markdown
Collaborator Author

robinbb commented May 12, 2026

Really, I want a term that says "this is a regression test that passes, but it is not documenting desired behaviour".

This is what I've been calling a reproduction case 🤷‍♀️

Fair enough. I would expect a "reproduction case" to reproduce the bug (fail the test), that being the difference.

Bar my comment about modifying the .ml file (and maybe the title of the test cctx-wide-cmi-deps.t being verbosely unclear), this is good to go

Excellent! Thanks for your work on this. I've made your suggested improvement. Please have another look, @ElectreAAS.

robinbb added a commit to robinbb/dune that referenced this pull request May 13, 2026
Match the four sibling tests in this directory
([load-from-{exe,lib}.t], [load-with-{pp,ppx}.t]) which all use
[make_dune_project] for [dune-project] generation. Pinned to
[3.24] to track current dev behaviour.

Addresses Copilot review at
ocaml#14476 (comment).

Signed-off-by: Robin Bate Boerop <me@robinbb.com>
robinbb added a commit to robinbb/dune that referenced this pull request May 13, 2026
The pipeline's exit status was [head]'s (always zero with input),
laundering away [dune build]'s exit status. Failure modes:

- [dune build @check] fails noisily: error text appears in pipe
  output, cram diff catches the unexpected output. Not masked.
- [dune build @check] fails silently: pipe output empty, matches
  expected empty output. Masked.

The cap added no value on success (output already empty); on
failure it acts as a masking surface. Drop the pipe — the bare
[$ dune build @check] catches both modes via the diff.

Addresses Copilot review at
ocaml#14476 (comment).

Signed-off-by: Robin Bate Boerop <me@robinbb.com>
robinbb added a commit to robinbb/dune that referenced this pull request May 13, 2026
Previously [m.ml] contained [let tag = Dep_for_intf.Tag], a real
textual reference to [Dep_for_intf]. The topmod rule's rebuild on
[dep_for_intf.cmi] change was therefore *legitimate*, not the
cctx-wide-glob over-invalidation under test. The prose claimed
[m.ml] never mentioned [Dep_for_intf]; the code contradicted it.

Fix: change [Dep_for_intf.t] from a constructor variant
([type t = Tag]) to a primitive alias ([type t = int]). [m.ml]
now constructs [let tag = 42] with no textual reference to
[Dep_for_intf]:
- ocamldep on [m.ml] reports no [Dep_for_intf] entry.
- The topmod compile of [m.ml] (with [.mli] dropped) infers
  [tag : int] without needing [dep_for_intf.cmi] for type-checking.
- The cctx-wide glob still pulls [dep_for_intf.cmi] into the
  topmod rule's deps — that is the over-invalidation.

[Dep_for_intf]'s [.mli]-only Perturbation 2 edit also moves to a
primitive shape (adds [val zero : t]), still mutating the cmi.

Addresses Copilot review at
ocaml#14476 (comment).

Signed-off-by: Robin Bate Boerop <me@robinbb.com>
robinbb added a commit to robinbb/dune that referenced this pull request May 13, 2026
…ertions

[stat -c '%y'] is GNU coreutils — not portable to macOS/BSD
[stat] (which uses [-f]). Replace the mtime-comparison harness
with trace-based assertions that test the property directly:
"did the topmod compile rule run during the second invocation?"

Each [dune ocaml top-module] invocation now writes to a
phase-specific trace file ([_build/trace-impl.csexp],
[_build/trace-intf.csexp]). After each perturbation, the test
queries the trace for any action whose target matches
[\\.topmod/mylib/m\\.ml/mylib__M\\.cmo$]. A non-empty array means
the rule re-evaluated; an empty array would mean it stayed cached.

Drops [stat -c] entirely and asserts the property under
observation rather than a file-system side-effect.

Addresses Copilot review at
ocaml#14476 (comment).

Signed-off-by: Robin Bate Boerop <me@robinbb.com>
robinbb added a commit to robinbb/dune that referenced this pull request May 13, 2026
Replace "Observational baseline: [dune ocaml top-module]
over-invalidates the top-module compile when an unreferenced
library's interface changes." with "Reproduction: `dune ocaml
top-module` over-rebuilds when an unreferenced library's interface
changes." — adopting the term, code formatting, and verb from
ElectreAAS's suggestion, while keeping the original phrase about
which library is involved.

Addresses review at
ocaml#14476 (comment).

Signed-off-by: Robin Bate Boerop <me@robinbb.com>
robinbb added a commit to robinbb/dune that referenced this pull request May 13, 2026
Per ElectreAAS's review of ocaml#14476:

- r3221893397: drop the "This test pins..." paragraph and
  fold its substance into the `Reproduction:` opener
  ("The test passes while the bug is present; promote when
  fixed."). Preserves the polarity signal and maintainer
  hint in fewer words.

- r3221942293: reword Control and Probe prose to lead with
  the `.mli` change. Both sections edit both `.ml` and
  `.mli`; the `.ml` change is scaffolding so the package
  builds, while the `.mli` change is what alters the `.cmi`
  and triggers the rebuild under test.

- r3221908837: simplify `mylib/m.ml` from two values to one
  (`let tag = Dep_for_impl.value`). Same cmi-deps; `tag` is
  now meaningful instead of throwaway.

- r3221888985: trim dune-internals from the cctx paragraph
  (`Compilation_context.set_obj_dir`,
  `Module_compilation.build_module`, `Module.set_source`,
  "cctx"). Keep the dropped-`.mli` precondition and the
  cmi-deps-from-full-closure fact. Add a labelled
  "Code under test: `src/dune_rules/top_module.ml`" pointer.

Also: convert remaining `[identifier]` code-spans to
backticks (follow-up from the r3221878493 thread; backticks
are the ~82% majority convention in this repo).

Signed-off-by: Robin Bate Boerop <me@robinbb.com>
robinbb added a commit to robinbb/dune that referenced this pull request May 13, 2026
Reviewers asked why the edit steps rewrote both the `.ml` and the
`.mli` when the test prose framed the scenario as an interface
change (PR ocaml#14476, r3221942293, r3226815617). Switch to the
"pre-implement, then expose" pattern: the `.ml` files carry the
full set of definitions from the start; the initial `.mli` exposes
a subset; the edit step adds a `val …` to the `.mli` alone.

Signed-off-by: Robin Bate Boerop <me@robinbb.com>
@ElectreAAS
Copy link
Copy Markdown
Collaborator

All good, I'd just change the name of the test: cctx-wide-cmi-deps.t is unclear and still mentions the compilation context which isn't relevant

@robinbb robinbb force-pushed the robinbb-test-topmod-over-invalidation branch from 033f154 to 7db6ba7 Compare May 18, 2026 18:27
robinbb added a commit to robinbb/dune that referenced this pull request May 18, 2026
Match the four sibling tests in this directory
([load-from-{exe,lib}.t], [load-with-{pp,ppx}.t]) which all use
[make_dune_project] for [dune-project] generation. Pinned to
[3.24] to track current dev behaviour.

Addresses Copilot review at
ocaml#14476 (comment).

Signed-off-by: Robin Bate Boerop <me@robinbb.com>
robinbb added a commit to robinbb/dune that referenced this pull request May 18, 2026
The pipeline's exit status was [head]'s (always zero with input),
laundering away [dune build]'s exit status. Failure modes:

- [dune build @check] fails noisily: error text appears in pipe
  output, cram diff catches the unexpected output. Not masked.
- [dune build @check] fails silently: pipe output empty, matches
  expected empty output. Masked.

The cap added no value on success (output already empty); on
failure it acts as a masking surface. Drop the pipe — the bare
[$ dune build @check] catches both modes via the diff.

Addresses Copilot review at
ocaml#14476 (comment).

Signed-off-by: Robin Bate Boerop <me@robinbb.com>
robinbb added a commit to robinbb/dune that referenced this pull request May 18, 2026
Previously [m.ml] contained [let tag = Dep_for_intf.Tag], a real
textual reference to [Dep_for_intf]. The topmod rule's rebuild on
[dep_for_intf.cmi] change was therefore *legitimate*, not the
cctx-wide-glob over-invalidation under test. The prose claimed
[m.ml] never mentioned [Dep_for_intf]; the code contradicted it.

Fix: change [Dep_for_intf.t] from a constructor variant
([type t = Tag]) to a primitive alias ([type t = int]). [m.ml]
now constructs [let tag = 42] with no textual reference to
[Dep_for_intf]:
- ocamldep on [m.ml] reports no [Dep_for_intf] entry.
- The topmod compile of [m.ml] (with [.mli] dropped) infers
  [tag : int] without needing [dep_for_intf.cmi] for type-checking.
- The cctx-wide glob still pulls [dep_for_intf.cmi] into the
  topmod rule's deps — that is the over-invalidation.

[Dep_for_intf]'s [.mli]-only Perturbation 2 edit also moves to a
primitive shape (adds [val zero : t]), still mutating the cmi.

Addresses Copilot review at
ocaml#14476 (comment).

Signed-off-by: Robin Bate Boerop <me@robinbb.com>
robinbb added a commit to robinbb/dune that referenced this pull request May 18, 2026
…ertions

[stat -c '%y'] is GNU coreutils — not portable to macOS/BSD
[stat] (which uses [-f]). Replace the mtime-comparison harness
with trace-based assertions that test the property directly:
"did the topmod compile rule run during the second invocation?"

Each [dune ocaml top-module] invocation now writes to a
phase-specific trace file ([_build/trace-impl.csexp],
[_build/trace-intf.csexp]). After each perturbation, the test
queries the trace for any action whose target matches
[\\.topmod/mylib/m\\.ml/mylib__M\\.cmo$]. A non-empty array means
the rule re-evaluated; an empty array would mean it stayed cached.

Drops [stat -c] entirely and asserts the property under
observation rather than a file-system side-effect.

Addresses Copilot review at
ocaml#14476 (comment).

Signed-off-by: Robin Bate Boerop <me@robinbb.com>
robinbb added a commit to robinbb/dune that referenced this pull request May 18, 2026
Replace "Observational baseline: [dune ocaml top-module]
over-invalidates the top-module compile when an unreferenced
library's interface changes." with "Reproduction: `dune ocaml
top-module` over-rebuilds when an unreferenced library's interface
changes." — adopting the term, code formatting, and verb from
ElectreAAS's suggestion, while keeping the original phrase about
which library is involved.

Addresses review at
ocaml#14476 (comment).

Signed-off-by: Robin Bate Boerop <me@robinbb.com>
robinbb added a commit to robinbb/dune that referenced this pull request May 18, 2026
Per ElectreAAS's review of ocaml#14476:

- r3221893397: drop the "This test pins..." paragraph and
  fold its substance into the `Reproduction:` opener
  ("The test passes while the bug is present; promote when
  fixed."). Preserves the polarity signal and maintainer
  hint in fewer words.

- r3221942293: reword Control and Probe prose to lead with
  the `.mli` change. Both sections edit both `.ml` and
  `.mli`; the `.ml` change is scaffolding so the package
  builds, while the `.mli` change is what alters the `.cmi`
  and triggers the rebuild under test.

- r3221908837: simplify `mylib/m.ml` from two values to one
  (`let tag = Dep_for_impl.value`). Same cmi-deps; `tag` is
  now meaningful instead of throwaway.

- r3221888985: trim dune-internals from the cctx paragraph
  (`Compilation_context.set_obj_dir`,
  `Module_compilation.build_module`, `Module.set_source`,
  "cctx"). Keep the dropped-`.mli` precondition and the
  cmi-deps-from-full-closure fact. Add a labelled
  "Code under test: `src/dune_rules/top_module.ml`" pointer.

Also: convert remaining `[identifier]` code-spans to
backticks (follow-up from the r3221878493 thread; backticks
are the ~82% majority convention in this repo).

Signed-off-by: Robin Bate Boerop <me@robinbb.com>
robinbb added a commit to robinbb/dune that referenced this pull request May 18, 2026
Reviewers asked why the edit steps rewrote both the `.ml` and the
`.mli` when the test prose framed the scenario as an interface
change (PR ocaml#14476, r3221942293, r3226815617). Switch to the
"pre-implement, then expose" pattern: the `.ml` files carry the
full set of definitions from the start; the initial `.mli` exposes
a subset; the edit step adds a `val …` to the `.mli` alone.

Signed-off-by: Robin Bate Boerop <me@robinbb.com>
@robinbb
Copy link
Copy Markdown
Collaborator Author

robinbb commented May 18, 2026

Thanks. Agreed. Renamed to over-rebuild-from-intf-only-dep.t — matches the directory's load-from-*.t verb-then-source pattern, captures the bug (over-rebuild) and the trigger (intf-only-dep), drops the internal cctx term.

Back to you, @ElectreAAS.

robinbb added 8 commits May 18, 2026 16:21
…validation

[dune ocaml top-module] derives a cctx via [Compilation_context.set_obj_dir]
to a private obj_dir and drops [.mli] via [Module.set_source ~ml_kind:Intf
None]. The derived cctx's compile rule pulls cmi-deps from the parent
library's full [(libraries ...)] closure, so editing any [(libraries ...)]
dep's interface invalidates the top-module artefact — even libraries
referenced ONLY from the (dropped) [.mli].

This test pins the current REBUILT behaviour. A future change that filters
[top_module]'s compile-rule cmi-deps to match the [.ml]'s actual references
would flip the assertion to NOT REBUILT and require promotion.

Reported and reproduced during review of ocaml#14474.

Signed-off-by: Robin Bate Boerop <me@robinbb.com>
Match the four sibling tests in this directory
([load-from-{exe,lib}.t], [load-with-{pp,ppx}.t]) which all use
[make_dune_project] for [dune-project] generation. Pinned to
[3.24] to track current dev behaviour.

Addresses Copilot review at
ocaml#14476 (comment).

Signed-off-by: Robin Bate Boerop <me@robinbb.com>
The pipeline's exit status was [head]'s (always zero with input),
laundering away [dune build]'s exit status. Failure modes:

- [dune build @check] fails noisily: error text appears in pipe
  output, cram diff catches the unexpected output. Not masked.
- [dune build @check] fails silently: pipe output empty, matches
  expected empty output. Masked.

The cap added no value on success (output already empty); on
failure it acts as a masking surface. Drop the pipe — the bare
[$ dune build @check] catches both modes via the diff.

Addresses Copilot review at
ocaml#14476 (comment).

Signed-off-by: Robin Bate Boerop <me@robinbb.com>
Previously [m.ml] contained [let tag = Dep_for_intf.Tag], a real
textual reference to [Dep_for_intf]. The topmod rule's rebuild on
[dep_for_intf.cmi] change was therefore *legitimate*, not the
cctx-wide-glob over-invalidation under test. The prose claimed
[m.ml] never mentioned [Dep_for_intf]; the code contradicted it.

Fix: change [Dep_for_intf.t] from a constructor variant
([type t = Tag]) to a primitive alias ([type t = int]). [m.ml]
now constructs [let tag = 42] with no textual reference to
[Dep_for_intf]:
- ocamldep on [m.ml] reports no [Dep_for_intf] entry.
- The topmod compile of [m.ml] (with [.mli] dropped) infers
  [tag : int] without needing [dep_for_intf.cmi] for type-checking.
- The cctx-wide glob still pulls [dep_for_intf.cmi] into the
  topmod rule's deps — that is the over-invalidation.

[Dep_for_intf]'s [.mli]-only Perturbation 2 edit also moves to a
primitive shape (adds [val zero : t]), still mutating the cmi.

Addresses Copilot review at
ocaml#14476 (comment).

Signed-off-by: Robin Bate Boerop <me@robinbb.com>
…ertions

[stat -c '%y'] is GNU coreutils — not portable to macOS/BSD
[stat] (which uses [-f]). Replace the mtime-comparison harness
with trace-based assertions that test the property directly:
"did the topmod compile rule run during the second invocation?"

Each [dune ocaml top-module] invocation now writes to a
phase-specific trace file ([_build/trace-impl.csexp],
[_build/trace-intf.csexp]). After each perturbation, the test
queries the trace for any action whose target matches
[\\.topmod/mylib/m\\.ml/mylib__M\\.cmo$]. A non-empty array means
the rule re-evaluated; an empty array would mean it stayed cached.

Drops [stat -c] entirely and asserts the property under
observation rather than a file-system side-effect.

Addresses Copilot review at
ocaml#14476 (comment).

Signed-off-by: Robin Bate Boerop <me@robinbb.com>
Replace "Observational baseline: [dune ocaml top-module]
over-invalidates the top-module compile when an unreferenced
library's interface changes." with "Reproduction: `dune ocaml
top-module` over-rebuilds when an unreferenced library's interface
changes." — adopting the term, code formatting, and verb from
ElectreAAS's suggestion, while keeping the original phrase about
which library is involved.

Addresses review at
ocaml#14476 (comment).

Signed-off-by: Robin Bate Boerop <me@robinbb.com>
Per ElectreAAS's review of ocaml#14476:

- r3221893397: drop the "This test pins..." paragraph and
  fold its substance into the `Reproduction:` opener
  ("The test passes while the bug is present; promote when
  fixed."). Preserves the polarity signal and maintainer
  hint in fewer words.

- r3221942293: reword Control and Probe prose to lead with
  the `.mli` change. Both sections edit both `.ml` and
  `.mli`; the `.ml` change is scaffolding so the package
  builds, while the `.mli` change is what alters the `.cmi`
  and triggers the rebuild under test.

- r3221908837: simplify `mylib/m.ml` from two values to one
  (`let tag = Dep_for_impl.value`). Same cmi-deps; `tag` is
  now meaningful instead of throwaway.

- r3221888985: trim dune-internals from the cctx paragraph
  (`Compilation_context.set_obj_dir`,
  `Module_compilation.build_module`, `Module.set_source`,
  "cctx"). Keep the dropped-`.mli` precondition and the
  cmi-deps-from-full-closure fact. Add a labelled
  "Code under test: `src/dune_rules/top_module.ml`" pointer.

Also: convert remaining `[identifier]` code-spans to
backticks (follow-up from the r3221878493 thread; backticks
are the ~82% majority convention in this repo).

Signed-off-by: Robin Bate Boerop <me@robinbb.com>
Reviewers asked why the edit steps rewrote both the `.ml` and the
`.mli` when the test prose framed the scenario as an interface
change (PR ocaml#14476, r3221942293, r3226815617). Switch to the
"pre-implement, then expose" pattern: the `.ml` files carry the
full set of definitions from the start; the initial `.mli` exposes
a subset; the edit step adds a `val …` to the `.mli` alone.

Signed-off-by: Robin Bate Boerop <me@robinbb.com>
@robinbb robinbb force-pushed the robinbb-test-topmod-over-invalidation branch from 7db6ba7 to a90d023 Compare May 18, 2026 23:22
@robinbb robinbb requested a review from ElectreAAS May 18, 2026 23:25
@ElectreAAS ElectreAAS merged commit 61403fd into ocaml:main May 19, 2026
50 of 51 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants