Skip to content

Commit 18ba424

Browse files
author
David Ledvinka
committed
Isolate upstream maintenance tooling under .github
1 parent e2e0d73 commit 18ba424

12 files changed

Lines changed: 87 additions & 99 deletions

File tree

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ import Lean.DeclarationRange
88
import Lean.Data.Json
99

1010
/-!
11-
Run `lake env lean scripts/ExportUpstream.lean > .lake/upstream-current.json` after building the
11+
Run `lake env lean .github/upstream/Export.lean > .lake/upstream-current.json` after building the
1212
tracked modules. Source ranges come from the running toolchain's declaration metadata, including
1313
class fields and definition bodies. They are meaningful only at the accompanying Lean commit.
1414
-/
@@ -27,6 +27,8 @@ run_cmd do
2727
let some idx := env.getModuleIdxFor? source
2828
| throwError "No module for upstream declaration `{source}`"
2929
let mod := env.header.moduleNames[idx.toNat]!
30+
unless [`Init, `Lean, `Std].any (·.isPrefixOf mod) do
31+
throwError "Analogue declaration `{source}` belongs to `{mod}`, not Lean core"
3032
let some ranges ← findDeclarationRanges? source
3133
| throwError "No source range for upstream declaration `{source}`"
3234
let first := ranges.range.pos.line
Lines changed: 20 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,8 @@
1-
# Tracking Lean core analogues
1+
# Lean core tracking
2+
3+
This directory contains maintainer tooling run by GitHub Actions. The library provides only the
4+
`analogue` attribute and its annotations; checking, baseline updates, and issue reporting live here.
5+
All commands below run from the repository root.
26

37
Use `@[analogue Core.declaration]` to link an adaptation to its Lean core counterpart:
48

@@ -12,8 +16,8 @@ def randOps ...
1216

1317
Multiple names use spaces: `@[analogue Pure Bind]`. Qualify names when necessary;
1418
`_root_.bind_congr`, for example, avoids resolving to the declaration being annotated.
15-
The attribute checks that each name exists in Lean core and records metadata without changing
16-
any definition. It is defined in `RandomDo/Util/Analogue.lean`.
19+
The attribute resolves and records names. It is defined in `RandomDo/Util/Analogue.lean` and has
20+
no dependency on this tooling. The exporter restricts this workflow's tracked origins to Lean core.
1721

1822
## Checking for changes
1923

@@ -22,12 +26,13 @@ To check the current toolchain against the recorded baselines:
2226

2327
```sh
2428
lake build --wfail RandomDo.Monad.MeasurableSpace RandomDo.Monad.Notation
25-
lake env lean scripts/ExportUpstream.lean > .lake/upstream-current.json
26-
python3 scripts/upstream.py check --metadata .lake/upstream-current.json
29+
lake env lean .github/upstream/Export.lean > .lake/upstream-current.json
30+
python3 .github/upstream/check.py check --metadata .lake/upstream-current.json
2731
```
2832

2933
The build workflow runs this check on PRs. The exporter gets declaration locations from Lean;
30-
`upstream.json` stores the source path, line range, and exact Lean commit for each mapping.
34+
`.github/upstream/baseline.json` stores the source path, line range, and exact Lean commit for each
35+
mapping.
3136
The Python script uses Git to compare those upstream regions with the target revision. It
3237
requires Python 3.10+ and Git, with no Python packages. Later line insertions do not shift the
3338
baseline because its range always refers to its recorded commit.
@@ -41,7 +46,7 @@ after merge to the default branch and can also be run manually.
4146
To run the same comparison locally:
4247

4348
```sh
44-
python3 scripts/upstream.py check --target HEAD --report .lake/upstream-report.md
49+
python3 .github/upstream/check.py check --target HEAD --report .lake/upstream-report.md
4550
```
4651

4752
Sources are fetched into `.lake/upstream-lean.git`; no future Lean code is built or executed.
@@ -55,13 +60,13 @@ Review the upstream diff, adapt the local code as needed, and explain intentiona
5560
in the PR. Then record just the declarations you reviewed:
5661

5762
```sh
58-
python3 scripts/upstream.py record --metadata .lake/upstream-current.json --only RDo.randOps
63+
python3 .github/upstream/check.py record --metadata .lake/upstream-current.json --only RDo.randOps
5964
```
6065

6166
Commit the generated baseline change with the adaptation. Other declarations retain their
6267
baselines. For a rename, pass both old and new names to `--only`; for a removed annotation,
6368
pass its old name. `--all` explicitly records every mapping. Checks never advance baselines.
64-
To track another module, import it in `scripts/ExportUpstream.lean`, export, and record its mappings.
69+
To track another module, import it in `.github/upstream/Export.lean`, export, and record its mappings.
6570

6671
## Scope
6772

@@ -71,14 +76,14 @@ correctness audit. Known differences include the omitted dead-syntax guard in th
7176
verification hooks, and the measurable hierarchy's intentional omission of applicative classes.
7277

7378
The checker reports edits within a declaration, including its body, and conservatively includes
74-
insertions and replacements at either boundary. Renames and file moves are reported at the old location. Changes
75-
to untracked helpers or surrounding sections can also affect behavior, so normal elaboration
76-
and behavior tests remain necessary.
79+
insertions and replacements at either boundary. Renames and file moves are reported at the old
80+
location. Changes to untracked helpers or surrounding sections can also affect behavior, so normal
81+
elaboration and behavior tests remain necessary.
7782

7883
The checker, attribute, and issue tests run offline:
7984

8085
```sh
81-
python3 -m unittest discover -s scripts/tests -p 'test_upstream.py'
82-
lake env lean scripts/tests/Analogue.lean
83-
node --test scripts/tests/test_upstream_issue.cjs
86+
python3 -m unittest discover -s .github/upstream/tests -p 'test_check.py'
87+
lake env lean .github/upstream/tests/Analogue.lean
88+
node --test .github/upstream/tests/test_notify.cjs
8489
```
Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,9 +12,9 @@
1212
from functools import cache
1313
from pathlib import Path, PurePosixPath
1414

15-
ROOT = Path(__file__).resolve().parents[1]
15+
ROOT = Path(__file__).resolve().parents[2]
1616
REPOSITORY = "https://github.com/leanprover/lean4"
17-
BASELINE = ROOT / "upstream.json"
17+
BASELINE = Path(__file__).with_name("baseline.json")
1818
SHA = re.compile(r"[0-9a-f]{40}\Z")
1919
HUNK = re.compile(r"@@ -(\d+)(?:,(\d+))? \+(\d+)(?:,(\d+))? @@")
2020

Lines changed: 7 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -19,16 +19,13 @@ run_cmd do
1919
@[analogue NoSuchCoreDeclaration]
2020
def unknownOrigin : Nat := 0
2121

22-
/-- error: Duplicate analogue declarations -/
23-
#guard_msgs in
24-
@[analogue Pure Pure]
25-
def duplicateOrigin : Nat := 0
26-
27-
/--
28-
error: Analogue declaration `MeasurableSpaceMonad` belongs to `RandomDo.Monad.MeasurableSpace`, not Lean core
29-
-/
30-
#guard_msgs in
22+
-- The attribute records relationships; the CI exporter enforces its Lean core scope.
3123
@[analogue MeasurableSpaceMonad]
32-
def nonCoreOrigin : Nat := 0
24+
def localOrigin : Nat := 0
25+
26+
run_cmd do
27+
let env ← getEnv
28+
unless analogueAttr.getParam? env `AnalogueTests.localOrigin == some #[`MeasurableSpaceMonad] do
29+
throwError "Analogue metadata should allow declarations outside Lean core"
3330

3431
end AnalogueTests
Lines changed: 35 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -10,17 +10,17 @@
1010
import unittest
1111

1212
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
13-
import upstream
13+
import check as checker
1414

1515

1616
class UpstreamTest(unittest.TestCase):
1717
def setUp(self):
1818
self.temp = tempfile.TemporaryDirectory()
1919
self.addCleanup(self.temp.cleanup)
2020
self.repo = Path(self.temp.name)
21-
upstream.git(self.repo, "init", "--quiet")
22-
upstream.git(self.repo, "config", "user.email", "test@example.invalid")
23-
upstream.git(self.repo, "config", "user.name", "Upstream tests")
21+
checker.git(self.repo, "init", "--quiet")
22+
checker.git(self.repo, "config", "user.email", "test@example.invalid")
23+
checker.git(self.repo, "config", "user.name", "Upstream tests")
2424
self.path = self.repo / "src" / "Example.lean"
2525
self.path.parent.mkdir()
2626
self.original = (
@@ -36,22 +36,22 @@ def setUp(self):
3636
"local": "Adapted.foo", "local_file": "RandomDo/Example.lean",
3737
"upstream": "Example.foo", "path": "src/Example.lean", "start": 3, "end": 4,
3838
}]}
39-
self.baseline = upstream.record(None, self.metadata, self.source(), None)
39+
self.baseline = checker.record(None, self.metadata, self.source(), None)
4040

4141
def source(self):
42-
return upstream.Source(self.repo)
42+
return checker.Source(self.repo)
4343

4444
def commit(self):
45-
upstream.git(self.repo, "add", ".")
46-
upstream.git(self.repo, "commit", "--quiet", "--no-gpg-sign", "-m", "fixture")
47-
return upstream.git(self.repo, "rev-parse", "HEAD").strip()
45+
checker.git(self.repo, "add", ".")
46+
checker.git(self.repo, "commit", "--quiet", "--no-gpg-sign", "-m", "fixture")
47+
return checker.git(self.repo, "rev-parse", "HEAD").strip()
4848

4949
def change(self, text):
5050
self.path.write_text(text)
5151
return self.commit()
5252

5353
def report(self, target):
54-
return upstream.check(self.baseline, self.source(), target)
54+
return checker.check(self.baseline, self.source(), target)
5555

5656
def test_unchanged_and_unrelated_edits(self):
5757
self.assertEqual(self.report(self.before)["changes"], [])
@@ -94,79 +94,79 @@ def test_deleted_or_moved_file_is_reported(self):
9494

9595
def test_nonexistent_source_is_an_error(self):
9696
self.baseline["entries"][0]["path"] = "src/Missing.lean"
97-
with self.assertRaises(upstream.CheckError):
97+
with self.assertRaises(checker.CheckError):
9898
self.report(self.before)
9999

100100
def test_missing_revision_is_an_error(self):
101-
with self.assertRaises(upstream.CheckError):
101+
with self.assertRaises(checker.CheckError):
102102
self.report("no-such-revision")
103103

104104
def test_source_range_and_identity_are_validated(self):
105105
for key, value in [("start", 0), ("end", 1000), ("path", "../outside.lean")]:
106106
baseline = copy.deepcopy(self.baseline)
107107
baseline["entries"][0][key] = value
108-
with self.subTest(key=key), self.assertRaises(upstream.CheckError):
109-
upstream.check(baseline, self.source(), self.before)
108+
with self.subTest(key=key), self.assertRaises(checker.CheckError):
109+
checker.check(baseline, self.source(), self.before)
110110
baseline = copy.deepcopy(self.baseline)
111111
baseline["entries"].append(baseline["entries"][0])
112-
with self.assertRaisesRegex(upstream.CheckError, "Duplicate"):
113-
upstream.entries(baseline, baseline=True)
112+
with self.assertRaisesRegex(checker.CheckError, "Duplicate"):
113+
checker.entries(baseline, baseline=True)
114114

115115
def test_new_removed_and_moved_annotations_require_review(self):
116-
upstream.validate_mappings(self.baseline, self.metadata)
116+
checker.validate_mappings(self.baseline, self.metadata)
117117
for key, value in [("local", "Adapted.new"), ("upstream", "Example.new"),
118118
("local_file", "RandomDo/New.lean")]:
119119
metadata = copy.deepcopy(self.metadata)
120120
metadata["entries"][0][key] = value
121-
with self.subTest(key=key), self.assertRaises(upstream.CheckError):
122-
upstream.validate_mappings(self.baseline, metadata)
121+
with self.subTest(key=key), self.assertRaises(checker.CheckError):
122+
checker.validate_mappings(self.baseline, metadata)
123123

124124
def test_range_must_match_exported_metadata_at_the_same_commit(self):
125125
baseline = copy.deepcopy(self.baseline)
126126
baseline["entries"][0]["start"] = 4
127-
with self.assertRaisesRegex(upstream.CheckError, "range disagrees"):
128-
upstream.validate_mappings(baseline, self.metadata)
127+
with self.assertRaisesRegex(checker.CheckError, "range disagrees"):
128+
checker.validate_mappings(baseline, self.metadata)
129129

130130
def test_invalid_commit_is_a_check_error(self):
131131
self.baseline["entries"][0]["commit"] = None
132-
with self.assertRaises(upstream.CheckError):
133-
upstream.entries(self.baseline, baseline=True)
132+
with self.assertRaises(checker.CheckError):
133+
checker.entries(self.baseline, baseline=True)
134134

135135
def test_partial_review_preserves_other_baselines(self):
136136
metadata = copy.deepcopy(self.metadata)
137137
metadata["entries"].append({**metadata["entries"][0], "local": "Adapted.second"})
138-
baseline = upstream.record(None, metadata, self.source(), None)
138+
baseline = checker.record(None, metadata, self.source(), None)
139139
metadata["commit"] = self.change(self.original.replace("x + 1", "x + 2"))
140-
reviewed = upstream.record(baseline, metadata, self.source(), ["Adapted.foo"])
140+
reviewed = checker.record(baseline, metadata, self.source(), ["Adapted.foo"])
141141
by_name = {e["local"]: e for e in reviewed["entries"]}
142142
self.assertEqual(by_name["Adapted.foo"]["commit"], metadata["commit"])
143143
self.assertEqual(by_name["Adapted.second"]["commit"], self.before)
144-
report = upstream.check(reviewed, self.source(), metadata["commit"])
144+
report = checker.check(reviewed, self.source(), metadata["commit"])
145145
self.assertEqual([c["local"] for c in report["changes"]], ["Adapted.second"])
146146

147147
def test_removing_or_renaming_one_annotation_preserves_other_baselines(self):
148148
for renamed in (False, True):
149149
with self.subTest(renamed=renamed):
150150
metadata = copy.deepcopy(self.metadata)
151151
metadata["entries"].append({**metadata["entries"][0], "local": "Adapted.second"})
152-
baseline = upstream.record(None, metadata, self.source(), None)
152+
baseline = checker.record(None, metadata, self.source(), None)
153153
metadata["commit"] = self.change(self.path.read_text() + "-- unrelated\n")
154154
metadata["entries"].pop(0)
155155
selected = ["Adapted.foo"]
156156
if renamed:
157157
metadata["entries"].append({**metadata["entries"][0], "local": "Adapted.new"})
158158
selected.append("Adapted.new")
159-
recorded = upstream.record(baseline, metadata, self.source(), selected)
160-
upstream.validate_mappings(recorded, metadata)
159+
recorded = checker.record(baseline, metadata, self.source(), selected)
160+
checker.validate_mappings(recorded, metadata)
161161
by_name = {e["local"]: e for e in recorded["entries"]}
162162
self.assertNotIn("Adapted.foo", by_name)
163163
self.assertEqual(by_name["Adapted.second"]["commit"], self.before)
164164
if renamed:
165165
self.assertEqual(by_name["Adapted.new"]["commit"], metadata["commit"])
166166

167167
def test_git_display_settings_cannot_hide_or_expand_changes(self):
168-
upstream.git(self.repo, "config", "color.ui", "always")
169-
upstream.git(self.repo, "config", "diff.interHunkContext", "100")
168+
checker.git(self.repo, "config", "color.ui", "always")
169+
checker.git(self.repo, "config", "diff.interHunkContext", "100")
170170
(self.repo / ".gitattributes").write_text("*.lean binary\n")
171171
self.assertFalse(self.report(self.change(self.original.replace("10", "20")))["changes"])
172172
self.assertTrue(self.report(self.change(self.original.replace("x + 1", "x + 2")))["changes"])
@@ -178,10 +178,10 @@ def test_no_automatic_baseline_update_and_exit_codes(self):
178178
target = self.change(self.original.replace("x + 1", "x + 2"))
179179
args = ["--baseline", str(lock), "--repo", str(self.repo), "check", "--target", target]
180180
with contextlib.redirect_stdout(io.StringIO()):
181-
self.assertEqual(upstream.main(args), 1)
182-
self.assertEqual(upstream.main(args + ["--advisory"]), 0)
181+
self.assertEqual(checker.main(args), 1)
182+
self.assertEqual(checker.main(args + ["--advisory"]), 0)
183183
with contextlib.redirect_stderr(io.StringIO()):
184-
self.assertEqual(upstream.main(args[:-1] + ["missing"] + ["--advisory"]), 2)
184+
self.assertEqual(checker.main(args[:-1] + ["missing"] + ["--advisory"]), 2)
185185
self.assertEqual(lock.read_bytes(), original)
186186

187187
def test_notification_fingerprint_ignores_unrelated_commits(self):
@@ -193,7 +193,7 @@ def test_notification_fingerprint_ignores_unrelated_commits(self):
193193

194194
def test_report_includes_mapping_and_upstream_diff(self):
195195
report = self.report(self.change(self.original.replace("x + 1", "x + 2")))
196-
text = upstream.markdown(report)
196+
text = checker.markdown(report)
197197
for expected in ["Adapted.foo", "Example.foo", "- x + 1", "+ x + 2"]:
198198
self.assertIn(expected, text)
199199

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
const assert = require('node:assert/strict');
22
const { test } = require('node:test');
3-
const updateIssue = require('../upstream-issue.cjs');
3+
const updateIssue = require('../notify.cjs');
44

55
const context = {
66
repo: { owner: 'example', repo: 'RandomDo' }, serverUrl: 'https://github.com', runId: 42,

‎.github/workflows/build.yml‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -57,14 +57,14 @@ jobs:
5757

5858
- name: Test upstream tracking
5959
run: |
60-
python3 -m unittest discover -s scripts/tests -p 'test_upstream.py'
61-
node --test scripts/tests/test_upstream_issue.cjs
62-
lake env lean scripts/tests/Analogue.lean
60+
python3 -m unittest discover -s .github/upstream/tests -p 'test_check.py'
61+
node --test .github/upstream/tests/test_notify.cjs
62+
lake env lean .github/upstream/tests/Analogue.lean
6363
6464
- name: Check adapted definitions against the pinned Lean toolchain
6565
run: |
66-
lake env lean scripts/ExportUpstream.lean > .lake/upstream-current.json
67-
python3 scripts/upstream.py check --metadata .lake/upstream-current.json \
66+
lake env lean .github/upstream/Export.lean > .lake/upstream-current.json
67+
python3 .github/upstream/check.py check --metadata .lake/upstream-current.json \
6868
--report .lake/upstream-report.md
6969
7070
- name: Show upstream changes

‎.github/workflows/upstream-core.yml‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -28,14 +28,14 @@ jobs:
2828

2929
- name: Compare tracked definitions with Lean's development branch
3030
run: |
31-
python3 scripts/upstream.py check --target HEAD --advisory \
32-
--report upstream-report.md --json-report upstream-report.json
33-
cat upstream-report.md >> "$GITHUB_STEP_SUMMARY"
31+
python3 .github/upstream/check.py check --target HEAD --advisory \
32+
--report .lake/upstream-report.md --json-report .lake/upstream-report.json
33+
cat .lake/upstream-report.md >> "$GITHUB_STEP_SUMMARY"
3434
3535
- name: Maintain one issue for changes needing review
3636
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8
3737
with:
3838
script: |
3939
const fs = require('fs');
40-
const report = JSON.parse(fs.readFileSync('upstream-report.json', 'utf8'));
41-
await require('./scripts/upstream-issue.cjs')({ github, context }, report);
40+
const report = JSON.parse(fs.readFileSync('.lake/upstream-report.json', 'utf8'));
41+
await require('./.github/upstream/notify.cjs')({ github, context }, report);

0 commit comments

Comments
 (0)