Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,9 @@ Each target's build is pinned in [`src/noldova.checkpoints.git/builds.json`](src

`builds.json` is the record: when it moves to a newer build, the package's `noldova.git.source` field names the new upstream version.

The Linux arm64 build reports itself as `git version 2.56.0.dirty`, though it is built from Git v2.56.0: dugite-native's Ubuntu build names the cross-compiler's strip tool by editing Git's tracked `Makefile` before it builds, and Git marks a build from a changed tree as dirty ([desktop/dugite-native#553](https://github.com/desktop/dugite-native/pull/553) passes the strip tool to `make` instead).
So that target's entry also holds `reportedVersion`, the version the build prints, which the tests check in its place; the weekly Git builds check's issue asks for it to be dropped once a newer build reports its Git version.

Each package's `package.json` names every file of its build in `noldovaDesktop.unpacked`, by its exact path from the package with `/` between folders, so Noldova Desktop's packaging keeps them outside the app's archive, where Git can run them.
The build writes that list from the build it unpacked, sorted, and refuses a committed manifest whose list differs.

Expand Down
3 changes: 3 additions & 0 deletions documents/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -486,6 +486,8 @@ The journal makes a restore finish whole or not at all.
The app ships Git as it ships Electron and Noldova Desktop's native addons: a pinned build per target, checked before use, unpacked from the app's archive and signed with the app, as Noldova Desktop's `documents/BUILD.md` says for its own.
The pinning, the checks and the list of files to unpack are the module's; Noldova Desktop's packaging unpacks and signs them, and the module runs Git from the unpacked copies.
Each target's build is pinned in `src/noldova.checkpoints.git/builds.json` by its address, its SHA-256 and the Git version and source it was built from.
A target whose build prints a version other than its Git version also holds `reportedVersion`, the version `git --version` prints, which the tests check in its place: dugite-native's Linux arm64 build prints `2.56.0.dirty`, as the README's Git builds section says.
The scripts refuse a `reportedVersion` that is not its `gitVersion`, a full stop and lowercase letters, through `TargetReportedVersionInvalidException`.

- Development and tests run the host's pinned build, from its installed package, the same files a packaged app runs.
There is no fallback to an installed Git: when the running target's package is missing, the module refuses to start Git, naming the target, as Noldova Desktop's runtime refuses a missing addon.
Expand All @@ -509,6 +511,7 @@ Each target's build is pinned in `src/noldova.checkpoints.git/builds.json` by it
A weekly scheduled workflow runs `scripts/check-git-builds.ts`, which compares the latest MinGit and dugite-native releases with the tags in `builds.json`'s addresses and files an issue in this repository for each newer one; the maintainers triage it, and a security fix goes ahead of feature work.
- The check orders tags by the numbers in them, so `v2.56.0.windows.2` follows `v2.56.0.windows.1` and `v2.53.0-4` follows `v2.53.0-3`.
- It titles each issue `Pin <source> <tag> in builds.json` and labels it `git-build`.
- When a target it would replace holds `reportedVersion`, the issue also asks for that field to be dropped where the new build reports its Git version.
The maintainers look for open `git-build` issues each week, and the check fails when GitHub leaves the label off.
- It files a release once: a hidden line at the end of its issue names the source and the tag, and a release whose issue it filed before, open or closed, is never filed again, whatever the issue's title has become.
- When a source's latest release has its issue, filed in this run or an earlier one, the check closes, with a comment naming that issue, each of the source's issues for an older release still open, since only the newest release is worth pinning.
Expand Down
24 changes: 24 additions & 0 deletions scripts/exceptions/target-reported-version-invalid.exception.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
/**
* @license
* Copyright (c) Noldova.
*
* This source code is licensed under the license found in the
* LICENSE file in the root directory of this source tree.
*/

import { Resources } from "../resources.ts";
import { PackageException } from "./package.exception.ts";

export class TargetReportedVersionInvalidException extends PackageException {
public readonly file: string;
public readonly gitVersion: string;
public readonly target: string;

public constructor(file: string, target: string, gitVersion: string) {
super(Resources.formatTargetReportedVersionInvalid(file, target, gitVersion));

this.file = file;
this.gitVersion = gitVersion;
this.target = target;
}
}
6 changes: 6 additions & 0 deletions scripts/models/git-target.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import { FieldNotTextListException } from "../exceptions/field-not-text-list.exc
import { TargetAddressInvalidException } from "../exceptions/target-address-invalid.exception.ts";
import { TargetFieldNotTextException } from "../exceptions/target-field-not-text.exception.ts";
import { TargetHashInvalidException } from "../exceptions/target-hash-invalid.exception.ts";
import { TargetReportedVersionInvalidException } from "../exceptions/target-reported-version-invalid.exception.ts";
import { TargetSizeInvalidException } from "../exceptions/target-size-invalid.exception.ts";
import { TargetUnknownException } from "../exceptions/target-unknown.exception.ts";
import { Resources } from "../resources.ts";
Expand All @@ -22,6 +23,7 @@ export class GitTarget {
public readonly path: readonly string[];
public readonly platform: string;
public readonly program: string;
public readonly reportedVersion: string | null;
public readonly sha256: string;
public readonly size: number;
public readonly source: string;
Expand All @@ -45,12 +47,15 @@ export class GitTarget {
const gitVersion = text(Resources.gitVersionField);
const searchPath = GitTarget.readTexts(fields[Resources.pathField], Resources.formatFieldPath(target, Resources.pathField), file);
const program = text(Resources.programField);
const reportedVersion = fields[Resources.reportedVersionField] === undefined ? null : text(Resources.reportedVersionField);
Comment thread
rostislavrotaru marked this conversation as resolved.
Resolved
const sha256 = text(Resources.sha256Field);
const source = text(Resources.sourceField);
const url = text(Resources.urlField);
const size = fields[Resources.sizeField];
if (typeof size !== "number" || !Number.isSafeInteger(size) || size <= 0)
throw new TargetSizeInvalidException(file, target);
if (reportedVersion !== null && !(reportedVersion.startsWith(`${gitVersion}.`) && Resources.reportedSuffixPattern.test(reportedVersion.slice(gitVersion.length + 1))))
throw new TargetReportedVersionInvalidException(file, target, gitVersion);
if (!Resources.sha256Pattern.test(sha256))
throw new TargetHashInvalidException(file, target);
const kind = url.endsWith(Resources.zipExtension) ? Resources.zipKind : url.endsWith(Resources.tarGzExtension) ? Resources.tarGzKind : null;
Expand All @@ -64,6 +69,7 @@ export class GitTarget {
this.path = searchPath;
this.platform = platform;
this.program = program;
this.reportedVersion = reportedVersion;
this.sha256 = sha256;
this.size = size;
this.source = source;
Expand Down
4 changes: 3 additions & 1 deletion scripts/models/pinned-tag.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,12 @@
*/

export class PinnedTag {
public readonly reportedVersion: string | null;
public readonly tag: string;
public readonly target: string;

public constructor(target: string, tag: string) {
public constructor(target: string, tag: string, reportedVersion: string | null) {
this.reportedVersion = reportedVersion;
this.tag = tag;
this.target = target;
}
Expand Down
9 changes: 8 additions & 1 deletion scripts/resources.ts
Original file line number Diff line number Diff line change
Expand Up @@ -265,6 +265,8 @@ export class Resources {
public static readonly releaseVersionPattern: RegExp = /^0\.0\.([1-9]\d*)$/;
public static readonly releaseVersionVariable: string = "RELEASE_VERSION";
public static readonly reportedDifferences: number = 20;
public static readonly reportedSuffixPattern: RegExp = /^[a-z]+$/;
public static readonly reportedVersionField: string = "reportedVersion";
public static readonly repositoryAddress: string = "git+https://github.com/noldova-com/checkpoints.git";
public static readonly repositoryVariable: string = "GITHUB_REPOSITORY";
public static readonly revParseCommand: string = "rev-parse";
Expand Down Expand Up @@ -744,7 +746,7 @@ export class Resources {
return `Installing the packed archives for ${packageName} failed with exit code ${exitCode}:\n${output}`;
}

public static formatIssueBody(source: string, tag: string, url: string, replaced: string, targets: string): string {
public static formatIssueBody(source: string, tag: string, url: string, replaced: string, targets: string, reported: string): string {
return [
"## What and why",
'',
Expand All @@ -754,6 +756,7 @@ export class Resources {
"## Done when",
'',
`- [ ] \`${Resources.buildsFile}\` pins ${source} \`${tag}\` for ${targets}, with each archive's address, size and SHA-256, and its Git version and source.`,
...(reported === "" ? [] : [`- [ ] \`${Resources.buildsFile}\` drops \`${Resources.reportedVersionField}\` for ${reported} where the new build reports its Git version.`]),
"- [ ] A release of the module follows.",
'',
"Filed by the weekly Git builds check.",
Expand Down Expand Up @@ -1049,6 +1052,10 @@ export class Resources {
return `${archiveName}'s package.json names no single target in ${field}, so npm would install it on every target.`;
}

public static formatTargetReportedVersionInvalid(file: string, target: string, gitVersion: string): string {
return `${file}'s ${target}.${Resources.reportedVersionField} must be ${gitVersion}, its gitVersion, then a full stop and lowercase letters, as 2.56.0.dirty is.`;
}

public static formatTargetSizeInvalid(file: string, target: string): string {
return `${file}'s ${target}.size must be the archive's size in bytes.`;
}
Expand Down
5 changes: 3 additions & 2 deletions scripts/services/git-build-check.ts
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,8 @@ export class GitBuildCheck {
const targets = newer.older.map(t => Resources.formatCode(t.target)).join(Resources.listSeparator);
const replaced = [...new Set(newer.older.map(t => t.tag))]
.map(t => Resources.formatReplacedRelease(t, newer.older.filter(u => u.tag === t).map(u => Resources.formatCode(u.target)).join(Resources.listSeparator)));
return Resources.formatIssueBody(newer.source.name, newer.release.tag, newer.release.url, replaced.join(Resources.clauseSeparator), targets);
const reported = newer.older.filter(t => t.reportedVersion !== null).map(t => Resources.formatCode(t.target)).join(Resources.listSeparator);
return Resources.formatIssueBody(newer.source.name, newer.release.tag, newer.release.url, replaced.join(Resources.clauseSeparator), targets, reported);
}

private static numberOf(issue: Readonly<Record<string, unknown>>, context: string): number {
Expand All @@ -142,7 +143,7 @@ export class GitBuildCheck {
private async findNewerAsync(api: GitHubRest, builds: GitBuilds, source: GitBuildSource): Promise<NewerRelease | null> {
const pinned = builds.targets.flatMap(t => {
const tag = source.tagOf(t);
return tag === null ? [] : [new PinnedTag(t.target, tag)];
return tag === null ? [] : [new PinnedTag(t.target, tag, t.reportedVersion)];
});
if (pinned.length === 0)
return null;
Expand Down
1 change: 1 addition & 0 deletions src/noldova.checkpoints.git/builds.json
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@
"size": 25250010,
"sha256": "d5ce120515bf3013a014e6a918aa87ec0f7a6f5f9687c48d50127e637cb9b6c2",
"gitVersion": "2.56.0",
"reportedVersion": "2.56.0.dirty",
"source": "https://github.com/git/git/tree/v2.56.0",
"program": "bin/git",
"execPath": "libexec/git-core",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -53,11 +53,13 @@ describe("GitBuildLocator", { concurrency: true }, () => {
it("runs the installed build for this system, which reports the Git version builds.json pins for it", async () => {
const build = await new GitBuildLocator(import.meta.dirname).locateAsync(process.platform, process.arch);
const builds = new JsonObjectReader(JSON.parse(await readFile(BUILDS_FILE, "utf8")), "Git builds");
const pin = new JsonObjectReader(new JsonObjectReader(builds.read("targets"), "Git builds").read(`${process.platform}-${process.arch}`), "Git build");
const fields = new JsonObjectReader(builds.read("targets"), "Git builds").read(`${process.platform}-${process.arch}`);
const pin = new JsonObjectReader(fields, "Git build");
const reported = typeof fields === "object" && fields !== null && "reportedVersion" in fields ? pin.readString("reportedVersion") : pin.readString("gitVersion");

const { stdout } = await promisify(execFile)(build.program, ["--version"]);

assert.equal(stdout.trim(), `git version ${pin.readString("gitVersion")}`);
assert.equal(stdout.trim(), `git version ${reported}`);
});

it("runs the program's copy in its exec path, where the build has one, so every Git the module starts is the same file", async () => {
Expand Down
Loading