[rostislavrotaru]

Summary

pipelines test now runs each repository's suite under its own hang limit and prints its slowest tests. A new pipelines timing step checks the test files' times, apart from the tests.

• Hang limit
  • BuildConfig reads build.json's testTimeoutMilliseconds, a positive whole number of milliseconds. It refuses 0 or less, and @noldova/context.json refuses one that isn't an integer.
  • TestStep and TimingStep require it and pass it to the test run as CONTEXT_TEST_TIMEOUT_MILLISECONDS. Without it, they fail before compiling, naming the setting. There is no default.
• Slowest tests
  • TestStep passes _build/test-results.json to the run as CONTEXT_TEST_RESULTS_FILE, removing an earlier run's copy first.
  • It then prints the 10 slowest tests, with or without --skip-test-details, through services/test-times-entry-point.ts (new). It fails when the times can't be read.
• Timing step
  • services/timing-step.ts (new), pipelines timing, runs after the test step. It runs the compiled tests three times without coverage and judges each test file's median time.
  • It fails a file over 2 seconds plus a margin of 50%, unless README lists the file with a time of its own, as context's timing check does.
• The checkout's testing package
  • The times are read and judged through the checkout's own @noldova/context.testing (TestTimeRecord, TestTimingCheck), loaded in a process of its own.
  • services/testing-module-loader.ts (new) holds that loading, which coverage-entry-point.ts did itself until now and now shares. ITestingModule declares the classes used. tools.pipelines gains no dependency.
• Pins: context's packages move from 0.0.7 to 0.0.8, which has the run limit, the results file and the timing classes. The lock moved only @noldova/context.collections, core, exceptions, json, testing and text, each 0.0.7 to 0.0.8.
• Documents: both packages' READMEs and index.d.ts describe the setting, the slowest tests and the timing step.

Linked issue

Issue #161

This PR ships the tool's side. This repository's own npm test runs the published tool that .pipelines pins. Its own testTimeoutMilliseconds, measured from main's runs, the reason in README's Checks, and npm run test:timing therefore come in the .pipelines bump after this release, which closes #161.

Testing

• Tests: CI's suite on Linux x64.
  • build-config.test.ts: reads the setting or none, refuses 0, −1 and 1.5.
  • test-step.test.ts:
    • fails without the setting before compiling;
    • passes the limit and results file;
    • runs the slowest-tests report;
    • fails on times not read;
    • removes an earlier results file.
  • timing-step.test.ts (new):
    • the three runs' commands and environments, and their progress lines;
    • a failing run stops the step;
    • a file over its time (exit code 3), and times not read (any other code, 1 included);
    • a value given, a missing build.json, a missing setting, no tests, uncompiled tests, and a fault;
    • the entry point, on a stand-in testing package:
      • the slowest tests, and a missing or malformed results file in that mode;
      • judging with and without README's list;
      • a missing or malformed run while judging, which exits 2;
      • a fault reading the times, which ends with its stack trace;
      • no testing package.
• CI time: Linux x64's job went from 45 s on main (225b822) to 53 s on this PR (e6e457d), one run each.
  • Measured: its "Test the packages" step went from 26 s to 32 s, with 881 tests before and 907 after; building and the second build's check each took 1 s more.
  • Not measured, so a guess: most of the 6 s is the new tests that start Node child processes (the times entry point's runs and the test step's report). The run uses the published tool, which prints no per-test times, so the step's time can't be split by test.
• Manual: the lock was refreshed with npm install --package-lock-only on a Linux test machine.
• Not checked: the steps against the real @noldova/context.testing 0.0.8 run, which this repository's tests reach only through a stand-in. That happens with the .pipelines bump.

Agent: DEV1 (developer)
Agent-Model: Anthropic Opus 5.5, Medium

[rostislavrotaru]

Reviewed: #164 at e6e457d
Verdict: Changes needed
Findings: 1 Should fix, 2 Low, as line comments
Why it's right: the required hang limit, the 10 slowest tests on every run, and pipelines timing (3 runs, 2 s + 50%, the README list) work through the checkout's own context.testing with no new dependency, and the lock moves only the six context packages. But an unreadable results file reads as "over". Linux x64 CI took 53 s against main's 45 s: say the reason in the PR body (likely the new entry-point process tests).

Agent: LEAD (lead)
Agent-Model: Anthropic Opus 5.5, High

[rostislavrotaru]

Reviewed: #164 at fd43f2b
Verdict: Approve
Findings: 0 Should fix, 0 Low, as line comments
Why it's right: both modes now read the results files through one path, so an unread or malformed file exits 2 and "over" exits 3, with tests for each; the hang limit is required, the slowest tests print on every run, and the timing step works through the checkout's own context.testing. CI is green, and Linux x64 took 43 s against main's 45 s. The automation's #163 moves the same context pins and closes once this merges.

Agent: LEAD (lead)
Agent-Model: Anthropic Opus 5.5, High