Skip to content

fix: run the two test-apps that turbo has never seen - #371

Merged
NullVoxPopuli merged 13 commits into
universal-ember:mainfrom
gitKrystan:gitkrystan/run-parked-test-apps
Aug 14, 2026
Merged

fix: run the two test-apps that turbo has never seen#371
NullVoxPopuli merged 13 commits into
universal-ember:mainfrom
gitKrystan:gitkrystan/run-parked-test-apps

Conversation

@gitKrystan

@gitKrystan gitKrystan commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

custom-root-url and markdown-and-gjs-md-app name their browser-test script _test:browser, so pnpm turbo test:browser has never matched either package. At first, this appears to have been intentional, but they appear to have real tests in them now that aren't running in CI. This PR renames the scripts and fixes what that turns up.

Why now

8333e36 noted "Omit not implemented tests from running yet", at the end of #264. Both apps were new in that PR, and custom-root-url's crawl really was a stub: a commented-out body behind an assert.ok(true). Both have had real tests for a while.

The gap costs something. For example, #369 rewrote markdown-and-gjs-md-app's crawl snapshot, and three of its six entries are markdown-only's URLs rather than this app's extensionless .gjs.md ones, with /my-folder-name/baz missing.

Worth a look

  • markdown-and-gjs-md-app also spelled its build build:test, which is not a turbo task. test:browser dependsOn build:tests, so renaming just the test script may have handed testem an unbuilt dist/. Both are renamed.
    • That app never wired handlePotentialIndexVisit, so every run logged Page not found for path "/Docs" ten times and passed regardless. It has app/routes/page.ts now and a test that asserts the redirect. The crawl cannot catch this by itself: it only records paths, and /Docs's redirect target still starts with /Docs, which is the whole of what the crawler compares.
  • markdown-only has the same missing hook and logs the same error. I left it alone, since it is green and running today, but it might need a fix in follow-up.
Claude spew: audit of each failure and what fixed it

Five failures, one silent pass, and one CI-only flake. Only one was a bug in the code under test.

Stale assertions: all five failures. Every one traces to 0dab708, which moved the Home group's nav link to the app root. The behavior that commit changed was the bug, and these tests still asserted the old href. Updating them lines up with the assertion 0dab708 already added to multiple-docs-routes.

Test Asserted Now Same property?
nav-active, Documentation page .../Home not active /my-github-project/ not active yes
nav-active, Home page .../Home is active /my-github-project/ is active yes, plus a new /Home doesNotExist guard
nav-active, casing .../Home not active /my-github-project/ not active yes
custom-root-url crawl /Home reachable /my-github-project/ reachable yes
markdown-and-gjs-md-app crawl /Home reachable, ten times absent yes

Nothing was loosened to make these pass. The one edit that looks like it might have been is deleting the KNOWN_REDIRECTS entry for /my-github-project/Home, which used to pin that visit to bar.md. That entry was already dead: the link is /my-github-project/ now, so the lookup missed and expected fell back to the href either way. The redirect itself is still pinned by index-redirect-test.ts, which visits /Home and asserts it lands on /my-folder-name/bar.md.

One real bug, which was not among the failures. markdown-and-gjs-md-app's /Docs group root errored on every run while the suite reported green. visitAllLinks compares URLs rather than checking that a page rendered, and an erroring page keeps its URL, so current.startsWith(expected) held. Wiring handlePotentialIndexVisit is the only change here that touches behavior rather than assertions.

One upstream bug, visible only on CI. The first CI run failed on Must pass an element, selector, or descriptor to click, the revisit flake 0d52f50 documented, in test-support rather than in kolay. The workaround for it is reverted now that 0.9.0 carries the real fix.

Claude spew: Why the two crawl snapshots are written differently

Both crawls now assert deepEqual(visited.sort(), [...]), per @NullVoxPopuli's review.

markdown-and-gjs-md-app is sorted rather than re-recording its visit order. Its old list disagreed with reality by more than the ten /Home visits, and pinning a fresh order would pin exactly the timing artifacts 0d52f50 called out. That app's runtime-compiled .gjs.md pages are the loosest in the repo on ordering.

test-support 0.9.0, which arrived with #369 partway through this branch, dedupes by target itself, so neither crawl needs its own Set any more.

An earlier push here skipped custom-root-url's crawl, because CI hit the revisit flake 0d52f50 left open. 0.9.0 fixed that too: it visits by URL instead of clicking an anchor unless asked for mode: 'click', and the flake lived entirely in the click path. The workaround is reverted and the crawl runs, in 57ms rather than 440ms. docs-app may no longer need its own skipAllLinks either, which I have not tested.

Testing

CI should be running all of these tests now. Locally, one app at a time:

pnpm build                                   # once, at the root
cd test-apps/<app> && pnpm build:tests && pnpm test:browser

Run them one at a time. pnpm turbo test:ember across every app in parallel starves them into 120s browser timeouts.

To see the bug this fixes, check out main and try pnpm test:ember in either app.

AI attribution: 🤖 Claude using Opus 5. Used skills including pr-description and humanizer. The code changes have been reviewed by the author.

gitKrystan and others added 2 commits August 14, 2026 11:24
`custom-root-url` and `markdown-and-gjs-md-app` name their browser-test
script `_test:browser`, so `pnpm turbo test:browser` — the whole of CI's
Browser Tests job — has never matched either package. Their `test:ember`
scripts were broken by the same typo: both call `pnpm test:browser`, a
script that does not exist, so running them locally fails immediately
with `Command "test:browser" not found`.

8333e36 parked them ("Omit not implemented tests from running yet") at
the tail of universal-ember#264, when both apps were brand new and custom-root-url's
crawl was a commented-out body behind an `assert.ok(true)` placeholder.
Both have had real tests for a while now, and 0d52f50 went as far as
registering test waiters to make custom-root-url's crawl deterministic —
so the underscore is stale, not load-bearing.

`markdown-and-gjs-md-app` also spelled its build `build:test`, which is
not a turbo task at all. Since turbo's `test:browser` dependsOn
`build:tests`, un-hiding the test script alone would have handed testem
an unbuilt `dist/` in CI.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both apps' assertions predate 0dab708, which pointed the co-located pages'
nav link at the app root instead of `rootURL + 'Home'`. Nothing has told
them so, because neither has ever run in CI — five failures, one cause.

custom-root-url: three nav-active-state tests looked up the group link by
`href="/my-github-project/Home"`, which no longer exists; they now look up
`/my-github-project/`. The active-state assertions themselves are unchanged,
since GroupNav marks the entry active by group name rather than by href. The
Home-page test also picks up the `/Home` doesNotExist guard that universal-ember#367 added
to multiple-docs-routes, so the link cannot quietly regress. The crawl drops
its dead `KNOWN_REDIRECTS` entry for `/my-github-project/Home` and swaps
`/Home` for `/my-github-project/` in the snapshot. The root needs no redirect
entry: it serves the root index rather than redirecting away from it.

markdown-and-gjs-md-app: rather than re-snapshot the exact visit order, this
adopts the sorted, deduplicated form 0d52f50 moved custom-root-url's crawl to
— visit order and per-source discovery counts are crawler timing artifacts,
but the set of reachable pages is not. Re-recording the order would have
meant pinning 21 timing-dependent entries, and the old list disagreed with
reality by more than its ten `/Home` visits. Dropping the per-visit 250ms
padding along with it takes the crawl from 5.6s to 0.36s; the test waiters
0d52f50 registered are what the delay was standing in for. Ran both apps five
times each: green every time.

`/Home` is simply absent from this app's list rather than replaced, because
its rootURL is `/` and the crawler skips a bare `/` as the page it started on.

Still logs `Page not found for path "/Docs"` ten times, and still passes
anyway: this app wires no `handlePotentialIndexVisit`, and the crawl callback
only records paths. Left alone here — it is a real bug about group roots, not
fallout from the rename.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@bolt-new-by-stackblitz

Copy link
Copy Markdown

Review PR in StackBlitz Codeflow Run & review this pull request in StackBlitz Codeflow.

@vercel

vercel Bot commented Aug 14, 2026

Copy link
Copy Markdown

@gitKrystan is attempting to deploy a commit to the universal-ember Team on Vercel.

A member of the Team first needs to authorize it.

gitKrystan and others added 2 commits August 14, 2026 12:41
Two things the first CI run on these apps turned up.

markdown-and-gjs-md-app never wired `handlePotentialIndexVisit`, so its group
roots errored: every run logged `Page not found for path "/Docs"` ten times and
passed regardless. The crawl cannot catch it — it only records paths, and
`/Docs`'s redirect target shares the `/Docs` prefix that visitAllLinks checks
against — so this went unnoticed for as long as the app went unrun. It now has
the same `app/routes/page.ts` custom-root-url has, plus a test that asserts the
redirect directly rather than trusting the crawl to notice. That module is
declared above the crawl in the same file, because nothing may run after the
crawl: compiling many pages and tearing the app down mid-flight wedges
ember-repl's module-level compiler.

custom-root-url's crawl trips the revisit flake 0d52f50 left open —
visitAllLinks clicks a found anchor without retrying when async content has not
re-rendered, so `find` returns null and `click` throws `Must pass an element,
selector, or descriptor`. It survives ten local runs and fails on CI, where the
same crawl takes 1166ms against 440ms here. Rather than land a known-flaky test,
its testem config now passes `skipAllLinks`, exactly as docs-app's does and for
exactly the same reason. The other eleven tests in that app run. Drop the flag
once test-support waits for the anchor before clicking it.

markdown-only reported a failure in that CI run too. It was collateral: turbo
sent SIGINT when custom-root-url failed. Turbo's own summary names one failing
task.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
universal-ember#369 landed test-support 0.9.0, which changes two things this branch was
working around.

It visits by URL instead of clicking an anchor, unless asked for `mode:
'click'`. The flake that failed this branch's first CI run lived entirely in
the click path — `find` returning null and `click` throwing — so it is gone.
custom-root-url's crawl is enabled again: the `skipAllLinks` added for it is
reverted, and the whole app is green with the crawl running, in 57ms rather
than 440ms.

It also dedupes by target rather than by (source page, target) pair, so the
crawl already yields a set. markdown-and-gjs-md-app's snapshot keeps its sort
— visit order is still timing-dependent, and that app's runtime-compiled
.gjs.md pages are the loosest in the repo — but no longer builds its own Set.

Conflict resolved in markdown-and-gjs-md-app's crawl snapshot, in this
branch's favour. universal-ember#369 rewrote that list too, but for paths this app does not
serve: three of its six entries are wrong — `/Home`, plus
`/my-folder-name/foo.md` and `/Docs/sub-folder/ember-resources.md`, which are
markdown-only's URLs rather than this app's extensionless .gjs.md ones — and
it drops `/my-folder-name/baz` entirely. Checked by running it: it fails.
Nothing caught that, because the app still does not run — its script is still
`_test:browser` on main, which is what the first commit here fixes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@gitKrystan
gitKrystan marked this pull request as ready for review August 14, 2026 21:04
// root rather than under its own name (0dab708). Nothing replaces it in the
// list: this app's rootURL is `/`, and the crawler skips a bare `/` as the
// page it already started on.
for (const path of [...visited].sort()) assert.step(path);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a little silly, just do assert.deepEqual(visit.sort(), [...]);

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

deepEqual done in 177a98d; custom-root-url's crawl matches now too (0d9884a).

On the comment about module order you dropped: I'd asserted that by analogy from custom-root-url and never checked it here. Ran this app with the crawl first and all three tests still pass, so the claim was wrong.

Drafted by Claude, reviewed before posting.

The crawl snapshot compared a sorted array by replaying it through
assert.step and verifying the sequence, which is a long way around
assert.deepEqual. Per @NullVoxPopuli's review.

Most of the comments this branch added were narrating the diff rather than
the code: which map key went away, which assertion did not change, what the
list used to contain. None of that helps someone reading the file as it
stands, so 26 comment lines are now 3 — the module-ordering constraint,
which is invisible in the code and easy to break; why a group-root test
exists when the crawl already visits /Docs; and why the list is sorted.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

const skippable = new URLSearchParams(location.search).has("skipAllLinks") ? skip : test;

// Must stay above the crawl: ember-repl's compiler wedges if anything runs after it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does "wedges" mean here?

module("Group index redirects", function (hooks) {
setupApplicationTest(hooks);

// The crawl can't catch this: it compares URLs, and an erroring `/Docs` keeps its own.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

keeps its own what?

Both were compressed to the point of needing a follow-up question, which is
its own kind of unhelpful. "wedges" was doing the work of "can no longer
serve a new test app", and "keeps its own" was missing its noun.

Still one line each. Per @gitKrystan's review.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Comment thread test-apps/markdown-and-gjs-md-app/tests/application-test.ts Outdated
NullVoxPopuli and others added 2 commits August 14, 2026 17:35
Same swap as the other app, so the two crawls read alike. test-support
0.9.0 visits each target once, so the Set that fed the old step loop had
nothing left to do.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Comment thread test-apps/markdown-and-gjs-md-app/tests/application-test.ts Outdated
Comment thread test-apps/markdown-and-gjs-md-app/tests/application-test.ts Outdated

@NullVoxPopuli NullVoxPopuli left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tyty!

gitKrystan and others added 2 commits August 14, 2026 14:38
@NullVoxPopuli removed the same sentence from markdown-and-gjs-md-app.
`assert.deepEqual(visited.sort(), [...])` says it on its own.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@NullVoxPopuli
NullVoxPopuli merged commit d868786 into universal-ember:main Aug 14, 2026
4 of 5 checks passed
@gitKrystan
gitKrystan deleted the gitkrystan/run-parked-test-apps branch August 14, 2026 21:58
NullVoxPopuli pushed a commit that referenced this pull request Aug 14, 2026
The last test-app without `handlePotentialIndexVisit`. Its `/Docs` group
root errored on every run and the suite reported green, the same way
markdown-and-gjs-md-app did before #371: the crawl compares URLs, and an
erroring `/Docs` keeps its own, so nothing failed. It has app/routes/page.ts
now, and a test that asserts the redirect rather than trusting the crawl to
notice.

The crawl moves to `assert.deepEqual(visited.sort(), [...])` to match the
other two apps, per @NullVoxPopuli on #371. Its per-visit 250ms padding goes
with it: that predates the test waiters 0d52f50 registered, and test-support
0.9.0 visits each target once. 1296ms down to 35ms.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants