fix: run the two test-apps that turbo has never seen - #371
Conversation
`custom-root-url` and `markdown-and-gjs-md-app` name their browser-test script `_test:browser`, so `pnpm turbo test:browser` — the whole of CI's Browser Tests job — has never matched either package. Their `test:ember` scripts were broken by the same typo: both call `pnpm test:browser`, a script that does not exist, so running them locally fails immediately with `Command "test:browser" not found`. 8333e36 parked them ("Omit not implemented tests from running yet") at the tail of universal-ember#264, when both apps were brand new and custom-root-url's crawl was a commented-out body behind an `assert.ok(true)` placeholder. Both have had real tests for a while now, and 0d52f50 went as far as registering test waiters to make custom-root-url's crawl deterministic — so the underscore is stale, not load-bearing. `markdown-and-gjs-md-app` also spelled its build `build:test`, which is not a turbo task at all. Since turbo's `test:browser` dependsOn `build:tests`, un-hiding the test script alone would have handed testem an unbuilt `dist/` in CI. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both apps' assertions predate 0dab708, which pointed the co-located pages' nav link at the app root instead of `rootURL + 'Home'`. Nothing has told them so, because neither has ever run in CI — five failures, one cause. custom-root-url: three nav-active-state tests looked up the group link by `href="/my-github-project/Home"`, which no longer exists; they now look up `/my-github-project/`. The active-state assertions themselves are unchanged, since GroupNav marks the entry active by group name rather than by href. The Home-page test also picks up the `/Home` doesNotExist guard that universal-ember#367 added to multiple-docs-routes, so the link cannot quietly regress. The crawl drops its dead `KNOWN_REDIRECTS` entry for `/my-github-project/Home` and swaps `/Home` for `/my-github-project/` in the snapshot. The root needs no redirect entry: it serves the root index rather than redirecting away from it. markdown-and-gjs-md-app: rather than re-snapshot the exact visit order, this adopts the sorted, deduplicated form 0d52f50 moved custom-root-url's crawl to — visit order and per-source discovery counts are crawler timing artifacts, but the set of reachable pages is not. Re-recording the order would have meant pinning 21 timing-dependent entries, and the old list disagreed with reality by more than its ten `/Home` visits. Dropping the per-visit 250ms padding along with it takes the crawl from 5.6s to 0.36s; the test waiters 0d52f50 registered are what the delay was standing in for. Ran both apps five times each: green every time. `/Home` is simply absent from this app's list rather than replaced, because its rootURL is `/` and the crawler skips a bare `/` as the page it started on. Still logs `Page not found for path "/Docs"` ten times, and still passes anyway: this app wires no `handlePotentialIndexVisit`, and the crawl callback only records paths. Left alone here — it is a real bug about group roots, not fallout from the rename. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
|
|
@gitKrystan is attempting to deploy a commit to the universal-ember Team on Vercel. A member of the Team first needs to authorize it. |
Two things the first CI run on these apps turned up. markdown-and-gjs-md-app never wired `handlePotentialIndexVisit`, so its group roots errored: every run logged `Page not found for path "/Docs"` ten times and passed regardless. The crawl cannot catch it — it only records paths, and `/Docs`'s redirect target shares the `/Docs` prefix that visitAllLinks checks against — so this went unnoticed for as long as the app went unrun. It now has the same `app/routes/page.ts` custom-root-url has, plus a test that asserts the redirect directly rather than trusting the crawl to notice. That module is declared above the crawl in the same file, because nothing may run after the crawl: compiling many pages and tearing the app down mid-flight wedges ember-repl's module-level compiler. custom-root-url's crawl trips the revisit flake 0d52f50 left open — visitAllLinks clicks a found anchor without retrying when async content has not re-rendered, so `find` returns null and `click` throws `Must pass an element, selector, or descriptor`. It survives ten local runs and fails on CI, where the same crawl takes 1166ms against 440ms here. Rather than land a known-flaky test, its testem config now passes `skipAllLinks`, exactly as docs-app's does and for exactly the same reason. The other eleven tests in that app run. Drop the flag once test-support waits for the anchor before clicking it. markdown-only reported a failure in that CI run too. It was collateral: turbo sent SIGINT when custom-root-url failed. Turbo's own summary names one failing task. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
universal-ember#369 landed test-support 0.9.0, which changes two things this branch was working around. It visits by URL instead of clicking an anchor, unless asked for `mode: 'click'`. The flake that failed this branch's first CI run lived entirely in the click path — `find` returning null and `click` throwing — so it is gone. custom-root-url's crawl is enabled again: the `skipAllLinks` added for it is reverted, and the whole app is green with the crawl running, in 57ms rather than 440ms. It also dedupes by target rather than by (source page, target) pair, so the crawl already yields a set. markdown-and-gjs-md-app's snapshot keeps its sort — visit order is still timing-dependent, and that app's runtime-compiled .gjs.md pages are the loosest in the repo — but no longer builds its own Set. Conflict resolved in markdown-and-gjs-md-app's crawl snapshot, in this branch's favour. universal-ember#369 rewrote that list too, but for paths this app does not serve: three of its six entries are wrong — `/Home`, plus `/my-folder-name/foo.md` and `/Docs/sub-folder/ember-resources.md`, which are markdown-only's URLs rather than this app's extensionless .gjs.md ones — and it drops `/my-folder-name/baz` entirely. Checked by running it: it fails. Nothing caught that, because the app still does not run — its script is still `_test:browser` on main, which is what the first commit here fixes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
| // root rather than under its own name (0dab708). Nothing replaces it in the | ||
| // list: this app's rootURL is `/`, and the crawler skips a bare `/` as the | ||
| // page it already started on. | ||
| for (const path of [...visited].sort()) assert.step(path); |
There was a problem hiding this comment.
this is a little silly, just do assert.deepEqual(visit.sort(), [...]);
There was a problem hiding this comment.
deepEqual done in 177a98d; custom-root-url's crawl matches now too (0d9884a).
On the comment about module order you dropped: I'd asserted that by analogy from custom-root-url and never checked it here. Ran this app with the crawl first and all three tests still pass, so the claim was wrong.
Drafted by Claude, reviewed before posting.
The crawl snapshot compared a sorted array by replaying it through assert.step and verifying the sequence, which is a long way around assert.deepEqual. Per @NullVoxPopuli's review. Most of the comments this branch added were narrating the diff rather than the code: which map key went away, which assertion did not change, what the list used to contain. None of that helps someone reading the file as it stands, so 26 comment lines are now 3 — the module-ordering constraint, which is invisible in the code and easy to break; why a group-root test exists when the crawl already visits /Docs; and why the list is sorted. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
|
||
| const skippable = new URLSearchParams(location.search).has("skipAllLinks") ? skip : test; | ||
|
|
||
| // Must stay above the crawl: ember-repl's compiler wedges if anything runs after it. |
There was a problem hiding this comment.
What does "wedges" mean here?
| module("Group index redirects", function (hooks) { | ||
| setupApplicationTest(hooks); | ||
|
|
||
| // The crawl can't catch this: it compares URLs, and an erroring `/Docs` keeps its own. |
There was a problem hiding this comment.
keeps its own what?
Both were compressed to the point of needing a follow-up question, which is its own kind of unhelpful. "wedges" was doing the work of "can no longer serve a new test app", and "keeps its own" was missing its noun. Still one line each. Per @gitKrystan's review. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Same swap as the other app, so the two crawls read alike. test-support 0.9.0 visits each target once, so the Set that fed the old step loop had nothing left to do. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@NullVoxPopuli removed the same sentence from markdown-and-gjs-md-app. `assert.deepEqual(visited.sort(), [...])` says it on its own. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The last test-app without `handlePotentialIndexVisit`. Its `/Docs` group root errored on every run and the suite reported green, the same way markdown-and-gjs-md-app did before #371: the crawl compares URLs, and an erroring `/Docs` keeps its own, so nothing failed. It has app/routes/page.ts now, and a test that asserts the redirect rather than trusting the crawl to notice. The crawl moves to `assert.deepEqual(visited.sort(), [...])` to match the other two apps, per @NullVoxPopuli on #371. Its per-visit 250ms padding goes with it: that predates the test waiters 0d52f50 registered, and test-support 0.9.0 visits each target once. 1296ms down to 35ms. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
custom-root-urlandmarkdown-and-gjs-md-appname their browser-test script_test:browser, sopnpm turbo test:browserhas never matched either package. At first, this appears to have been intentional, but they appear to have real tests in them now that aren't running in CI. This PR renames the scripts and fixes what that turns up.Why now
8333e36 noted "Omit not implemented tests from running yet", at the end of #264. Both apps were new in that PR, and
custom-root-url's crawl really was a stub: a commented-out body behind anassert.ok(true). Both have had real tests for a while.The gap costs something. For example, #369 rewrote
markdown-and-gjs-md-app's crawl snapshot, and three of its six entries aremarkdown-only's URLs rather than this app's extensionless.gjs.mdones, with/my-folder-name/bazmissing.Worth a look
markdown-and-gjs-md-appalso spelled its buildbuild:test, which is not a turbo task.test:browserdependsOnbuild:tests, so renaming just the test script may have handed testem an unbuiltdist/. Both are renamed.handlePotentialIndexVisit, so every run loggedPage not found for path "/Docs"ten times and passed regardless. It hasapp/routes/page.tsnow and a test that asserts the redirect. The crawl cannot catch this by itself: it only records paths, and/Docs's redirect target still starts with/Docs, which is the whole of what the crawler compares.markdown-onlyhas the same missing hook and logs the same error. I left it alone, since it is green and running today, but it might need a fix in follow-up.Claude spew: audit of each failure and what fixed it
Five failures, one silent pass, and one CI-only flake. Only one was a bug in the code under test.
Stale assertions: all five failures. Every one traces to 0dab708, which moved the Home group's nav link to the app root. The behavior that commit changed was the bug, and these tests still asserted the old href. Updating them lines up with the assertion 0dab708 already added to
multiple-docs-routes..../Homenot active/my-github-project/not active.../Homeis active/my-github-project/is active/HomedoesNotExistguard.../Homenot active/my-github-project/not activecustom-root-urlcrawl/Homereachable/my-github-project/reachablemarkdown-and-gjs-md-appcrawl/Homereachable, ten timesNothing was loosened to make these pass. The one edit that looks like it might have been is deleting the
KNOWN_REDIRECTSentry for/my-github-project/Home, which used to pin that visit tobar.md. That entry was already dead: the link is/my-github-project/now, so the lookup missed andexpectedfell back to the href either way. The redirect itself is still pinned byindex-redirect-test.ts, which visits/Homeand asserts it lands on/my-folder-name/bar.md.One real bug, which was not among the failures.
markdown-and-gjs-md-app's/Docsgroup root errored on every run while the suite reported green.visitAllLinkscompares URLs rather than checking that a page rendered, and an erroring page keeps its URL, socurrent.startsWith(expected)held. WiringhandlePotentialIndexVisitis the only change here that touches behavior rather than assertions.One upstream bug, visible only on CI. The first CI run failed on
Must pass an element, selector, or descriptor to click, the revisit flake 0d52f50 documented, in test-support rather than in kolay. The workaround for it is reverted now that 0.9.0 carries the real fix.Claude spew: Why the two crawl snapshots are written differently
Both crawls now assert
deepEqual(visited.sort(), [...]), per @NullVoxPopuli's review.markdown-and-gjs-md-appis sorted rather than re-recording its visit order. Its old list disagreed with reality by more than the ten/Homevisits, and pinning a fresh order would pin exactly the timing artifacts 0d52f50 called out. That app's runtime-compiled.gjs.mdpages are the loosest in the repo on ordering.test-support 0.9.0, which arrived with #369 partway through this branch, dedupes by target itself, so neither crawl needs its own
Setany more.An earlier push here skipped
custom-root-url's crawl, because CI hit the revisit flake 0d52f50 left open. 0.9.0 fixed that too: it visits by URL instead of clicking an anchor unless asked formode: 'click', and the flake lived entirely in the click path. The workaround is reverted and the crawl runs, in 57ms rather than 440ms.docs-appmay no longer need its ownskipAllLinkseither, which I have not tested.Testing
CI should be running all of these tests now. Locally, one app at a time:
Run them one at a time.
pnpm turbo test:emberacross every app in parallel starves them into 120s browser timeouts.To see the bug this fixes, check out
mainand trypnpm test:emberin either app.AI attribution: 🤖 Claude using Opus 5. Used skills including
pr-descriptionandhumanizer. The code changes have been reviewed by the author.