[pull] canary from vercel:canary - #1280
Merged
Merged
Conversation
## Summary Avoid decoding route parameters a second time while selecting prerender metadata. This fixes valid percent-containing URLs returning 500, including both build-generated closed routes and runtime-generated open routes. For example, a request for `/open/docs/space%20here/with%2Fslash/fallback%25` is already normalized for manifest lookup before reaching `PrerenderManifestMatcher`. Its final parameter now contains a literal `%`. The matcher previously used `getRouteMatcher`, which tried to decode the captures again and threw even though this call only needed to determine whether the route structure matched. Test the existing, lazily constructed route regular expression directly instead. Matcher precedence, source-page filtering, and the current server parameter encoding behavior stay unchanged. This does not introduce a new manifest or adapter contract. The open fallback is requested twice to cover generation and reuse. The existing fixture uses `dynamicParams = false` and remains excluded from the Cache Components matrix, which does not support that configuration. This is an independent prerequisite for #97393: the failure reproduces without the parameter-matching API and should be fixed for existing apps too. ## Verification - Before the fix, the new closed and open catch-all requests returned 500 with a parameter-decoding error. After the fix, both return 200 and retain the existing encoded server parameter values. - The focused matcher unit suite passes all six tests. - The three encoding regressions pass with production Webpack after restacking. The API routing suite also passes all 36 tests with fresh-native production Turbopack above this prerequisite, including its existing percent-containing URL cases. - The earlier isolated verification passed package TypeScript and changed-file lint. Deployment verification remains for CI. <!-- NEXT_JS_LLM -->
Make invalidating deleted tasks a no-op. Invalidators now represent a 'weak' reference to a task, and invalidations don't assert task existence. We know that the task exists when the invalidator is created but by the time an invalidation occurs it might not exist anymore. So treat the dependency as weak. Take care to not create 'blank' tasks when querying for them, this can confuse assertions that occur later. ## Why is this safe? Currently it is safe because we never reuse task ids, so invalidating a deleted task is perfectly reasonable, much like invalidating a non-active task, there is nothing to do In the future if we start reusing task ids, then this becomes an ABA problem. An invalidator can point at a new task. This is also not too bad, a spurious invalidation is self healing. Also most invalidators are not actually persisted which limits the risk. So we can either decide to build a mechanism to tear down these stale edges or tolerate spurious invalidations. I think we could make invalidator users subscribe to a 'deleted task id bus' which would allow them to drop references, this could be useful and then the few cases where we _persist_ invalidators (by way of `State` objects) would need to devise a new mechanism (or decide to tolerate the spurious invalidations) --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>
…t transient tasks (#98615) Fix refcounting for transient tasks - ensure transient tasks refcount each other correctly, and are eligible for collection - ensure root tasks are born with a transient_ref_count of 1 Also drop some ultimately harmful conditions from the gc predicate - don't consider `cell_dependent`. cell dependencies form cycles and are subsumed by ancestors - to read a cell you have to read it from a child (covered by parent_count) or be passed it from a parent. For the parent to have passed it to you it must have a child dependency on the producer or be passed it by its parent (recursively). So the task that passed you the vc must be holding ownership over both tasks. - therefore a stale cell_dependency can only exist within a task that is effectively dead, however because we do not reliably clean up all tasks in a session (due to leaking root tasks), we cannot fully exclude the existence of stale cell_dependency edges. We just know they must only exist within unreachable tasks. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ldren (#98843) ## What When removing edges, only enqueue `lost follower` jobs when we actually remove children. ## Why GC can race with other kinds of task completion which also remove outgoing edges. If task completion drops a child from a task that also becomes collectible. Then it is possible for there to be an enqueued CleanupOldEdges job at the time GC runs. Then we will have two jobs that remove children, only one will succeed but both will enqueue `InnerOfUppersLostFollowersJob` for all the chidren. This can lead to panics since one of those jobs will fail to remove the follower The fix is simple, only remove tasks as followers when you actually removed them as children.
Fix the definition of GC roots to account for all transient references There was a categorical mistake in how `uppers` and `collectible_dependents` were interpreted. Both of these refer to a task above us and this is it is a reference but the reference might be comming from transient or persistent tasks. This is what kept confusing the 'root' definition So the new definition is simply * You are collectible if there are no references (persistent or transient) * You are a root if there are only transient references, which includes transient uppers and collectible_dependents The old definition expected roots to have no uppers at all, which isn't correct. A `run_once` task becomes an aggregation root and so the nested persistent tasks reference it as an upper. Those tasks should become gc roots since they are on the transient <--> persistent boundary and the only thing retaining them is references from transient tasks.
Part of a series of internal refactors to improve param tracking on the client. A route tree node used to come in two variants, a layout and a page, distinguished by an isPage flag that every constructor had to keep consistent with the VaryPath it built. The variants only differed in whether the vary path carried a search params entry, and the code that needed to know either checked the flag or read the vary path's shape positionally. Collapse them into one RouteTree type with one VaryPath type. The few places that still care whether a node is a page derive it from the segment. When something needs to read the search params, they can be accessed from the RouteTree or VaryPath like we do for other params. This is a pure refactor of the existing code. It reads more naturally with a single tree type and a single vary path type, which the rest of this stack builds on.
Refactors the CacheNode tree to use the RouteTree<T> data type. CacheNode used to be a tree-shaped object, with a similar structure to RouteTree<T>. CacheNode is now a data container instead, embedded inside a RouteTree structure. So the usage sites change from CacheNode to RouteTree<CacheNode>. As a result, we can now diff against the render tree directly during prefetches and navigations, instead of diffing against FlightRouterState. This gets us closer to migrating everything away from FlightRouterState to our shared data types.
Adds a `varyParams` field to CacheNode. When present, they represent the set of params that the data for that segment varies by. (When null, the set is unknown.) Currently, the vary params are only known when the data is fully cached; vary params are not recorded during a dynamic render. However, this will change in a future PR once we've integrated with the experimental Ledgers feature in React.
Throughout most of the client router implementation, the head is treated as if it belongs to the page segment. For example, CacheNode has separate `rsc` and `headRsc` fields, and the page's CacheNode owns both. This had led to a bunch of special cases related to the head. The Segment Cache largely (though not entirely) avoids this by instead modeling the head as a special kind of segment. It has its own vary path, its own SegmentCacheEntry, etc. There are still a few places where treatment of the head is special, because it doesn't exist inside the normal route tree structure. But we treat it like segment wherever possible. This PR updates the navigation implementation to use the same model: treat the head as a segment that is a sibling to the route segment tree. This is a large diff but the changes are almost entirely mechanical. Although the added/removed balance is about even, that includes a new test case. The net number of implementation lines has gone down.
Changing a page’s search params doesn’t change which page it is, but it may change the data that page needs. The router currently mixes these concerns by appending search params to `__PAGE__` segments. Remove that encoding so we can compare which route is being rendered separately from the param values used to render it. Param values are now compared through the existing VaryPath data structure, while route structure is compared during the existing tree traversals. Each traversal can decide which comparisons matter for the work it’s doing. Client-side comparisons and history restoration still need the search params, so store them in a separate FlightRouterState slot for now. As before, previous-page search params are stripped from normal request headers. This is a temporary step that lets us migrate incrementally. Eventually, FlightRouterState will be replaced by a type that represents the route and its params more directly. This is mostly a refactor, though it fixes an accidental inconsistency in search-param handling compared with regular route params, avoiding some redundant prefetch work. Fully cached pages already track their vary params correctly. This separation prepares us to do the same for partially dynamic segments during navigation, so changing an unrelated param won’t require rendering their dynamic content again.
The prefetch scheduler's static walk decided at the parent which child was still part of the current page and which began the new part of the route, and it fetched the head through two functions of its own that copied the per-segment decision (a static attempt, then a runtime deopt if that wasn't enough) for a node that hangs off the route root instead of sitting in the tree. Give the walk the same shape as the navigation's. The walk over the part of the route the current page also has owns the comparison for its own node — first whether the route position still matches, then whether any of its params changed — and hands off to the walk over the new part of the route when either fails, so the root enters the shared walk like every other node. The per-segment decision, a static attempt or a runtime deopt, lives in one function that the new-part walk calls for every segment, and the runtime request's walk gets the same split between iterating the children and deciding for one node. The head is a one-node route tree like any other segment now, so it goes through the same functions: the new-part static walk and the direct runtime fetch it already had. This is a pure restructuring and issues the same requests as before: a segment the navigation keeps is still prefetched at the ordinary static tier without a runtime deopt, the head is still fetched whenever anything is, and a chain of inlined segments is still dropped where a Shell-phase walk crosses from a kept node into the new part. A later change uses the shape to prefetch only what the navigation would fetch.
### What? Replace the Graphite-specific CI optimizer with a **read-only TypeScript gate** for PR stacks formed by ordinary GitHub base/head branch links. This PR includes the work previously reviewed in #99116 and #99122. ### Why? Rebasing a large stack starts expensive `build_and_test` jobs on every PR at once. Let the first three PRs and the top PR run immediately, but defer middle PRs until nearby CI provides a useful signal. A deferred PR must **not** appear mergeable just because its jobs have not started. ### How? - **Find the stack from branch relationships**, not GitHub stack metadata. Middle PRs poll their three nearest predecessors every **five minutes**. Any predecessor with a successful, current-head/current-base `thank you, next` check releases full CI; three unsuccessful results **fail the gate and required aggregate**. While waiting, the gate stays in progress. After five hours without a decision, it releases full CI rather than stranding the PR. - **Revalidate before deciding.** Refresh PR and check data before release/failure so rebases, retargeting and reruns cannot make a stale check decisive. Transient GitHub 5xx/429 errors retry; permanent errors or ambiguous topology start full CI rather than claiming CI passed. - **Keep polling cheap and read-only.** Reuse PR head/base data from `pulls.list` (11 → **8 REST reads** per ordinary waiting poll), while still checking *all three* predecessors. The bundled action is checked out at the current test-merge `${{ github.sha }}` without persisted credentials; forks bypass the checkout/gate and run full CI. The six-hour gate stays on `ubuntu-latest` because `ubuntu-slim` has a 15-minute limit. <details> <summary>Implementation and review tradeoffs</summary> - The local action bundles a pinned Octokit. Isolated Jest tests use fake timers and a real check-run response fixture. CI runs the existing Next.js lint/examples/externals/browser checks before action-specific install, typecheck, build and Jest; a generated-diff guard catches stale bundles. - `uses: ./...` requires action files already in the job workspace; it does not fetch them like `owner/repo@ref`. The sparse checkout fetches only `action.yml` and `dist/index.js`. Caller and reusable workflow grant read-only `checks`, `contents` and `pull-requests` permissions; the gate inherits no secrets. - Cheap-check graph splitting was deliberately deferred. Job-level concurrency does not enforce predecessor *success*; stopping at the first pending predecessor misses an older success. GraphQL batching needs a verified check-association projection and measured cost before replacing the current REST lookup. - When the child commits became reachable from this branch, GitHub automatically marked #99116 and #99122 merged; no PR merge operation was used. [Action README](https://github.com/vercel/next.js/blob/ci/branch-stack-polling-gate/.github/actions/pr-stack-ci-gate/README.md) documents the contract. </details> ### Verification - **Focused checks:** TypeScript, Prettier, deterministic bundle/diff checks and **24/24 Jest** passed. The final comment-only change in `src/gate.ts` produced a byte-identical bundle. The [current root run](https://github.com/vercel/next.js/actions/runs/35967795126) passed its gate, lint and required `thank you, next` on **attempt 2** (same SHA); attempt 1 failed an intermittent redbox test. - **Live six-PR test on the compiled action:** the first three and top passed the gate immediately. Middle #99097 and #99098 **waited**. On initial attempts, both **failed closed** when their three predecessors were unsuccessful, with no expensive jobs started. On rerun, #99097 **released on #99086's success** despite #99096 failing and #99095 still pending; full CI ran but failed unrelated tests. #99098 waited for *its own* three predecessors, then failed closed again. Gate release never marked full CI successful. - **Known limitation:** product-test failures—not the gate—left four immediate/released PRs with failed full CI on various attempts. All five test drafts remain open for reuse. Workflow reruns in this experiment were initiated externally; this agent's GitHub integration received 403 (`actions=write` required) when attempting a rerun. <details> <summary>Current compiled-action six-PR results, with workflow and gate logs (2026-09-24)</summary> Each draft is one Jest assertion commit over its signed parent; the cumulative focused suite passes **24/24**. Times below are UTC. “Required” refers to `thank you, next`, not the gate check. | Position | PR / signed head / workflow | Gate outcome | Required check | Expensive work | | :-- | :-- | :-- | :-- | :-- | | 1 | [#99086](https://github.com/vercel/next.js/actions/runs/35967795126) `85444c9e` | immediate pass | failed attempt 1; **passed attempt 2** | ran twice | | 2 | [#99095](https://github.com/vercel/next.js/actions/runs/35969902709) `bc19787a` | immediate pass | failed attempts 1 and 2 | ran twice | | 3 | [#99096](https://github.com/vercel/next.js/actions/runs/35970016367) `98bfab58` | immediate pass | failed | ran | | 4 | [#99097](https://github.com/vercel/next.js/actions/runs/35970109573) `e260bf96` | [attempt 1](https://github.com/vercel/next.js/actions/runs/35970109573/job/107537584401): waited → failed 08:02; [attempt 2](https://github.com/vercel/next.js/actions/runs/35970109573/job/107609256427): **released 11:19** | failed both attempts (attempt 2: product tests) | none on attempt 1; full CI ran on attempt 2 | | 5 | [#99098](https://github.com/vercel/next.js/actions/runs/35970206069) `01222d4a` | [attempt 1](https://github.com/vercel/next.js/actions/runs/35970206069/job/107537887530): waited → failed 08:03; [attempt 2](https://github.com/vercel/next.js/actions/runs/35970206069/job/107609367420): waited → failed 11:50 | failed both attempts | **never started** | | 6 (top) | [#99099](https://github.com/vercel/next.js/actions/runs/35970312588) `c6f3cf4a` | immediate pass | failed attempts 1 and 2 | ran twice | **Decision evidence:** On attempt 1, #99097 saw #99096/#99095/#99086 all unsuccessful, and #99098 saw #99097/#99096/#99095 all unsuccessful; their required checks failed and downstream jobs were skipped *because their gates failed*. After #99086's successful rerun, #99097 attempt 2 logged `#99096=unsuccessful, #99095=waiting, #99086=success`; its gate passed and [build-next actually ran and passed](https://github.com/vercel/next.js/actions/runs/35970109573/job/107609336752). #99098 could not use #99086 (outside its nearest-three window); it failed at 11:50 when #99097/#99096/#99095 had all finished without success. Completed failed gates do not restart automatically when a predecessor reruns. </details> <details> <summary>Unrelated CI failures and recovered flaky tests</summary> - #99086's attempt 1 failed `lazy-dynamic-imports › does not parse a dynamic import target before activation`: redbox source was `null` after all in-job retries ([job](https://github.com/vercel/next.js/actions/runs/35967795126/job/107530613870)). **The same-head workflow passed on attempt 2.** - #99095, #99096, #99097 attempt 2 and #99099 failed `instant-insights-tab-overlay › should wrap the mobile overlay header only when it does not fit`: observed header top difference **24**, expected **< 4**, after in-job retries ([example job](https://github.com/vercel/next.js/actions/runs/35969902709/job/107537341995)). #99096 also failed `turbopack-loader-file-dependencies › should update when a build dependency changes` (`build-one` instead of `build-two`, [job](https://github.com/vercel/next.js/actions/runs/35970016367/job/107537932737)). #99097 attempt 2 additionally failed the `lazy-dynamic-imports` redbox assertion in an experimental cache-components shard ([job](https://github.com/vercel/next.js/actions/runs/35970109573/job/107609732882)). These failures are not gate-test failures. - **Recovered in-job flakes:** #99096 `instant-validation/head-and-reporting › invalid - runtime prefetch - dynamic viewport blocks navigation` (redbox did not open; passed retry 1/2); #99095 `use-cache-without-experimental-flag › should recover from the build error if useCache flag is set` (Playwright execution context destroyed on navigation; passed retry 1/2); #99097 attempt 2 `lazy-dynamic-imports` in a different Turbopack-dev shard (passed retry 2/2), `instant-validation/suspense-boundaries` (redbox did not open; passed retry 2/2), and `instant-validation/head-and-reporting.partial-prefetching` (redbox did not open; passed retry 1/2). </details> <details> <summary>Earlier implementation smoke tests and CJS-gate stack runs</summary> - The [folded-code root head `89f0ca34` full run](https://github.com/vercel/next.js/actions/runs/35965443069) passed gate, build, lint (including **24/24 Jest**) and required `thank you, next`. The earlier [compiled-action child head `25647f00` run](https://github.com/vercel/next.js/actions/runs/35922700289) also passed full CI, proving same-SHA sparse checkout before the lint-order follow-up was folded into root. - **CJS-gate fail-closed run (base `ec0abf41`, 2026-09-23):** [#99086](https://github.com/vercel/next.js/actions/runs/35875541530), [#99095](https://github.com/vercel/next.js/actions/runs/35875598833), [#99096](https://github.com/vercel/next.js/actions/runs/35875651890) and top [#99099](https://github.com/vercel/next.js/actions/runs/35875952426) ran full CI and failed unrelated tests. Middle [#99097](https://github.com/vercel/next.js/actions/runs/35875701404/job/107231099534) failed its gate at 15:20 and [#99098](https://github.com/vercel/next.js/actions/runs/35875873698/job/107231827956) at 15:21; required checks failed and expensive jobs never started. A Turbopack source-map snapshot failure was subsequently fixed in [#99106](#99106). - **CJS-gate release run (base `8ff7f2bd`, 2026-09-23):** [#99086](https://github.com/vercel/next.js/actions/runs/35887913444) and [#99095](https://github.com/vercel/next.js/actions/runs/35887957948) passed full CI. [#99097](https://github.com/vercel/next.js/actions/runs/35888074168/job/107273254039) and [#99098](https://github.com/vercel/next.js/actions/runs/35888146874/job/107273514756) waited, then released at the next poll on #99095's green check while a nearer predecessor was unresolved. Their later full CI failed; top [#99099](https://github.com/vercel/next.js/actions/runs/35888202575) started immediately (later canceled). These precede the TypeScript action and are not claimed as evidence for its current head. </details> <!-- NEXT_JS_LLM --> <!-- fleet d2ae64f1-7106-4ba4-b88a-7785c19904c6 --> --------- Co-authored-by: vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com> Co-authored-by: Tobias Koppers <1365881+sokra@users.noreply.github.com>
When deleting a task, remove the reverse dependeny edges also. Generally this should be empty, but there can be tasks if those tasks are owned by a root that is pinned but not active. So in the rare case that such tasks are 'reactivated' this also dirties the task. This closes a hole where a task with a cell_dependency can reference a deleted task. Then new test demonstrates the bug, without the fix it will panic when collecting the second task
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )