Speed up the portfolio e2e job - #2364
Open
nicosampler wants to merge 3 commits into
Open
Conversation
…ev server The portfolio e2e job served the dApp through the Vite dev server, so every page the tests opened made the browser pull hundreds of unbundled modules that Vite transformed on demand. Measured on the transfers spec against the same LocalNet, switching to the production build cut the suite by 19%: each screen appeared in about half the time and navigations dropped from ~400ms to under 100ms, while the steps that go through the wallet gateway stayed flat. The build already happens in the CI build job; it just was not part of the artifact the e2e jobs download, hence the extra path. The preview server gets its port from the Vite config so it does not have to be repeated here. Signed-off-by: nicosampler <nf.dominguez.87@gmail.com>
The suite ran one test at a time, and since most of its wall clock is spent waiting on the ledger rather than using CPU, that wait was pure dead time. Running two workers takes it from 11.6 to 6.5 minutes locally, with the per-step latencies unchanged, so the concurrency is essentially free. What kept it serial was shared state: the gateway scopes wallets and the primary wallet per user, and every test connected with the client id from the gateway's network config, so two tests at once would fight over which wallet is primary. Each worker now connects as its own user, derived from the Playwright parallel index, which is why the `serial` describes can go. That user id also travels to the participant as the token subject, and Canton rejects subjects it does not know, so the global setup creates the workers' users up front, through LedgerClient. It also takes over funding the validator operator, which two specs each did in a `beforeAll`: with several workers those taps would otherwise run at the same time and contend over the same contracts. Note this now runs on every invocation, including single-spec runs that do not need it. Four workers is worse than two: the ledger wait doubles and flakes go from one to three, so `workers` stays at 2 and takes PW_WORKERS to experiment. Note the CI runner has half the cores of the machine these numbers come from, so expect a smaller gain there. Signed-off-by: nicosampler <nf.dominguez.87@gmail.com>
Every test built its scenario by driving the wallet gateway's web UI: creating parties and picking the primary wallet each reloaded its parties page, 78 times across the suite at ~1.5s each. None of that is what these tests assert, which is the dApp. They are the same JSON-RPC calls that UI makes, so tests now make them directly through the generated User API client, and keep the UI for approving transactions, which is the integration under test. Measured on the transfers spec, scaffolding went from 79s to 37s. The suite total moves less than that, because creating a party costs 3.3s over the API against 5.5s through the UI: the difference is the page load, the rest is the ledger allocating the party either way. Two things that are not obvious from the API: - `selfSignedAccessToken` mints a token without a `sub` unless it is given a `clientId`, and the gateway then rejects that token itself. - Sessions are keyed by user *and origin*, and a new one replaces the old, so the test session uses its own origin. Sharing the dApp's would have the dApp's connect kill it and every later call come back 401. Trade-off worth naming: the wallet's parties page used to be exercised incidentally by these tests and no longer is. A smoke test for it is pending. Signed-off-by: nicosampler <nf.dominguez.87@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #2363.
portfolio-e2etakes 21 minutes and runs twice per PR, so it sets the wall clockfor the whole run. Three independent commits:
b77d857 serves the dApp from the production build instead of the Vite dev server.
In dev the browser pulls hundreds of unbundled modules per page, which shows up on
every screen the tests open; from the build, each one loads in about half the time.
The build already happened in CI, it just was not in the artifact the e2e jobs
download.
4343699 runs the tests on two workers. They mostly wait on the ledger rather than
use CPU, so overlapping them is nearly free: a full local run went from 11.6 to 6.5
minutes with per-step latencies unchanged.
What kept them serial was shared state: the gateway scopes wallets, and which one is
primary, per user, and every test connected as the same user. Each worker now
connects as its own, derived from the Playwright parallel index. That id also travels
to the participant as the token subject, and Canton rejects subjects it does not
know, so a new global setup creates those users first.
fefe891 changes how the tests set up their wallets. They used to create parties and
pick the primary wallet by clicking through the gateway's web UI, which meant
reloading its parties page 78 times across the suite, about 1.5s each, for something
no test actually asserts. The helpers now call the gateway's User API directly, the
same JSON-RPC methods that UI uses under the hood. Approving transactions still goes
through the UI, since that is the integration under test. On the transfers spec,
setup went from 79s to 37s.
The numbers come from a dev machine with 8 cores. CI runners have 4, and Canton plus
splice take about 2 before any test starts, so expect less there. Four workers was
already worse than two locally.