Repository navigation
fix: e2e harness restores the stack, runs via uv - #222
Merged
Merged
Conversation
`make test-e2e`, `test-resilience` and `test-flows` run through `uv run --script`. The scripts declare PyYAML 6.0.3 and Playwright 1.63.0 inline (PEP 723), so a host needs uv and nothing else from PyPI. The suite reads the compose project before it stops anything, then on every way out (a failure and a SIGTERM included) starts each service that was running or had run to completion again, from the compose files, project and environment it came from. A last verdict fails when one comes back on a different config hash. `make test-e2e` no longer leaves only ClickHouse running. The data file goes to the ingest edge 16 requests at a time. The receiver answers once the loader has flushed the record, so one request at a time paid a flush per event, and 1000 events took 323 s. post.py deletes its three ingest rows once every claim has run, the console claim that reads them back included. A table still carrying the `_timestamp_optimized` projection refuses a lightweight DELETE with Code 344, so it falls back to an `ALTER TABLE ... DELETE` mutation, polled for up to 60 s and reported rather than failed.
A container an older e2e run started is labelled with that run's override under `.tmp/`, which the run deletes when it finishes. Naming it in the restore made compose refuse the whole `up`, so the next run failed to put that container back. A file that no longer exists is now left out and named in the log, and the config hash check still decides whether the container came back as it was.
dfe-engine depends on both brokers, each optional and gated by its own compose profile, so the restore's `--profile "*"` started both and the second failed on port 9092. The restore now asks compose for every service those files define and enables only the profiles of the services it starts, which is the set the original `make ci` enabled.
`scripts/test_e2e.py.lock` and `scripts/test_flows.py.lock` (from `uv lock --script`) record every package with a sha256 per file, the transitive greenlet, pyee and typing-extensions included. The make targets and the shebang run `uv run --locked`, so a lock that no longer matches its script stops the run instead of resolving something new. Renovate's pep723 manager is opt-in (`managerFilePatterns: []`) and has no artifact step, so it can move an inline pin but never rewrites the lock beside it. `make check-locks` runs `uv lock --check` on each script with uv 0.13.0, and CI runs it after the unit tests. A stdlib test asserts every script with inline metadata is pinned exactly, has a lock recording those pins, and is named by `check-locks`.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Four harness defects that turned up running
make test-e2eon the docker GA target. One PR, because they all sit in the same run.make test-e2e,test-resilienceandtest-flowsran bare systempython3, so a host without PyYAML or Playwright failed at setup. The search-to-rule test failed that way on the GA target. The scripts now declarepyyaml==6.0.3andplaywright==1.63.0inline (PEP 723) and the targets runuv run --script, so a host needs uv and Google Chrome, nothing else._timestamp_optimizedprojection refuses a lightweight DELETE with Code 344 on ClickHouse 26.3, and a query-levellightweight_mutation_projection_modedoes not lift it, so it falls back toALTER TABLE ... DELETE, polled for up to 60 s and reported rather than failed.Done when CI is green here and a full
make test-e2eon the GA target leaves the stack exactly as it found it.Heads up: #143 touches the same harness and already conflicts with main in 30 files. This branch adds no new conflicted file, but both change the test teardown in
scripts/test_e2e.py, so whichever lands second re-resolves it.