Run production dataplane tests and simplify harness architecture - #14
Open
lucarlig wants to merge 6 commits into
Open
Run production dataplane tests and simplify harness architecture#14lucarlig wants to merge 6 commits into
lucarlig wants to merge 6 commits into
Conversation
Signed-off-by: lucarlig <luca.carlig@ibm.com>
Signed-off-by: lucarlig <luca.carlig@ibm.com>
Signed-off-by: lucarlig <luca.carlig@ibm.com>
Signed-off-by: lucarlig <luca.carlig@ibm.com>
Signed-off-by: lucarlig <luca.carlig@ibm.com>
Signed-off-by: lucarlig <luca.carlig@ibm.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Standalone workflows run against production dataplane images without
with_tools. The harness owns ephemeral RSA signing, loopback JWKS, fixture discovery and MessagePack configuration publishing. Official conformance and Inspector install their npm packages in a Docker image and run there; the CLI no longer needs host Node/npm or a local npm cache.The architecture review covered CLI/configuration, checkouts and assets, Docker routing/lifecycle, MCP and Locust adapters, conformance, and CI/release orchestration. Changes:
CommandSpecdirectly instead ofStackCommandPlan, remove unused MCP transport features, and consolidate conformance artifact validation, baseline evaluation and reporting. Share asynchronous process handling and the CI/release quality workflow.Public routing, published-image defaults, managed cleanup and transactional baseline updates are preserved. Rust dependencies replace the removed npm helper dependencies; the cryptographic backend was already used by the HTTP stack.
Validation at
957eeb1:7f1961c: a fresh ARM64 production release image and the exact Linux x86-64 CI artifact both pass the production health/auth boundary checks and authenticated probe. Load smoke completes 42 and 27 requests respectively, with zero failures.ci prepare-image. Each image completes strictmake conformanceusing dataplane-owned baselines: 44 passed, 40 expected failures, no unexpected failures/passes, skips or unknowns. All 311 checks match between images. Both runs leave host Node/npm untouched and preserve both repositories' baselines.-D warnings, formatting, package verification and Actionlint pass. Cargo audit reports zero vulnerabilities and no warnings.with_tools.node,npmandnpxreplaced by failing traps, all 84 modern conformance scenarios complete. No trap is invoked and no host npm cache is created.a44a505: 148 direct-fixture checks, 124 external server checks and 39 client checks. Server scenarios total 41 passed, 16 expected failures and 23 failures; client scenarios total 3 passed and 1 expected failure. The same existing local-image baseline mismatches cause exit 1. Baselines are unchanged.Inspector's pinned upstream release uses initialization; its documented invocation selects the legacy protocol. Containerization does not add modern Inspector support.