Repository navigation
feat(gke-test): run e2e as a server cluster plus an in-cluster test Job - #34
Merged
Merged
Conversation
A repository can now have more than one node pool, one per role, so the
test client never takes CPU from the server it tests.
- pools.yaml: optional `pool` key (`<repo>` or `<repo>-<role>`). Terraform
keys pools by pool name; a row without `pool` keeps its old key, so
pool-keep does not change. New row: keep-client (e2-medium spot, 0..1).
- Shared workflow: new `test-job` input. The run applies the Job after
every rollout, waits for Complete or Failed, and puts the Job logs in the
run log and the evidence. `test-command` is now optional.
- run.sh checks, before any node starts:
- every pod selects a pool that pools.yaml gives to this repository;
- the cluster has that pool (catches a row that is not applied yet);
- a StatefulSet or Deployment with more than 1 replica spreads one
replica per node, and its replicas fit max_nodes;
- the test Job folder renders exactly one Job and only namespaced kinds.
- docs/standards/test-cluster.md: what runs where (unit on the runner, e2e
and perf on the cluster), the test Job rules, and the 6-hour job limit.
… namespace An operator can create a ClusterRole or ClusterRoleBinding at run time (lumen makes one auth-delegator binding per instance). It is not in the rendered manifests, so teardown left it on the cluster. Objects labelled faberline.dev/owner-namespace=<ns> are now deleted by teardown after the namespace, and by the reaper when <ns> no longer exists. The reaper's ClusterRole gets get/list/delete on clusterroles and clusterrolebindings.
chrischeng-c4
added a commit
that referenced
this pull request
Oct 15, 2026
…lient lumen moves its kind e2e (direct install and operator, 100k docs) to the shared cluster in the #34 shape: the server as a cluster on pool-lumen, and the checks as an in-cluster test Job on pool-lumen-client. pool-lumen holds the operator's raft StatefulSet, one replica per node (required hostname anti-affinity), up to 3 voters at 1 CPU / 4Gi, plus the 1-replica direct-install Deployment and the operator. pool-lumen-client runs the test Job (kubectl, curl, jq, python3) at 500m / 1Gi. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 15, 2026
… matrix runs A performance matrix calls the workflow once per cell, from one workflow run, with a client image that is not the server image. - test-image + test-image-name: set the client image in the test Job only. The name must differ from image-name, which render() sets last. - test-env: lines of KEY=VALUE, put in the ConfigMap gke-test-env in the test Job (made also when empty, so envFrom always works). - evidence-suffix: added to the artifact name, so two calls in one run do not collide.
chrischeng-c4
added a commit
that referenced
this pull request
Oct 15, 2026
…lient lumen moves its kind e2e (direct install and operator, 100k docs) to the shared cluster in the #34 shape: the server as a cluster on pool-lumen, and the checks as an in-cluster test Job on pool-lumen-client. pool-lumen holds the operator's raft StatefulSet, one replica per node (required hostname anti-affinity), up to 3 voters at 1 CPU / 4Gi, plus the 1-replica direct-install Deployment and the operator. pool-lumen-client runs the test Job (kubectl, curl, jq, python3) at 500m / 1Gi. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
4 of 5 tasks
chrischeng-c4
added a commit
that referenced
this pull request
Oct 15, 2026
…lient lumen moves its kind e2e (direct install and operator, 100k docs) to the shared cluster in the #34 shape: the server as a cluster on pool-lumen, and the checks as an in-cluster test Job on pool-lumen-client. pool-lumen holds the operator's raft StatefulSet, one replica per node (required hostname anti-affinity), up to 3 voters at 1 CPU / 4Gi, plus the 1-replica direct-install Deployment and the operator. pool-lumen-client runs the test Job (kubectl, curl, jq, python3) at 500m / 1Gi. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
chrischeng-c4
added a commit
that referenced
this pull request
Oct 15, 2026
…erf-client) (#33) * gke-test: add pool-lumen (e2-standard-4 Spot, max 3) and pool-lumen-client lumen moves its kind e2e (direct install and operator, 100k docs) to the shared cluster in the #34 shape: the server as a cluster on pool-lumen, and the checks as an in-cluster test Job on pool-lumen-client. pool-lumen holds the operator's raft StatefulSet, one replica per node (required hostname anti-affinity), up to 3 voters at 1 CPU / 4Gi, plus the 1-replica direct-install Deployment and the operator. pool-lumen-client runs the test Job (kubectl, curl, jq, python3) at 500m / 1Gi. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * gke-test: add pool-lumen-perf and pool-lumen-perf-client for lumen's perf matrix Both are non-Spot with max 1 node: e2-highmem-4 for one server cell (2.5 CPU, 16Gi) and e2-standard-4 for the load generator. The e2e client pool stays e2-medium Spot; its comment now says jq, not python3, builds the fixture. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Putting every test on one node does not test a cluster. e2e should deploy the
server the way a real cluster does, and send test load from a separate pod in
the cluster, not through
kubectl port-forwardfrom the runner.What
pools.yamlgets an optionalpoolkey:
<repo>or<repo>-<role>. Terraform refuses other names. New row:keep-client(e2-medium Spot, 0..1).test-jobinput: a kustomizefolder with one Job. The run applies it after every rollout, waits for
CompleteorFailed, and keeps its logs.test-commandis now optional.put one replica on each node, and its replicas must fit
max_nodes.run.shstops before any node starts when a pod selects apool that is not this repository's, or a pool that is not applied yet.
faberline.dev/owner-namespace=<ns>is deleted by teardown after thenamespace, and by the reaper when
<ns>is gone. The reaper may now deletethose two kinds.
test-image+test-image-name(client image inthe test Job only),
test-env(KEY=VALUE lines in the ConfigMapgke-test-env), andevidence-suffix(unique artifact name per call).docs/standards/test-cluster.md: what runs where, the test Jobrules, the 6-hour job limit.
Checks
shellcheck -x -S warning scripts/gke-test/*.shactionlint .github/workflows/gke-test.ymlterraform fmt -checkandterraform validateterraform plan: 1 to add (pool-keep-client), 1 to change (reaper ClusterRole), 0 to destroygke-test-envrendered; bad KEY=VALUE, same image name, half-set image, and TEST_ENV without a Job are each refusedterraform apply(needs the owner's consent)🤖 Generated with Claude Code