feat(infra): a stable cross-cluster address for the Core and its three surfaces - #417
Merged
Merged
Conversation
Core and Tracker are separate products in separate repositories that talk
over HTTP, so validating them in ONE cluster would hand them in-cluster
DNS and hide the defects that only appear across a network boundary. Two
clusters is the honest topology and already the de-facto one — the
Tracker's manifest deploys into its own `evolith-tracker` cluster.
What was missing was the link. Everything was reached with
`kubectl port-forward`, which is right inside one cluster and wrong
between two: it is a host process nothing in another cluster can point
at, and it dies on every rollout of the target — so redeploying the Core
breaks the Tracker and the symptom looks like a Tracker defect.
There are TWO addresses now, and they are not interchangeable:
FROM ANOTHER CLUSTER http://<cluster>-control-plane:30080
FROM THIS MACHINE http://localhost:30080
The cross-cluster one does not use the host mapping at all. Every kind
cluster joins the same `kind` Docker network, so the consumer's pod
reaches the node container by name and hits the NodePort directly. The
host mapping serves the other consumer — browser, curl, CLI — and is
bound to 127.0.0.1, which is also precisely why it cannot serve the first
case.
`host.docker.internal` was the first thing written here and it is WRONG:
it fails with `Could not resolve host` inside a kind pod, which resolves
through CoreDNS in the node and never sees the Docker Desktop entry. The
comments say so, because the wrong URL is the plausible one.
Measured on a throwaway pair of clusters with the real core-api image,
then torn down: a pod in a second cluster got `{"status":"OK"}` by name
and by IP, the host port answered, and both survived `rollout restart`.
`local-test.sh url` proves BOTH paths on demand rather than asserting
them — they fail independently, so one check would cover for the other.
It picks a RUNNING peer: the first version picked a stopped cluster,
reported the network as broken, and the real cause was a node container
dead for two days.
The port mapping is fixed at cluster creation and cannot be added later,
so `kind_create` says when the existing cluster predates the config
instead of leaving a port that was never bound to be discovered by a
refused connection.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… the probe three times
"Evolith Core and its interfaces" is REST, MCP and the agent runtime. The
first version gave a stable address to core-api alone, which would have
let a consumer reach one surface of three while the docs implied parity —
so mcp (30081) and agent-runtime (30082) now carry pinned node ports too,
and the kind config maps all three.
The probe that proves it needed fixing three times, each a different way
of reporting a defect that was not there:
- it picked a STOPPED peer cluster, failed, and blamed the Docker
network; the node container had been dead for two days;
- it matched `"status":"OK"` literally, so mcp and agent-runtime — which
answer with a bare `"status":"ok"` rather than the ADR-0073 envelope
core-api uses — were reported unreachable while serving fine;
- it used `kubectl run -i --rm`, which attaches AFTER creating the pod
and loses the stdout of a container that already exited. Two
consecutive runs disagreed about which surface was reachable.
All three are pinned in the comments, because each produced a confident
red that pointed at the wrong layer.
Measured, not asserted: three clean clusters' worth of runs with the real
images. Three consecutive `url` invocations agreed 3/3 on both paths, and
all six checks held after `rollout restart` of all three deployments.
Test clusters torn down; `evolith-cluster` untouched (it has been stopped
for 47h, exit 137).
Worth a look separately: the three surfaces disagree on the /health
shape. core-api returns the ADR-0073 envelope, the other two a bare
object. Not changed here — that is a contract question, not an infra one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
📊 Bilingual Coverage ImpactPR Changes
Repository Coverage
✅ Good: All EN changes have ES counterparts. Generated by GitHub Actions |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Core and Tracker are separate products in separate repositories that talk over HTTP. Validating them in one cluster would hand them in-cluster DNS and hide the defects that only appear across a network boundary — a malformed base URL, a missing header, a timeout, CORS. Two clusters is the honest topology and already the de-facto one: the Tracker's manifest deploys into its own
evolith-trackercluster.What was missing was the link. Everything was reached with
kubectl port-forward, which is right inside one cluster and wrong between two: it is a host process nothing in another cluster can point at, and it dies on every rollout of the target — so redeploying the Core breaks the Tracker and the symptom looks like a Tracker defect.Two addresses, not interchangeable
http://<cluster>-control-plane:3008xkindDocker network → NodePort. Does not use the host mappinghttp://localhost:3008xextraPortMappings, bound to 127.0.0.1Because the host mapping is on loopback it cannot serve the cross-cluster case — which is exactly why both exist.
host.docker.internaldoes not work and was the first thing written here: inside a kind pod it fails withCould not resolve host, because the pod resolves through CoreDNS in the node and never sees the Docker Desktop entry. The comments say so, since the wrong URL is the plausible one.All three surfaces, not just the Core
core-api30080 ·mcp30081 ·agent-runtime30082. A consumer that reaches one of three has not exercised the product, and the docs would have implied a parity never tested.The probe needed fixing three times
Each version produced a confident red pointing at the wrong layer, and each is now pinned in a comment:
"status":"OK"literally, so mcp and agent-runtime — which answer with a bare"status":"ok"rather than core-api's ADR-0073 envelope — were reported unreachable while serving fine.kubectl run -i --rm, which attaches after creating the pod and loses the stdout of a container that already exited. Two consecutive runs disagreed about which surface was reachable.Verification — measured, not asserted
Real images, throwaway cluster pairs, then torn down:
urlfrom this machineurlfrom a second kind clusterrollout restartof all threeci-runner.mjs governancegitleaks dir .Not done, stated rather than dropped
up/apps-uppath (Docker Desktop k8s, no kind) inherits the NodePort values and was not exercised.kind_createnow says when the existing cluster predates the config instead of leaving a never-bound port to be found as a refused connection./healthshape. That is a contract question, not an infra one, and is untouched here.🤖 Generated with Claude Code