Consolidate Docker Compose into base + thin overrides

[Babissimo]

What & why

Consolidates the repo's four standalone Docker Compose files into one production base plus thin per-environment overrides, leaving a single container entrypoint and a single UI. Motivation: the four files were near-complete copies that drifted, and the old docker-compose.test.yml diverged from the deploy on three axes at once (a legacy start-test.sh entrypoint, the old tar1090 static HTML UI, and an 8s/x4 stress fleet), so the laptop stack was not a faithful mirror of production. The aim is a laptop stack that reproduces production's optimisation and configuration behaviour, which local Docker previously could not.

Result

• 4 compose files -> 3. docker-compose.yml stays the base (= production; a bare docker compose up is unchanged). docker-compose.staging.yml and docker-compose.test.yml become thin overrides layered on it (-f docker-compose.yml -f <override>.yml), so they inherit the base and cannot drift.
• 2 entrypoints -> 1. The NGINX_PROFILE selection logic moved from the legacy start-test.sh into start.sh; start-test.sh deleted.
• 2 UIs -> 1. The tar1090 static HTML UI (nginx-test.conf, COPY tar1090/html) is removed. The tar1090 JSON data format and API routes (tar1090_data, tar1090_aircraft/tar1090_receiver, aircraft.json generation) are untouched.
• The laptop stack is now docker compose -f docker-compose.yml -f docker-compose.test.yml up -d --build: production, minus certs, on port 8080, with RETINA_ENV=test auth.

Notable specifics

• FRAME_WORKERS=8 surfaced from the droplet's backend/.env into the base so it is visible in git and inheritable (staging overrides to 6).
• Staging thinned to deltas; the resolved docker compose config is byte-identical to the previous standalone file (verified before/after), so the staging->prod promotion gate behaviour is unchanged. The five deploy-staging CI invocations are prefixed with the base; the production deploy job is untouched. Two orphaned tar1090 clone steps removed from CI.
• deploy/deploy-test-network.sh retired (its test-network droplet no longer exists).
• The stress fleet profile was dropped deliberately; ad hoc load testing overrides FLEET_INTERVAL/FLEET_NODES.
• Safety hardening: start.sh now fails closed if NGINX_PROFILE names a profile with no matching nginx-<profile>.conf, instead of silently falling back to the baked production nginx config. Production itself runs with an empty profile and is unaffected.

Tests

• New shell unit test deploy/tests/test_nginx_profile.sh covers profile selection (staging/local/production) and the fail-closed case.
• Laptop stack verified with a live bring-up: React UI (not tar1090), 196/196 fleet nodes connected, non-empty /api/radar/nodes, clean teardown.
• docker build . succeeds after the Dockerfile trims; all three compose files render.
• No application code changed, so the full backend/frontend/dashboard suites run here in CI rather than locally.

Follow-on (not in this PR)

• A broader justfile up-recipe revamp (this PR made only the minimal change forced by the file deletion: removing the dead testmap profile).

🤖 Generated with Claude Code

[claude]

Claude finished @Babissimo's task in 6m 1s —— View job

---

Code Review

Tasks

• Gather context (PR diff, description, changed files)
• Read repo CLAUDE.md for conventions (none present)
• Review compose files (base/staging/test overrides)
• Review deploy/start.sh fail-closed logic and deploy/tests/test_nginx_profile.sh
• Review CI workflow and Dockerfile changes
• Review justfile changes
• Post final review feedback

---

Overall this is a clean, well-documented consolidation. The base/override split checks out: docker-compose.yml is untouched as the production config, and both docker-compose.staging.yml/docker-compose.test.yml use YAML merge tags (!override/!reset) correctly to strip only what each environment can't support (certs, retina-edge, pinned image names). I traced the fail-closed logic in deploy/start.sh (lines 45-60) against all four cases in deploy/tests/test_nginx_profile.sh and the branching is correct: empty profile keeps the baked prod config, staging/local copy their matching file, and any other named profile (e.g. test) now hits exit 1 instead of silently falling back to the production nginx config. No stale references to the deleted start-test.sh, docker-compose.local.yml, nginx-test.conf, or deploy-test-network.sh remain anywhere in the repo. All 5 deploy-staging compose invocations in .github/workflows/ci.yml are correctly prefixed with -f docker-compose.yml; the production job (lines 304-330) correctly stays on the bare docker compose (base file only).

One gap worth fixing:

deploy/tests/test_nginx_profile.sh is never invoked by CI. The PR description lists this as a new test covering the fail-closed nginx-profile selection, but grepping .github/workflows/ci.yml (the repo's only workflow) and the justfile shows nothing calls it — not in docker-build, not in backend-tests, nowhere. As written, it only runs if someone remembers to invoke it manually, so a future regression in the start.sh profile-selection block (including the new fail-closed safety check) won't be caught automatically. Suggest adding a step (e.g. in the docker-build job) that runs bash deploy/tests/test_nginx_profile.sh.

Fix this →

Minor/non-blocking notes:

• docker-compose.staging.yml still carries a hedge comment ("If Compose rejects an empty !override list, use env_file: !reset null instead") next to env_file: !override [] (around line 60). Since the PR description says the resolved staging config was verified byte-identical, it'd be worth confirming that syntax was actually exercised and dropping the hedge if so — otherwise it reads as an untested fallback left in a comment.
• docker-compose.test.yml's tower-finder service no longer sets an explicit container_name (the old docker-compose.test.yml had retina-test-server), so the laptop container now gets Compose's auto-generated name. Confirmed nothing else in the repo references the old name, so this is just a heads-up, not a bug.

[claude]

Claude finished @Babissimo's task in 1m 55s —— View job

---

Code Review — updated after latest push

Tasks

• Gather context (diff since previous review, PR description)
• Check what changed since the previous review round (commit 8ba88df)
• Verify previously-flagged issues
• Review changes for correctness
• Post final review feedback

---

The latest commit (8ba88df, "unify on the 'test' profile and resolve review findings") is a solid follow-up: it renames nginx-local.conf → nginx-test.conf and drops the redundant NGINX_PROFILE=local override, re-adds RADAR_TCP_PORT=3012 to the test override for a faithful fresh-clone default, documents the Compose ≥2.24 requirement for the !override/!reset merge tags, and trims verbose comments. I traced nginx-test.conf against the new naming (testmap.localhost / testapi.localhost) and it's internally consistent, and the justfile's testmap profile removal lines up cleanly with the deleted docker-compose.local.yml.

Previously flagged, still open:

• deploy/tests/test_nginx_profile.sh is still not wired into CI. Re-checked .github/workflows/ci.yml (git diff origin/main...HEAD -- .github/workflows/ci.yml) — this update only touched the tar1090 clone-step removal and prefixed the deploy-staging compose invocations with the base file; the docker-build job (lines 73-79) still just runs docker build, nothing invokes the new test script. This is the one substantive gap from the last review that wasn't addressed in the follow-up commit. Note: I can't edit .github/workflows/ci.yml myself (GitHub App permissions block workflow file changes), so this needs to be wired in by a maintainer or via the "Fix this" link below.

Fix this →

Previously flagged, minor/non-blocking, unchanged:

• docker-compose.staging.yml:61-62 still carries the hedge comment ("If Compose rejects an empty !override list, use env_file: !reset null instead") next to env_file: !override []. Since staging's resolved config was verified byte-identical, this untested fallback note could be dropped now that the syntax is confirmed working — purely cosmetic.
• docker-compose.test.yml's tower-finder service still has no explicit container_name (unlike fleet, staging's services, and prod's fleet), so it gets Compose's auto-generated name. Still just a heads-up — nothing else in the repo references the old retina-test-server name.

Nothing new introduced by this round of changes looks problematic — the fixes for the four naming/design questions and the RADAR_TCP_PORT gap are correct and well-reasoned. The only actionable item remaining is wiring the nginx-profile test into CI.

[Babissimo]

Follow-up commit (8ba88df) resolves the five review findings and folds in four naming/design refinements raised in review:

RETINA_ENV / NGINX_PROFILE unified on "test". Renamed nginx-local.conf → nginx-test.conf and dropped the explicit NGINX_PROFILE=local from the test override. The stack already runs RETINA_ENV=test, and start.sh's profile default (NGINX_PROFILE:-RETINA_ENV) now selects nginx-test.conf on its own, so the separate local name and the override were both redundant. One fewer knob, no local/test split.

Coupling: by default, not hard-wired. NGINX_PROFILE stays an overridable axis so nginx can be swapped without changing the app's auth environment (the reason the previous author split them). The unit test now covers the default mapping, the override case, and a fail-closed unknown profile.

Defaulting to prod is deliberate/fail-closed — unset RETINA_ENV gives strict backend auth and the baked prod nginx.conf; forgetting a var never downgrades security. No change.

Findings: re-added RADAR_TCP_PORT=3012 to the test override (app default is already 3012, belt-and-braces for a fresh clone with no backend/.env); recorded the Compose ≥ 2.24 requirement for the staging merge tags (staging droplet runs 2.40, verified); reworded the banned "not X, it is Y" comment; cross-referenced the duplicated retina-edge neutralisation; trimmed the verbose comments.

Verified: profile unit test passes; both merged configs render; staging resolved config byte-identical to HEAD (my staging edits are comment-only); fresh-clone render with no backend/.env succeeds. FRAME_WORKERS=8 in the base matches prod's .env on the droplet, so that surfacing is faithful (unchanged).

[claude]

Claude finished @Babissimo's task in 59s —— View job

---

Code Review — updated after latest push

Tasks

• Gather context (diff since previous review round, commit 8ba88df)
• Check what changed in the latest commit (69357f1)
• Verify previously-flagged issues (CI wiring, staging hedge comment, container_name)
• Review for correctness
• Post final review feedback

---

Commit 69357f1 ("ci: run the nginx-profile unit test; tidy two review notes") closes out all three items from the last two review rounds:

• CI wiring (the one substantive gap). .github/workflows/ci.yml:79-81 now runs bash deploy/tests/test_nginx_profile.sh as a step in the docker-build job, placed before docker build -t tower-finder:ci . for fast feedback, with a comment explaining it guards the fail-closed branch. I read deploy/tests/test_nginx_profile.sh end-to-end: it extracts the # >>> nginx-profile-select >>> / # <<< nginx-profile-select <<< block directly out of start.sh (so the test tracks the real code, not a copy), stubs nginx-staging.conf/nginx-test.conf in a temp dir, and asserts all five cases — staging-via-RETINA_ENV, test-via-RETINA_ENV, NGINX_PROFILE override, prod (nothing selected, baked default kept), and the fail-closed unknown-profile case (non-zero exit, baked default left untouched). The set -e / || rc=$? handling around the bash -c subshell is correct for capturing the expected failure without the outer script aborting early. This is exactly what was missing before — the gap is resolved.
• Staging hedge comment (minor/non-blocking, previously flagged). docker-compose.staging.yml:58-60 now drops the "if Compose rejects an empty !override list" hedge next to env_file: !override [], since the commit message confirms it was verified via docker compose config. Resolved as suggested.
• Missing container_name on test's tower-finder service (minor/non-blocking, previously flagged). docker-compose.test.yml:19 now restores container_name: retina-test-server, matching retina-test-fleet and staging's naming convention. Resolved as suggested.

No new issues introduced by this commit — the diff is small (3 files, +6/-2) and each change is a direct, correctly-scoped fix for a previously-raised point. I don't have a way to execute docker build/docker compose in this sandboxed environment to re-run the live verification, but the test script's logic was traced manually against all five cases and is correct, and the CI step placement is sound.

This PR is in good shape — all substantive and minor review feedback across both prior rounds has been addressed.

[Babissimo]

Thanks — addressed in 69357f1.

CI gap (the main one): added a nginx-profile selection unit test step to the docker-build job that runs bash deploy/tests/test_nginx_profile.sh, placed before the image build so a profile-selection regression (including the fail-closed branch) fails fast. Confirmed the workflow still parses and the test passes.

Staging hedge comment: dropped the "if Compose rejects an empty !override list…" note. env_file: !override [] renders cleanly via docker compose config, so the untested fallback is no longer warranted.

Test container_name: restored retina-test-server on the test tower-finder service, matching retina-test-fleet and the staging server's naming (it was dropped in the consolidation, leaving the container auto-named).

One note on the review's profile-selection trace: it predates the earlier follow-up commit (8ba88df), which renamed nginx-local.conf → nginx-test.conf and unified this stack on the test profile. So test now resolves to a real config rather than hitting fail-closed; the fail-closed path is still covered by the unit test via an unknown (bogus) profile, and it's that test which is now wired into CI.

[Babissimo]

Force-pushed a rewritten branch (was 69357f1, now dbceeda). Heads-up: history was rewritten, so any inline comments pinned to the old commits may detach. The net "Files changed" is unchanged in shape from the last push apart from the two items below; the rewrite only removes a rename round-trip from the commit history.

Two substantive changes since the last review:

1. Prod-safety fix — nginx profile decoupled from RETINA_ENV. Prod deliberately runs RETINA_ENV=test (admin-access-for-all while building). The previous start.sh derived the nginx profile from RETINA_ENV (NGINX_PROFILE=${NGINX_PROFILE:-${RETINA_ENV}}), so a prod deploy would have selected the cert-free laptop config and dropped prod's TLS listeners and real domains. start.sh now uses an explicit NGINX_PROFILE only: unset keeps the baked production nginx.conf; the staging/test overrides set it explicitly. Verified by simulation against the real deploy/ dir: prod (RETINA_ENV=test, no profile) keeps all 8 listen 443 blocks and the retina.fm domains; staging → nginx-staging.conf; laptop → nginx-local.conf. A unit-test case (RETINA_ENV=test, no profile → baked default) guards this.

2. Laptop nginx profile renamed back to local (nginx-local.conf, NGINX_PROFILE=local). "local" fits the plain-HTTP localhost transport better than "test"; RETINA_ENV=test still names the backend auth environment, which is a separate axis. This supersedes my two earlier comments on this PR that claimed RETINA_ENV=test could auto-select the nginx profile and that defaulting off RETINA_ENV was safe — that was wrong precisely because prod runs RETINA_ENV=test.

Also folded in (unchanged from the prior push, now in one commit): re-added RADAR_TCP_PORT=3012 to the test override, the Compose >= 2.24 note, container_name restore, staging hedge-comment removal, the "not X, it is Y" reword, and the CI step running the profile unit test.