Skip to content

dev-environment.md

Latest commit

 

History

History
218 lines (182 loc) · 12 KB

dev-environment.md

File metadata and controls

218 lines (182 loc) · 12 KB
Raw

Dev environment: bootstrap and restore

How to stand up the complete platform-lite dev infrastructure on OVH Public Cloud (region waw) — from an empty project to a working pnpm dev. Executing this document verbatim is the drill required by G10 (SPEC.md §7); after any real run, correct the document wherever reality disagreed with it.

Target state (SPEC.md §8):

  • one d2-2 instance (~€7/month) running Docker with PostgreSQL 17 bound to 127.0.0.1 — reachable only through an SSH tunnel, never exposed publicly
  • object storage bucket platform-dev (S3 endpoint https://s3.waw.io.cloud.ovh.net)
  • per-developer databases platform_<handle> and platform_test_<handle> (SPEC.md §4)

Part 1 — manual console steps (one-time, ~15 minutes)

These cannot be scripted from zero (they create the credentials everything else uses) or they touch payment:

  1. Public Cloud project — OVH Control Panel → Public Cloud → create a project (requires a payment method).
  2. OpenStack user (drives the openstack CLI) — in the project: Project Management → Users & Roles → create a user with the Administrator role (simplest for a solo dev project) → download the OpenRC file for region WAW1. Store it outside the repo, e.g. ~/.ovh/openrc.sh — it is a credential.
  3. S3 credentials — Object Storage → the users list → Create user → select the OpenStack user from step 2 (it needs the Administrator or ObjectStore operator role; being one is a precondition for the keys, not the keys themselves). The access key and secret key are shown once; afterwards the access key stays visible in the user's row and the secret is behind the row's “…” menu → View the secret key. Corrected 04.09.2026 against the console and the OVHcloud documentation: an earlier version of this document pointed at a separate “S3 users” section, which is not where the console puts it. Equivalent without the console, once the OpenRC file works: openstack ec2 credentials list (or create if the list is empty) — OVH stores these in Keystone.

Nothing from this part ever goes into the repo or the chat: credentials live in the OpenRC file and in your .env (gitignored).

Part 2 — scripted bootstrap

Run from the repo root in bash (Git Bash on Windows). One-time tools — install them into their own virtual environment, not the system Python: on Windows a pip install into C:\Python310 fails part-way with a locked-file OSError, leaving the CLIs half-installed (seen 04.09.2026).

python -m venv ~/.venvs/ovh
~/.venvs/ovh/Scripts/python.exe -m pip install python-openstackclient awscli
export PATH="$HOME/.venvs/ovh/Scripts:$PATH"   # every shell that runs the bootstrap

source ~/.ovh/openrc.sh                            # OpenStack credentials
export AWS_ACCESS_KEY_ID=<s3 access key>
export AWS_SECRET_ACCESS_KEY=<s3 secret key>
export GITHUB_HANDLE=<your-github-handle>
export POSTGRES_PASSWORD=$(openssl rand -hex 32)   # hex only — safe for the template
echo "$POSTGRES_PASSWORD"                          # store it in your password manager
./scripts/bootstrap-dev.sh

The script reuses the keypair and security group if they already exist, and is loud about the one expensive step: creating the instance starts the ~€7/month billing. What it does:

  1. verifies the d2-2 flavor and the Ubuntu 24.04 image exist in the region,
  2. uploads your SSH public key as keypair platform-dev,
  3. creates security group platform-dev-ssh — inbound TCP 22 only — if the project can. A Public Cloud project may carry a security_groups quota of 0, against OVH's documented default of 100; this one did on 04.09.2026, and the bootstrap failed on it with a misleading 409 Quota exceeded. Raising the quota is a manually processed support ticket (Control Panel → Quota & Regions → Increase your quota!), so the step is optional: the script says loudly when it skips, the instance boots into default, and the host firewall below carries the job alone. Re-run once the quota clears and the group is added,
  4. renders scripts/cloud-init.yaml.tmpl (ufw denying every inbound port but 22, Docker, postgres:17 on 127.0.0.1, the two per-developer databases) and boots instance platform-dev with it,
  5. creates the platform-dev bucket,
  6. writes the bucket lifecycle rule expire-staged-uploads: objects under <prefix>staging/ expire after 1 day. Abandoned avatar uploads (#12) leave objects with no files row, invisible to the A9 quota. The rule is a backstop only — S3 expiration is expressed in whole days while the presign TTL is 120 s, so issue #30 makes the application itself account for and sweep staged bytes. The script refuses to overwrite lifecycle rules it did not write; SKIP_LIFECYCLE=1 opts out,
  7. prints the exact values for your .env.

How a photo becomes publicly readable

Two OVHcloud facts, both found the hard way against the real bucket on 04.09.2026:

  • Path-style addresses are not served to the public. An anonymous GET https://s3.waw.io.cloud.ovh.net/platform-dev/<key> is refused with InvalidRequest / Reason: Not S3 request, while the identical address works when signed. Public addresses therefore use the virtual-host form https://platform-dev.s3.waw.io.cloud.ovh.net/<key>, which is what storage.publicUrl emits; signed traffic keeps path style.
  • The bucket needs a CORS rule for uploads. The browser PUTs straight to a presigned URL (G4), which is cross-origin: with no CORS configuration the preflight is refused with 403 and the upload never leaves the page, while every server-side call keeps working. The signature, not the origin, authorizes the write, so the dev bucket accepts any origin — developer machines, PR previews and phones on the LAN all differ. Production (#24) should narrow it to its own domain — for GET as well as PUT since #101: the rule also allows GET and exposes Content-Range, Accept-Ranges, Content-Length and ETag. The R360 reader (A13) fetches single frames out of an archive already in the bucket through a presigned GET with a Range header (a plain bytes=N-M range is CORS-safelisted, so no preflight runs — the method has to be granted for the answer to carry Allow-Origin, and Content-Range has to be exposed for the script to learn the archive's size). The widening moves no confidentiality line: the bucket authorises by the per-object ACL and the signature, never by the origin. A bucket created before 09.09.2026 needs the rule re-applied — re-run the CORS step of scripts/bootstrap-dev.sh or aws s3api put-bucket-cors with the same JSON.
  • There are no bucket policies. PutBucketPolicy answers NotImplemented, so public access is a per-object ACL. putObject takes an explicit publicRead flag, and only the 512/128 variants get it — the full-resolution original and every staged upload stay private. Nothing renders the original, and a public address for it would be derivable from any variant URL.

Where the inbound boundary lives

ufw on the instance is the enforcing layer: default deny inbound, port 22 open, configured by cloud-init before Docker starts. The OpenStack security group is a second, network-level layer on top — valuable, but not what the safety rests on.

That ordering is deliberate rather than a workaround. The only service listening publicly on this instance is sshd: PostgreSQL is published on 127.0.0.1:5432, so it is unreachable from the network with or without a firewall. Note that ufw does not filter ports Docker publishes on 0.0.0.0 — Docker writes its own iptables rules — which is precisely why the database is pinned to loopback rather than trusted to the firewall.

Part 3 — verify (this is the drill)

  1. cp .env.example .env (if you have none) and fill in the printed values.

  2. pnpm db:tunnel in a separate terminal — uses DEV_SSH_HOST from .env and maps localhost:5433 to the instance's 127.0.0.1:5432.

  3. Databases exist (cloud-init needs 2–3 minutes after boot — retry before assuming failure). Without a local psql, over SSH:

    ssh <DEV_SSH_HOST> sudo docker exec postgres psql -U postgres -c '\l'

    With a local psql, through the tunnel (this also proves the exact path the app will use): psql "<DATABASE_URL>" -c '\l'. Both platform_<handle> and platform_test_<handle> must be listed.

  4. aws --endpoint-url https://s3.waw.io.cloud.ovh.net s3 ls s3://platform-dev — empty listing, no error.

  5. pnpm db:migrate, then pnpm db:seed — both go through the tunnel; the seed reports 14 created profiles and prints the password. pnpm dev then renders http://localhost:3000; sign in with a seed account (<handle>@seed.example) and the settings pages show the seeded profile (the public page arrives with #18).

Restore after a disaster (G10 direction)

Dev data is disposable: on a fresh database, pnpm db:migrate && pnpm db:seed recreates it (G7); the seed adds, it never resets. Restore = delete and re-run with the same inputs, then rebuild the data through the tunnel (Part 3, steps 1–2):

openstack server delete platform-dev
./scripts/bootstrap-dev.sh
pnpm db:migrate
pnpm db:seed

The bucket and its objects live independently of the instance and survive its loss. Prod restore (managed database, snapshots) is defined by task #24 and will extend this document.

The databases on the instance

The cluster holds more than the two databases cloud-init made:

Database What it is
platform_<handle> dev's own, and what a developer's tunnel points at
platform_test_<handle> the integration suite's (SPEC §6)
platform_pr_<n> one per open pull request preview (#113)

A preview's database is a pg_dump copy of dev taken when the preview starts, migrated by the pull request's own image, and dropped by preview-down.sh when the pull request closes. Two consequences worth knowing before you look at one:

  • A preview shows dev's data as of its start. Anything added to dev afterwards is not there, and anything added in the preview — an account, a work — exists only there and goes away with it. The objects those uploads put in the bucket do NOT: they stay under pr-<n>/ with nothing naming them (the same tail as #34 and #111).

  • You have to sign in to a preview. Its copy is restored with sessions and verifications emptied: a session copied out of dev would stay valid in the copy after it was revoked on dev, and nothing could reach in to end it.

  • A preview cannot delete dev's objects, even though its rows name them: src/lib/storage.ts refuses a delete whose key is outside the environment's own S3_PREFIX and logs what it refused.

  • A copy can be left behind if a preview is removed some other way than preview-down.sh — a cancelled CI run, or an older branch whose preview-down.sh predates #113. The next preview-up.sh drops every copy with no container of its own, so this heals on the next preview; to see or do it by hand:

    ssh <DEV_SSH_HOST> docker exec postgres psql -U postgres -Atc "select datname from pg_database where datname like 'platform\_pr\_%'"
ssh <DEV_SSH_HOST> docker exec postgres psql -U postgres -c "drop database if exists platform_pr_<n> with (force)"

Decisions behind this setup

  • SSH tunnel instead of an allow-listed IP (30.08.2026) — survives home-IP rotation, and the database is never internet-facing; even a security-group mistake exposes nothing, because Postgres binds to 127.0.0.1.
  • No Terraform — consciously out of scope (SPEC.md §11); one instance and one bucket do not justify the tooling or its exit cost.
  • Manual part kept minimal — only project creation, payment and initial credentials; everything repeatable lives in scripts/ so the restore drill stays honest.