Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 25 additions & 5 deletions .github/workflows/run_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@ on:
paths:
- 'directus-cms/extensions/directus-extension-programmierbar-bundle/**'
- 'nuxt-app/**'
# The extension bundle imports from shared-code/, so a change there can break its build or
# tests. Without this line those changes skip the gate entirely.
- 'shared-code/**'
- '.github/workflows/run_tests.yml'

jobs:
Expand All @@ -17,26 +20,43 @@ jobs:
- name: Checkout repository
uses: actions/checkout@v4

# Deliberately still 22 while nuxt-app moves to 24: this tree has its own lockfile, its own
# Jest setup, and is frozen pending the Directus licence clarification. Moving its runtime
# would be an untested change to a tree nobody is allowed to upgrade.
- name: Use Node.js 22.x
# Reads the bundle's own .nvmrc rather than naming a version here, so CI, the Docker image and
# local development have one file to change instead of three.
#
# Deliberately still 22 while nuxt-app is on 24 — but *not* because of the Directus licence
# block, which an earlier version of this comment claimed. Every @directus/* package this
# bundle uses is MIT; only the `directus` server package is licence-blocked, and it lives in
# directus-cms/package.json. The real reason is Dockerfile.directus: Node 24 brings npm 11,
# which gates the install scripts this tree's native dependencies need.
- name: Use Node.js
uses: actions/setup-node@v4
with:
node-version: 22
node-version-file: directus-cms/extensions/directus-extension-programmierbar-bundle/.nvmrc
cache: 'npm'
cache-dependency-path: 'directus-cms/extensions/directus-extension-programmierbar-bundle/package-lock.json'

# Keep `npm ci`, not `npm install`, so the lockfile stays authoritative.
- name: Install dependencies
run: npm ci

# Keep `prettier:check`, not `prettier` — the latter writes, so it would pass by mutating.
# Without this step formatting is voluntary, which is how 64 files drifted here unnoticed.
- name: Check formatting
run: npm run prettier:check

# Keep `lint`, not `eslint` — the latter runs with --fix, so it mutates instead of failing.
- name: Run ESLint
run: npm run lint

- name: Run tests
run: npm test

# Nothing above confirms the bundle still compiles: `directus-extension build` has its own
# Rollup/esbuild pipeline that neither ESLint nor Jest exercises, so a change that breaks the
# actual artefact Directus loads could otherwise merge green.
- name: Build
run: npm run build

nuxt-app-test:
runs-on: ubuntu-latest
defaults:
Expand Down
23 changes: 16 additions & 7 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,9 +66,15 @@ npm run migrate:db # Database migrations

```bash
# In directus-cms/extensions/directus-extension-programmierbar-bundle/
npm test # Run Jest tests
npm run prettier:check # Formatting — fails, never rewrites
npm run lint # ESLint over .ts and the one .vue file
npm test # Jest
npm run build # directus-extension build
```

That is the full CI gate, in order. There is deliberately no typecheck step yet — `tsc` cannot
currently run on that tree at all. See [the tooling plan](docs/directus-extension-tooling-plan.md).

## Code Principles

### Consolidation & DRY
Expand Down Expand Up @@ -114,16 +120,19 @@ Additional hints can be found in:

### Formatting is enforced, not requested

CI runs `npm run prettier:check` in `nuxt-app`, so unformatted code fails the build. Nobody should be
expected to remember the formatter — turn on **format on save** and it never comes up:
CI runs `npm run prettier:check` in **both** `nuxt-app` and the Directus extension bundle, so
unformatted code fails the build. Nobody should be expected to remember the formatter — turn on
**format on save** and it never comes up:

- **WebStorm**: Settings → Languages & Frameworks → JavaScript → Prettier → *On save*
- **VS Code**: the Prettier extension, plus `"editor.formatOnSave": true`

Both read `nuxt-app/.prettierrc` on their own, and `nuxt-app/.editorconfig` covers indentation and line
endings before that is set up. Keep those two in step — they overlap, and an editor that indents to a
different width than Prettier produces a diff on every save. If a PR fails the check, `npm run prettier`
fixes it — never hand-edit to satisfy it.
Editors pick up whichever `.prettierrc` is nearest the file, and each gated tree ships one alongside an
`.editorconfig` that covers indentation and line endings before the formatter is set up. Keep each pair
in step — they overlap, and an editor that indents to a different width than Prettier produces a diff on
every save. The two configs are identical apart from `nuxt-app`'s Tailwind class-sorting plugin, so the
same editor setup works in both. If a PR fails the check, `npm run prettier` fixes it — never hand-edit
to satisfy it.

Note that `npm run lint` (ESLint) needs `nuxt prepare` to have run first, since `eslint.config.mjs`
extends the generated `.nuxt/eslint.config.mjs`. `npm ci` does this via `postinstall`.
Expand Down
15 changes: 12 additions & 3 deletions Dockerfile.directus
Original file line number Diff line number Diff line change
@@ -1,4 +1,10 @@
# Choose a base image
#
# Keep this in step with `directus-cms/extensions/directus-extension-programmierbar-bundle/.nvmrc`,
# which CI reads — the image and CI must build the extension on the same Node, or CI stops being
# evidence about production. Moving to 24 is deliberately deferred: Node 24 ships npm 11, which
# gates dependency install scripts, and this tree needs them to run (sharp, sqlite3, isolated-vm,
# esbuild all build native code on install). See docs/directus-extension-tooling-plan.md.
FROM node:22

# Set working directory
Expand All @@ -9,7 +15,10 @@ COPY directus-cms/package.json .
COPY directus-cms/package-lock.json .

# Install dependencies
RUN npm install
#
# `npm ci`, not `npm install`: the latter is free to resolve versions the lockfile does not record,
# so the image could ship dependencies CI never tested. `npm ci` fails instead, which is the point.
RUN npm ci

# Copy the shared code that lives outside of directus dir
COPY shared-code ../shared-code
Expand All @@ -20,8 +29,8 @@ COPY directus-cms .
# Set working directory to interface extension
WORKDIR /usr/src/app/directus-cms/extensions/directus-extension-programmierbar-bundle

# Install publishable interface extension dependencies
RUN npm install
# Install extension bundle dependencies (its own lockfile, copied in by `COPY directus-cms .` above)
RUN npm ci

# Build the extension
RUN npm run build
Expand Down
140 changes: 140 additions & 0 deletions _ADRs/0003-three-tier-extension-testing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
# ADR 0003: Three-tier testing for the Directus extension bundle

- **Status:** Accepted
- **Date:** 2026-08-05
- **Scope:** `directus-cms/extensions/directus-extension-programmierbar-bundle`
- **Relationship to other ADRs:** **complements** [ADR 0001](0001-jest-runs-in-cjs-mode.md) — its
`util/`-extraction guidance is Tier 1 here and still stands. Supersedes nothing. Gives
[ADR 0002](0002-batch-updates-use-updateone.md)'s unenforced convention a place to be enforced.

## Context

At the time of writing, 9 of the bundle's 26 entries have any test, and the ones that do follow a
single pattern: extract business logic into a `util/` module with no framework imports, unit-test
that. `post-to-discord` is the model — it tests `buildNewsEmbed` for URL construction, brand colour
and slug fallback, and touches neither the hook registration nor the HTTP call.

That pattern is good and cheap, and it has a structural blind spot: **it cannot see the hook's
contract with Directus.** Every bug this bundle is known to have shipped lives in exactly that blind
spot:

- **ADR 0002's bug.** `ItemsService.updateMany` fires a *single* action carrying `metadata.keys[]`.
Downstream hooks read `metadata.key || metadata.keys[0]`, so a cascade of five items reindexed only
the first. The other four were written to the database correctly and silently never reached Algolia.
Nothing threw.
- **The buzzsprout-class crash** referenced in ADR 0001: an un-awaited promise in a hook handler
taking down the CMS. The `no-floating-promises` lint rule now catches that specific shape, but not
the general class.
- The two lifecycle traps in [the conventions doc](../.claude/rules/directus-conventions.md): whether
a hook fires when an item is *created* already in the triggering state, and whether a guard prevents
it firing twice.

None of these are reachable from a pure function. So the gap is not "17 entries are untested" — it is
that **nothing tests the hook-to-Directus contract, including for the 9 entries that are tested.**

A real-Directus E2E suite is the obvious answer and the wrong first answer: it is slow, it flakes, and
writing one scenario per hook would put a multi-minute, network-shaped suite on the PR gate.

## Decision

Three tiers, with explicit ownership of what each is for.

### Tier 1 — unit tests on extracted pure functions

Unchanged from ADR 0001. Logic moves into `util/`, gets tested without mocks. Cheapest tests in the
tree; keep writing them, and keep extracting to make them possible.

**Owns:** decisions, payload shapes, formatting, parsing, validation.

### Tier 2 — hook contract tests

Run the **real hook module** against a **fake Directus context**. Import the hook's default export,
pass a hand-built `{ services: { ItemsService }, getSchema, env, logger }`, capture the registered
`action`/`filter` callbacks and invoke them directly. No Directus process, no network, milliseconds.

**Owns:** that the hook registers for the right events; that the create path and the update path both
behave; that guards prevent double execution; that the **ADR 0002 contract** holds — assert per-item
`updateOne` calls rather than one `updateMany`; that failures reach the notification path.

This is not a new invention. Five test files already do it — `cascade-publish`, `create-news`,
`fetch-open-graph`, `newsletter-double-opt-in`, `schedule-publication`. The decision is to make it the
**default for every entry**, and to consolidate the per-file fake contexts into one shared helper as
the sixth is written.

### Tier 3 — E2E against a real Directus, deliberately thin

The **built bundle** loaded into a **real Directus 11.17.4**. Roughly five scenarios, not
twenty-six.

**Owns only what Tier 2 fakes:** that the bundle *loads at all* (23 hooks + 2 endpoints register);
that a real `ItemsService` write fires a real action; that the `publishable` interface renders in the
admin UI. It is also the only thing that can validate `@directus/extensions-sdk` 18 against the
frozen host, which is why it must exist before that upgrade.

**Not on the PR gate.** Same reasoning as `nuxt-app`'s `smoke_tests.yml`: a harness with a real server
and a real database will flake, and a flaky *required* check trains people to ignore red. Schedule it,
run it on demand, and promote it only once it has proven quiet.

## External services in Tier 3: redirect, do not mock

In Tier 3 the hook runs inside a separate Directus process, so module mocking is unavailable — there
is no `vi.mock` across a process boundary. The only lever is **configuration**: point every outbound
base URL at one local stub server.

Audited 2026-08-05, all 11 outbound integrations:

| Already redirectable via env | Hardcoded — needs a code change |
| --- | --- |
| Buzzsprout (`BUZZSPROUT_API_URL`) | Gemini — `GEMINI_API_BASE` module const |
| Browserless (`BROWSERLESS_API_URL`) | Bluesky — `BSKY_SERVICE = 'https://bsky.social'` |
| Deepgram (`DEEPGRAM_API_URL`) | Slack — `new WebClient(token)` at module load |
| Discord (`DISCORD_WEBHOOK_URL`) | Algolia — host derived from `ALGOLIA_APP_ID` |
| Vercel (`VERCEL_DEPLOY_WEBHOOK_URL`) | Wallet — `walletobjects.`/`oauth2.googleapis.com` |
| Mastodon (`MASTODON_INSTANCE_URL`) | |

Six already work. The remaining five must become configurable — which
[AGENTS.md](../AGENTS.md) already requires under "No Hidden Behavior" (*"No hardcoded defaults buried
in business logic … If it affects behavior, it must be visible and configurable"*), so that refactor
is justified independently of testing.

Two rules for the harness:

1. **Fail closed.** The stub returns 500 for any unrecognised path and fails the run, so a missed
redirect is loud.
2. **No secrets in the environment at all** — not fake-but-plausible values. A missed redirect must
die on a connection error, never reach the production Slack workspace.

**Explicitly excluded:** hitting real third-party APIs, including sandboxes. That imports their uptime
into CI. Contract drift at Buzzsprout is a scheduled canary against staging, not a PR gate.

## Consequences

**Positive**

- The bug classes that have actually cost us are testable, in the fast suite, without infrastructure.
- ADR 0002's convention stops being unenforceable — a future `updateMany` fails a test.
- Tier 3 stays small enough to be maintainable, because Tier 2 covers the per-hook logic.
- The endpoint refactor Tier 3 forces is something the conventions already demanded.

**Negative / costs**

- Tier 2 tests are coupled to the *shape* of the Directus hook context. If a Directus upgrade changes
that shape, the fakes drift from reality and can pass while production breaks. **This is the main
risk, and it is what Tier 3 is insurance against** — the two tiers must not both be faked.
- One shared fake-context helper becomes a load-bearing test utility. Under-building it means 18
divergent copies; over-building it means a mini-framework. Consolidate at the sixth use, not the
first.
- Tier 3 adds ~2 minutes of CI and a stub server to maintain.
- Tier 3 only proves behaviour against 11.17.4. Correct while the server is frozen; revisit when the
licence question resolves.

## Ordering

Tiers 2 and 3 both come **before** the dependency upgrades, which is a change from the first draft of
[the tooling plan](../docs/directus-extension-tooling-plan.md). The reason is measured, not
stylistic: of the 16 files the planned bumps touch, **3 have a test covering them**, and
`algolia-index` — largest module in the bundle, zero tests — is hit by four separate bumps
(`sanitize-html`, `algoliasearch`, `@directus/sdk`, `meow`).

The one exception is the `sanitize-html` critical, which ships as a carve-out with unit tests for its
consumer in the same PR. Holding a security patch behind a multi-PR test phase is the wrong trade.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
22
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Local editor/agent tool state. `prettier . --write` walks the whole tree, and without this it
# rewrites contributors' untracked `.claude/settings.local.json` — a file the formatter has no
# business touching and that nobody reviews.
#
# `node_modules` and `dist` are not listed here on purpose: Prettier reads `.gitignore` in addition
# to this file, and the bundle's `.gitignore` already covers both. Duplicating them would mean two
# places to keep in step.
.claude
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@
"jsxSingleQuote": false,
"trailingComma": "es5",
"bracketSpacing": true,
"jsxBracketSameLine": false,
"arrowParens": "always",
"endOfLine": "lf"
}
Loading
Loading