feat: add ETag/304 caching to GET /api/apis

docs/etag-caching.md

@@ -0,0 +1,151 @@
+# ETag / 304 Caching — GET /api/apis
+
+**Issue:** #866  
+**Added in:** `src/middleware/etagCache.ts`, `src/routes/apis.ts`
+
+## Overview
+
+`GET /api/apis` and `GET /api/apis/:id` support HTTP conditional requests via
+strong ETags and `304 Not Modified` responses. Clients that cache the response
+can supply an `If-None-Match` header on subsequent requests; when the response
+content has not changed, the server replies with `304` and an empty body,
+saving bandwidth and client-side parsing time.
+
+## How It Works
+
+1. On every successful `GET /api/apis` or `GET /api/apis/:id` response, the
+   server serialises the response body to JSON, computes a 32-character
+   SHA-256 hex digest, and emits it as a strong `ETag` header:
+
+   ```
+   ETag: "a7ffc6f8bf1ed76651c14756a061d662"
+   ```
+
+2. The client stores the ETag alongside the cached response body.
+
+3. On the next request, the client sends the stored ETag in `If-None-Match`:
+
+   ```
+   GET /api/apis HTTP/1.1
+   If-None-Match: "a7ffc6f8bf1ed76651c14756a061d662"
+   ```
+
+4. If the current response body would hash to the same digest, the server
+   returns `304 Not Modified` with an empty body (saving the transfer cost of
+   the JSON payload).
+
+5. If the data has changed (new APIs added, existing ones updated), the hash
+   differs and the server returns the full `200` response with the updated body
+   and the new ETag.
+
+## Routes Covered
+
+| Route | ETag? | Notes |
+|---|---|---|
+| `GET /api/apis` | ✅ | Covers all query params (limit, offset, category, search). Different params → different ETags. |
+| `GET /api/apis/:id` | ✅ | Per-resource ETag, includes endpoint list in the hash. |
+| `POST /api/apis` | ❌ | Write operation — no caching. |
+| `POST /api/apis/:id/endpoints/bulk` | ❌ | Write operation — no caching. |
+
+## Response Headers
+
+| Header | Example | Description |
+|---|---|---|
+| `ETag` | `"a7ffc6f8bf1ed76651c14756a061d662"` | Strong ETag. Always present on 200 responses from the covered GET routes. |
+
+## Request Headers
+
+| Header | Example | Description |
+|---|---|---|
+| `If-None-Match` | `"a7ffc6f8bf1ed76651c14756a061d662"` | Single ETag, comma-separated list, weak ETag (`W/"..."`), or wildcard (`*`). |
+
+## ETag Format
+
+ETags are **strong** (no `W/` prefix). The value is the first 32 hex characters
+of the SHA-256 digest of the JSON-serialized response body, wrapped in
+double-quotes.
+
+- **Strong** because the digest changes if and only if the response bytes
+  change — precise byte-level equivalence, not just semantic equivalence.
+- **Body-derived** (not timestamp- or version-derived) so two requests that
+  return the same data always produce the same ETag, regardless of when they
+  are made.
+
+## Express Built-in ETag
+
+Express 4.x generates **weak** ETags by default (`app.set('etag', 'weak')` is
+the implicit default). The Callora backend does **not** disable Express's
+default ETag generation globally, because that would affect all other routes.
+
+Instead, `apis.ts` sets the `ETag` header explicitly before calling
+`res.json()`. When an `ETag` header is already present at response finalisation
+time, Express skips its own ETag generation for that response.
+
+## Interaction with the ListingsCache
+
+`GET /api/apis` already uses an in-process `ListingsCache` (30-second TTL by
+default) to skip DB reads on repeated requests. ETag evaluation happens on top
+of this layer:
+
+- **Cache hit + matching ETag:** Both the DB read *and* the HTTP body transfer
+  are skipped. This is the fully-optimised path.
+- **Cache hit + stale/absent ETag:** The DB read is skipped (ListingsCache
+  hit), but the full 200 body is returned.
+- **Cache miss + matching ETag:** The DB read happens (cache miss), the
+  response is built, the ETag is computed, and the 304 shortcut is applied.
+  The body transfer is saved even on cache misses with a warm client.
+
+## Example with curl
+
+**First request — get the ETag:**
+
+```bash
+curl -si https://api.callora.io/api/apis | grep -E '^(HTTP|etag|ETag)'
+```
+
+```
+HTTP/2 200
+etag: "a7ffc6f8bf1ed76651c14756a061d662"
+```
+
+**Subsequent request — 304 when data is unchanged:**
+
+```bash
+curl -si https://api.callora.io/api/apis \
+  -H 'If-None-Match: "a7ffc6f8bf1ed76651c14756a061d662"'
+```
+
+```
+HTTP/2 304
+etag: "a7ffc6f8bf1ed76651c14756a061d662"
+```
+
+**Subsequent request — 200 when data has changed:**
+
+```bash
+curl -si https://api.callora.io/api/apis \
+  -H 'If-None-Match: "a7ffc6f8bf1ed76651c14756a061d662"'
+```
+
+```
+HTTP/2 200
+etag: "b3d2ae19f7c4e0a5d8f92c6b1e4a8305"
+content-type: application/json; charset=utf-8
+
+{"data":[...],"meta":{...}}
+```
+
+## Implementation Notes
+
+The ETag computation uses the **compute-then-compare** approach: the route
+builds the full response object before computing the ETag and issuing a 304.
+For `GET /api/apis`, skipping the full response build on 304 would require
+restructuring the cache layer significantly. Given that the ListingsCache
+already skips the DB on cache hits, and that `JSON.stringify` + SHA-256 on a
+20-item listing is sub-millisecond, the compute-then-compare cost is
+negligible.
+
+The implementation is in `src/middleware/etagCache.ts` and exports three pure
+functions: `computeStrongETag`, `parseIfNoneMatch`, and `isETagMatch`. No new
+npm dependencies were introduced — only Node.js's built-in `node:crypto` module
+is used.

src/lib/listingsCache.ts

@@ -46,6 +46,8 @@ export interface ListingsCacheKeyParams {
   search?: string;
   /** Opaque cursor string for keyset pagination. When present, offset is ignored. */
   cursor?: string;
+  /** Optional status filter. When absent, the route returns active APIs by default. */
+  status?: string;
 }
 
 // ── Cache key builder ─────────────────────────────────────────────────────────
@@ -67,6 +69,9 @@ export function buildCacheKey(params: ListingsCacheKeyParams): string {
     category: params.category ?? null,
     search: params.search ?? null,
     cursor: params.cursor ?? null,
+    // Status is included so ?status=draft and ?status=active are cached
+    // independently and never serve each other's data.
+    status: params.status ?? null,
   });
 }
 

src/middleware/etagCache.test.ts

@@ -0,0 +1,183 @@
+/**
+ * @file src/middleware/etagCache.test.ts
+ * @description Focused unit tests for the ETag / 304 caching utilities in
+ * src/middleware/etagCache.ts.
+ *
+ * These tests cover the pure helper functions in isolation.  The integration
+ * tests that exercise the full HTTP layer (ETag on GET /api/apis, 304 response,
+ * body absent on 304, etc.) live in src/routes/apis.etag.test.ts.
+ */
+
+import { computeStrongETag, parseIfNoneMatch, isETagMatch } from './etagCache.js';
+
+// ────────────────────────────────────────────────────────────────────────────
+// computeStrongETag
+// ────────────────────────────────────────────────────────────────────────────
+
+describe('computeStrongETag', () => {
+  it('returns a quoted string', () => {
+    const tag = computeStrongETag({ data: [], meta: {} });
+    expect(tag).toMatch(/^"[0-9a-f]+"$/);
+  });
+
+  it('returns the same ETag for identical payloads', () => {
+    const body = { data: [{ id: 1, name: 'Test' }], meta: { total: 1 } };
+    expect(computeStrongETag(body)).toBe(computeStrongETag(body));
+  });
+
+  it('returns different ETags for different payloads', () => {
+    const a = computeStrongETag({ data: [{ id: 1 }] });
+    const b = computeStrongETag({ data: [{ id: 2 }] });
+    expect(a).not.toBe(b);
+  });
+
+  it('is sensitive to field order in the JSON representation', () => {
+    // JSON.stringify preserves insertion order; objects with different key
+    // orders produce different JSON strings and thus different ETags.
+    const a = computeStrongETag({ a: 1, b: 2 });
+    const b = computeStrongETag({ b: 2, a: 1 });
+    // These MAY differ because insertion order differs.
+    // We just verify both are valid quoted strings.
+    expect(a).toMatch(/^"[0-9a-f]+"$/);
+    expect(b).toMatch(/^"[0-9a-f]+"$/);
+  });
+
+  it('produces a 34-character string (32 hex chars + 2 quotes)', () => {
+    const tag = computeStrongETag({ x: 1 });
+    expect(tag).toHaveLength(34); // '"' + 32 hex + '"'
+  });
+
+  it('handles an empty object', () => {
+    const tag = computeStrongETag({});
+    expect(tag).toMatch(/^"[0-9a-f]{32}"$/);
+  });
+
+  it('handles null', () => {
+    const tag = computeStrongETag(null);
+    expect(tag).toMatch(/^"[0-9a-f]{32}"$/);
+  });
+
+  it('handles arrays', () => {
+    const tag = computeStrongETag([1, 2, 3]);
+    expect(tag).toMatch(/^"[0-9a-f]{32}"$/);
+  });
+
+  it('handles deeply nested structures', () => {
+    const tag = computeStrongETag({ a: { b: { c: [1, 2, { d: 'e' }] } } });
+    expect(tag).toMatch(/^"[0-9a-f]{32}"$/);
+  });
+});
+
+// ────────────────────────────────────────────────────────────────────────────
+// parseIfNoneMatch
+// ────────────────────────────────────────────────────────────────────────────
+
+describe('parseIfNoneMatch', () => {
+  it('returns an empty set for undefined', () => {
+    expect(parseIfNoneMatch(undefined).size).toBe(0);
+  });
+
+  it('returns an empty set for empty string', () => {
+    expect(parseIfNoneMatch('').size).toBe(0);
+  });
+
+  it('returns a Set containing "*" for the wildcard', () => {
+    const tags = parseIfNoneMatch('*');
+    expect(tags.has('*')).toBe(true);
+    expect(tags.size).toBe(1);
+  });
+
+  it('parses a single quoted ETag', () => {
+    const tags = parseIfNoneMatch('"abc123"');
+    expect(tags.has('abc123')).toBe(true);
+    expect(tags.size).toBe(1);
+  });
+
+  it('parses multiple comma-separated ETags', () => {
+    const tags = parseIfNoneMatch('"aaa", "bbb", "ccc"');
+    expect(tags.has('aaa')).toBe(true);
+    expect(tags.has('bbb')).toBe(true);
+    expect(tags.has('ccc')).toBe(true);
+    expect(tags.size).toBe(3);
+  });
+
+  it('strips the W/ prefix from weak ETags', () => {
+    const tags = parseIfNoneMatch('W/"weaketag"');
+    expect(tags.has('weaketag')).toBe(true);
+  });
+
+  it('strips W/ prefix (case-insensitive)', () => {
+    const tags = parseIfNoneMatch('w/"weaketag"');
+    expect(tags.has('weaketag')).toBe(true);
+  });
+
+  it('handles a mix of strong and weak ETags', () => {
+    const tags = parseIfNoneMatch('"strong", W/"weak"');
+    expect(tags.has('strong')).toBe(true);
+    expect(tags.has('weak')).toBe(true);
+  });
+
+  it('ignores empty segments from trailing commas', () => {
+    const tags = parseIfNoneMatch('"abc",');
+    expect(tags.has('abc')).toBe(true);
+    expect(tags.size).toBe(1);
+  });
+
+  it('handles extra whitespace around tags', () => {
+    const tags = parseIfNoneMatch('  "abc"  ,  "def"  ');
+    expect(tags.has('abc')).toBe(true);
+    expect(tags.has('def')).toBe(true);
+  });
+
+  it('returns an empty set for a completely malformed header (no quotes)', () => {
+    // "notquoted" without surrounding double-quotes is still handled:
+    // the regex strips quotes only if present, leaving the bare value.
+    const tags = parseIfNoneMatch('notquoted');
+    // The value is returned as-is after stripping (no quotes to strip).
+    expect(tags.has('notquoted')).toBe(true);
+  });
+});
+
+// ────────────────────────────────────────────────────────────────────────────
+// isETagMatch
+// ────────────────────────────────────────────────────────────────────────────
+
+describe('isETagMatch', () => {
+  const currentETag = '"abc123def456abc123def456abc12345"'; // 32-char hex digest
+
+  it('returns false when If-None-Match header is absent', () => {
+    expect(isETagMatch(currentETag, undefined)).toBe(false);
+  });
+
+  it('returns false when If-None-Match is empty string', () => {
+    expect(isETagMatch(currentETag, '')).toBe(false);
+  });
+
+  it('returns true when If-None-Match contains the matching ETag', () => {
+    expect(isETagMatch(currentETag, '"abc123def456abc123def456abc12345"')).toBe(true);
+  });
+
+  it('returns true for wildcard "*"', () => {
+    expect(isETagMatch(currentETag, '*')).toBe(true);
+  });
+
+  it('returns false when If-None-Match contains a different ETag', () => {
+    expect(isETagMatch(currentETag, '"different000000000000000000000000"')).toBe(false);
+  });
+
+  it('returns true when the matching ETag is one of many in a list', () => {
+    const header = '"other0000000000000000000000000000", "abc123def456abc123def456abc12345", "another00000000000000000000000000"';
+    expect(isETagMatch(currentETag, header)).toBe(true);
+  });
+
+  it('returns true when the matching ETag is supplied as a weak ETag in If-None-Match', () => {
+    // Per RFC 9110 §13.1.2, If-None-Match uses weak comparison — a weak client
+    // tag matching the strong server tag is still a match.
+    expect(isETagMatch(currentETag, 'W/"abc123def456abc123def456abc12345"')).toBe(true);
+  });
+
+  it('returns false when none of the listed ETags match', () => {
+    const header = '"aaa00000000000000000000000000000", "bbb00000000000000000000000000000"';
+    expect(isETagMatch(currentETag, header)).toBe(false);
+  });
+});