Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 25 additions & 10 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Security

- **`fast-copy` 4.0.x → 4.1.2** (stack exhaustion on deeply nested values),
pulled in by the dev-only `pino-pretty` in the backend and `mcp-docs`. Not
shipped in production images.
- **`fast-copy` 4.0.x → 4.1.2** (GHSA-jggr-w7fw-pc2j: stack exhaustion on
deeply nested values), pulled in by the dev-only `pino-pretty` in the
backend and `mcp-docs`. Not shipped in production images.

- **`@modelcontextprotocol/sdk` 1.30.0 → 1.31.0** in the backend (root
lockfile) and the `mcp-docs` sidecar (GHSA-6qxp-vccf-f47h: the OAuth client
Expand All @@ -29,14 +29,29 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
attribute names (`db.system`, `db.statement`, `db.name`, `http.method`,
`http.status_code`, `net.peer.*`) must move to `db.system.name`,
`db.query.text`, `db.namespace`, `http.request.method`,
`http.response.status_code`, `server.*`.
`http.response.status_code`, `server.*`. The HTTP duration histograms are
renamed **and change unit**: `http.server.duration` / `http.client.duration`
(milliseconds) become `http.server.request.duration` /
`http.client.request.duration` (seconds). Panels on the old names go empty,
and a plain rename leaves latency thresholds off by 1000× — divide them by
1000. See *ADMIN-GUIDE → OpenTelemetry*.
- **Transitive dependency security updates.** `katex` 0.16.47 → 0.18.10
(lazy-loaded by mermaid for `$$…$$` diagram labels; prototype-pollution
`trust` bypass), `sharp` 0.35.4 → 0.35.5 (bundled librsvg 2.63.2), and the
build/dev-only `shell-quote` 1.10.0 → 1.12.0, `postcss-selector-parser`
6.0.10 → 7.1.6 and `smol-toml` 1.8.0 → 1.9.0. `sprintf-js` 1.0.3 stays: no
patched release exists, and it is reachable only through mammoth's
command-line entry point, which the backend never loads.
(GHSA-238p-pmpm-9mq7: lazy-loaded by mermaid for `$$…$$` diagram labels;
prototype-pollution `trust` bypass), `sharp` 0.35.4 → 0.35.5
(GHSA-wq5f-xc86-pv6w: bundled librsvg 2.63.2), and the build/dev-only
`shell-quote` 1.10.0 → 1.12.0 (GHSA-pqg4-j6r4-53mv),
`postcss-selector-parser` 6.0.10 → 7.1.6 (GHSA-rj75-hqrm-r3gf) and
`smol-toml` 1.8.0 → 1.9.0 (GHSA-r4xh-jqrq-34v2). `sprintf-js` 1.0.3 stays
(GHSA-hp3w-g68c-fv3c): no patched release exists, and it is reachable only
through mammoth's command-line entry point, which the backend never loads.

### Changed

- **Node.js floor is now 22.13.0** (`engines.node`; `mcp-docs` 22.12.0). With
`engine-strict=true` the locked tree already refused older Node 22 releases
(jsdom 28 and isomorphic-dompurify need ^22.13.0, vitest 5 and vite 8 need
`>=22.12.0`), so the old `>=22.0.0` promised installs that failed with
`EBADENGINE`.

## [0.9.0] - 2026-10-06

Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Thank you for your interest in contributing to Compendiq! This guide will help y

## Prerequisites

- **Node.js** >= 22.0.0 (24 recommended)
- **Node.js** >= 22.13.0 (24 recommended)
- **Docker** + **Docker Compose** (for PostgreSQL and Redis)
- **Git**

Expand Down
2 changes: 1 addition & 1 deletion backend/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -88,5 +88,5 @@ HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \

ENTRYPOINT ["dumb-init", "--"]
# `--import` preloads the OTel SDK before the app module graph loads so
# auto-instrumentation attaches to http/fastify/pg/redis (issue #922).
# auto-instrumentation attaches to http/undici/pg/redis (issue #922).
CMD ["node", "--import", "./dist/telemetry-register.js", "dist/index.js"]
147 changes: 145 additions & 2 deletions backend/src/dependency-security-invariants.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,10 @@ import { describe, expect, it } from 'vitest';
*/
const repoRoot = join(dirname(fileURLToPath(import.meta.url)), '..', '..');

type LockPackages = Record<string, { version?: string } | undefined>;
type LockPackages = Record<
string,
{ version?: string; dependencies?: Record<string, string> } | undefined
>;

function lockPackages(): LockPackages {
return JSON.parse(readFileSync(join(repoRoot, 'package-lock.json'), 'utf8'))
Expand Down Expand Up @@ -40,12 +43,96 @@ function belowFloor(name: string, floor: string): string[] {
return versions.filter((version) => !versionAtLeast(version, floor));
}

type Version = [number, number, number];
type UpperBound = { version: Version; inclusive: boolean };

function compareVersions(a: Version, b: Version): number {
for (let i = 0; i < 3; i++) {
if (a[i] !== b[i]) return a[i]! - b[i]!;
}
return 0;
}

/** `1.2.3` → given 3; `1.2` / `1.2.x` → given 2; `*` / `x` → given 0. */
function parsePartial(text: string): { version: Version; given: number } {
const parts = text.split('-')[0]!.split('.');
const numbers: number[] = [];
for (const part of parts) {
if (part === '' || /^[xX*]$/.test(part)) break;
if (!/^\d+$/.test(part)) throw new Error(`unsupported version "${text}"`);
numbers.push(Number(part));
}
const given = numbers.length;
while (numbers.length < 3) numbers.push(0);
return { version: numbers.slice(0, 3) as Version, given: Math.min(given, 3) };
}

function bump(version: Version, index: number): Version {
return version.map((part, i) => (i < index ? part : i === index ? part + 1 : 0)) as Version;
}

/** Lower and upper bound of one comparator (`^1.2.3`, `~1.2`, `>=1`, `1.2.x`, …). */
function comparatorBounds(comparator: string): { lower?: Version; upper?: UpperBound } {
const [, operator = '', operand = ''] = comparator.match(/^(\^|~|>=|<=|>|<|=)?v?(.*)$/)!;
const { version, given } = parsePartial(operand);
if (given === 0) {
if (operator === '' || operator === '=' || operator === '>=') return {};
throw new Error(`unsupported comparator "${comparator}"`);
}
switch (operator) {
case '>=':
case '>':
return { lower: version };
case '<':
return { upper: { version, inclusive: false } };
case '<=':
return { upper: { version, inclusive: true } };
case '^': {
const firstNonZero = version.slice(0, given).findIndex((part) => part !== 0);
const index = firstNonZero === -1 ? given - 1 : firstNonZero;
return { lower: version, upper: { version: bump(version, index), inclusive: false } };
}
case '~':
return { lower: version, upper: { version: bump(version, given >= 2 ? 1 : 0), inclusive: false } };
default:
return given === 3
? { lower: version, upper: { version, inclusive: true } }
: { lower: version, upper: { version: bump(version, given - 1), inclusive: false } };
}
}

/**
* Whether `spec` admits any version at or above `floor`. Covers the range
* forms parents publish (exact, `^`, `~`, x-ranges, comparator sets, `||`
* unions); anything else throws, so an unparseable spec fails loudly instead
* of silently disabling the override expiry check below. A `>X` lower bound
* is treated as `>=X`, which only differs when the upper bound is also X.
*/
function rangeAdmitsAtLeast(spec: string, floor: string): boolean {
const minimum = parsePartial(floor).version;
return spec.split('||').some((alternative) => {
if (/\s-\s/.test(alternative)) throw new Error(`unsupported hyphen range "${spec}"`);
let lower: Version = minimum;
let upper: UpperBound | undefined;
for (const comparator of alternative.trim().split(/\s+/).filter(Boolean)) {
const bounds = comparatorBounds(comparator);
if (bounds.lower && compareVersions(bounds.lower, lower) > 0) lower = bounds.lower;
if (bounds.upper && (!upper || compareVersions(bounds.upper.version, upper.version) < 0)) {
upper = bounds.upper;
}
}
if (!upper) return true;
const gap = compareVersions(lower, upper.version);
return gap < 0 || (gap === 0 && upper.inclusive);
});
}

describe('npm advisory floors', () => {
it('resolves smol-toml past the quadratic parseKey scan (GHSA-r4xh-jqrq-34v2)', () => {
expect(belowFloor('smol-toml', '1.9.0')).toEqual([]);
});

it('resolves sharp to a build bundling librsvg 2.63.2 (GHSA-wq5f-xc86-pv6w)', () => {
it('resolves sharp past the bundled-librsvg advisory (GHSA-wq5f-xc86-pv6w)', () => {
expect(belowFloor('sharp', '0.35.5')).toEqual([]);
});

Expand All @@ -65,3 +152,59 @@ describe('npm advisory floors', () => {
expect(installed(lockPackages(), 'extract-zip')).toEqual([]);
});
});

/**
* Root overrides that exist only because the parent's own range excludes every
* patched release (see `//tooling-deps-note` in the root package.json). Once
* the parent widens that range the override is dead weight — and can force a
* downgrade below what the parent now asks for — so it has to be dropped.
*/
const parentBlockedOverrides = [
{ dep: 'katex', parent: 'mermaid', patched: '0.18.2' },
{ dep: 'postcss-selector-parser', parent: '@tailwindcss/typography', patched: '7.1.6' },
];

describe('root overrides waiting on a parent range', () => {
const rootManifest: { overrides?: Record<string, string> } = JSON.parse(
readFileSync(join(repoRoot, 'package.json'), 'utf8'),
);
const overrides = rootManifest.overrides ?? {};

it.each(parentBlockedOverrides)(
'$dep is overridden only while $parent still excludes a patched release',
({ dep, parent, patched }) => {
expect(overrides, `the ${dep} override is gone — remove its row here`).toHaveProperty(dep);
const spec = lockPackages()[`node_modules/${parent}`]?.dependencies?.[dep];
expect(spec, `${parent} no longer depends on ${dep} — drop the override in package.json`)
.toBeTypeOf('string');
expect(
rangeAdmitsAtLeast(spec!, patched),
`${parent} widened its ${dep} range to ${spec} — drop the override in package.json`,
).toBe(false);
},
);

it.each([
['^0.16.47', '0.18.2', false],
['^0.18.0', '0.18.2', true],
['^0.19.0', '0.18.2', true],
['~0.18.1', '0.18.2', true],
['0.18.x', '0.18.2', true],
['^0.0.3', '0.0.4', false],
['6.0.10', '7.1.6', false],
['7.1.6', '7.1.6', true],
['^6.0.10', '7.1.6', false],
['^7.0.0', '7.1.6', true],
['>=6.0.0 <7.1.6', '7.1.6', false],
['>=6.0.0 <=7.1.6', '7.1.6', true],
['^6.0.10 || ^7.0.0', '7.1.6', true],
['*', '7.1.6', true],
] as const)('range %s admits a version >= %s: %s', (spec, floor, admits) => {
expect(rangeAdmitsAtLeast(spec, floor)).toBe(admits);
});

it('refuses range forms it cannot evaluate', () => {
expect(() => rangeAdmitsAtLeast('6.0.0 - 7.2.0', '7.1.6')).toThrow(/unsupported/);
expect(() => rangeAdmitsAtLeast('github:user/repo', '7.1.6')).toThrow(/unsupported/);
});
});
2 changes: 1 addition & 1 deletion backend/src/telemetry-register.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import { fileURLToPath } from 'node:url';
/**
* OpenTelemetry preload invariants (issue #922).
*
* Auto-instrumentation can only monkey-patch http/fastify/pg/redis if the SDK
* Auto-instrumentation can only monkey-patch http/undici/pg/redis if the SDK
* starts BEFORE those modules are first imported. Starting it from inside
* index.ts (after the app module graph has already been evaluated) is too late
* and the instrumentations silently never attach. The fix runs the SDK from a
Expand Down
2 changes: 1 addition & 1 deletion backend/src/telemetry-register.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
* Launched via Node's `--import` hook BEFORE the application module graph
* (`dist/index.js`) is evaluated — see the runtime `CMD` in backend/Dockerfile
* and the `start` script in backend/package.json. This is what lets
* auto-instrumentation monkey-patch http/fastify/pg/redis before those modules
* auto-instrumentation monkey-patch http/undici/pg/redis before those modules
* are first imported. Starting the SDK from inside index.ts (as the old code
* did) ran too late: the module graph had already been loaded, so the
* instrumentations never attached.
Expand Down
83 changes: 83 additions & 0 deletions backend/src/telemetry.integration.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
import { execFile } from 'node:child_process';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';
import { promisify } from 'node:util';
import { describe, expect, it } from 'vitest';
import { isDbAvailable } from './test-db-helper.js';

/**
* Database spans must not leak the DB username (GHSA-qqmp-wf37-98f9:
* `@opentelemetry/instrumentation-pg` ≤ 0.72 stamped every span with
* `db.user`). Exercised end to end: the real auto-instrumentations, as
* `startTelemetry()` configures them, record a real query against the test
* database into an in-memory exporter.
*
* Runs in a child process because instrumentation only attaches if it is
* registered before `pg` is first loaded — production guarantees that with the
* `--import ./dist/telemetry-register.js` preload, and this vitest worker has
* already loaded `pg` through test-setup.
*/
const dbAvailable = await isDbAvailable();
const backendDir = join(dirname(fileURLToPath(import.meta.url)), '..');

const PROBE = `
import { NodeSDK, tracing } from '@opentelemetry/sdk-node';
import { getNodeAutoInstrumentations } from '@opentelemetry/auto-instrumentations-node';

const exporter = new tracing.InMemorySpanExporter();
const processor = new tracing.SimpleSpanProcessor(exporter);
const sdk = new NodeSDK({
serviceName: 'pg-span-probe',
spanProcessors: [processor],
instrumentations: [
getNodeAutoInstrumentations({ '@opentelemetry/instrumentation-fs': { enabled: false } }),
],
});
sdk.start();

// Dynamic on purpose: a static import would load pg before sdk.start().
const { default: pg } = await import('pg');
const client = new pg.Client({ connectionString: process.env.PROBE_POSTGRES_URL });
await client.connect();
await client.query('SELECT 1');
await client.end();

// Spans wait for the async resource detectors before they reach the exporter;
// reading without a flush races them and intermittently sees no spans at all.
await processor.forceFlush();
const spans = exporter.getFinishedSpans().map((span) => ({ name: span.name, attributes: span.attributes }));
await sdk.shutdown();
process.stdout.write(JSON.stringify(spans));
`;

interface ExportedSpan {
name: string;
attributes: Record<string, unknown>;
}

describe.skipIf(!dbAvailable)('OpenTelemetry pg instrumentation', () => {
it('records pg query spans without the database username', async () => {
const postgresUrl = process.env.POSTGRES_URL!;
const username = decodeURIComponent(new URL(postgresUrl).username);
expect(username).not.toBe('');

const { stdout } = await promisify(execFile)(
process.execPath,
['--input-type=module', '-e', PROBE],
{ cwd: backendDir, env: { PATH: process.env.PATH, PROBE_POSTGRES_URL: postgresUrl } },
);
const spans: ExportedSpan[] = JSON.parse(stdout);

const querySpan = spans.find((span) => span.name.startsWith('pg.query:'));
expect(querySpan, `no pg.query span among ${spans.map((span) => span.name).join(', ')}`)
.toBeDefined();

const leaks = spans.flatMap((span) =>
Object.entries(span.attributes)
.filter(([key, value]) => key === 'db.user' || value === username)
.map(([key]) => `${span.name} ${key}`),
);
expect(leaks).toEqual([]);
expect(querySpan!.attributes['db.system.name']).toBe('postgresql');
}, 60_000);
});
5 changes: 3 additions & 2 deletions backend/src/telemetry.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
*
* The SDK is started from `telemetry-register.ts`, which is loaded via Node's
* `--import` preload hook BEFORE the application module graph — that ordering
* is what lets auto-instrumentation monkey-patch http/fastify/pg/redis before
* is what lets auto-instrumentation monkey-patch http/undici/pg/redis before
* those modules are first imported (issue #922). The started SDK and tracer are
* stashed on `globalThis` so `getTracer`/`shutdownTelemetry` can reach them
* regardless of which entrypoint started the SDK.
Expand Down Expand Up @@ -85,7 +85,8 @@ export async function startTelemetry(): Promise<void> {
serviceName,
instrumentations: [
getNodeAutoInstrumentations({
// Instrument HTTP, Fastify, pg, Redis automatically
// HTTP, undici, pg and Redis are instrumented automatically. No Fastify
// instrumentation is installed: requests appear as HTTP server spans.
'@opentelemetry/instrumentation-fs': { enabled: false }, // too noisy
}),
],
Expand Down
27 changes: 27 additions & 0 deletions docs/ADMIN-GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -1943,6 +1943,33 @@ OTEL_EXPORTER_OTLP_ENDPOINT=http://your-collector:4318

This exports traces to any OTLP-compatible collector (Jaeger, Grafana Tempo, Datadog, etc.).

The backend auto-instruments HTTP (including incoming Fastify requests, as HTTP
server spans), undici, `pg` and Redis. There is no Fastify-specific route
instrumentation.

#### Stable semantic conventions (since the 0.221 OTel release)

The OpenTelemetry instrumentations now emit **only** the stable semantic
conventions; there is no opt-out back to the old names. Collector rules,
dashboards and alerts keyed on the old names stop matching after the upgrade:

| Old | New |
|-----|-----|
| span attribute `db.system` | `db.system.name` |
| span attribute `db.statement` | `db.query.text` |
| span attribute `db.name` | `db.namespace` |
| span attribute `http.method` | `http.request.method` |
| span attribute `http.status_code` | `http.response.status_code` |
| span attributes `net.peer.name` / `net.peer.port` | `server.address` / `server.port` |
| metric `http.server.duration` (histogram, **ms**) | `http.server.request.duration` (histogram, **s**) |
| metric `http.client.duration` (histogram, **ms**) | `http.client.request.duration` (histogram, **s**) |

The HTTP duration metrics changed **unit** as well as name: renaming a query
is not enough — latency thresholds and panel units must move from
milliseconds to seconds (a 500 ms alert threshold becomes `0.5`), otherwise
they are off by a factor of 1000. Database spans no longer carry `db.user`
or `db.connection_string` (GHSA-qqmp-wf37-98f9).

### Audit Logging

Compendiq logs user actions and system events. Query audit logs via the admin API: `GET /api/admin/audit-log` (admin-authenticated, supports pagination/filtering). Logs are stored in PostgreSQL and include:
Expand Down
Loading
Loading