Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
98 changes: 96 additions & 2 deletions .github/contributing/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -312,6 +312,100 @@ often a different vitest project — from the one being re-rendered.
A full `pnpm run test:update` fixes it in one pass. `test:update` exists so
that the safe command is the short one, and it takes no path on purpose.

## Coverage

```bash
pnpm run test:coverage # the whole suite, instrumented
```

Off by default. Instrumenting every worker costs about 80s on top of a 240s
suite, which is worth paying once in CI and not on every local run. CI runs
this instead of a plain `vitest run`, uploads `coverage/` as an artifact, and
fails when the run drops below the thresholds in `vitest.config.ts`.

**It only works as a whole run.** `pnpm run test:coverage --project vue`, or
with a path filter, still checks the thresholds against the whole of `src/` —
so a partial run is red by construction, on code that never had a chance to
execute. That is vitest's behaviour, not a regression in your branch. Use a
plain `pnpm run test <name>` while iterating.

### Scope

`src/runtime` and `src/theme` — what a browser receives.

The module half of `src/` (`module.ts`, `unplugin.ts`, `vite.ts`, `plugins/`,
`templates.ts`) is deliberately out. It is exercised by `test/module`, which
runs under its own config in its own process and is not instrumented, so
counting it here reported about 175 statements as untested that are in fact
tested — and no amount of module testing could ever have moved the number.

The extensions are spelled out (`**/*.{ts,vue}`) because a bare `**` pulls in
the 36 design-token stylesheets and two token JSON files, which carry no
statements, cannot be covered, and land in the report as three dozen rows at
0%.

### The baseline and what the thresholds buy

Measured on Node 24, which is what CI pins:

```
Statements : 71.89% ( 5265/7323 )
Branches : 69.86% ( 5134/7348 )
Functions : 71.04% ( 1875/2639 )
Lines : 71.45% ( 4659/6520 )
```

Thresholds are that **minus one point, floored** — `70 / 68 / 70 / 70`. Not
the baseline floored: rounding alone produces wildly uneven margins, and the
first version of this gate proved it by putting functions at 69 against a
measured 69.04, which tolerated exactly *one* new uncovered function while
statements tolerated 38.

A point of margin buys this much new uncovered code before the gate goes red:

| metric | headroom |
|---|---|
| statements | 198 |
| branches | 202 |
| functions | 39 |
| lines | 135 |

Against the shape of this repository — 182 components, median 14 statements
and 6 functions, p75 38 and 15, largest `InputMenu.vue` at 190 and 88 — that
means a **large new area arriving with no tests is caught**, on functions and
lines first. A single median component is not.

The margin is a point rather than a rounding because the denominator wobbles.
Three full runs of the same clean tree gave two different totals — 7323 and
7324 statements — and it is not the Node version: both came out of Node 24,
with Node 22 landing on the second. One fully-covered function in `Button.vue`
appears or does not, worth about 0.01pp. Tiny, but it means the number is not
reproducible to the digit, and a threshold set exactly at the baseline would
eventually go red on nothing at all.

### What it does not catch

A component *losing* the tests it had. Measured rather than assumed —
`describe.skip` on the whole `Button` suite stops 212 tests and leaves the
numbers where they were:

```
Statements : unchanged
Functions : unchanged
Lines : unchanged
Branches : -0.19pp
```

`Button.vue` is mounted by dozens of other specs — inside forms, menus,
toolbars — so its code keeps executing whether or not anything asserts about
it. **Coverage measures execution, not assertion.**

A per-file threshold would close that gap and cannot be turned on yet: 93
files sit at 0% today, so `perFile` would be red on the first run.

Treat the number as a floor under the shipped surface, not as evidence that a
component is tested. For that, the component needs its own spec.

## Running Tests

```bash
Expand All @@ -327,8 +421,8 @@ pnpm build && pnpm run test:smoke
# Run specific test file
pnpm run test Button

# Run with coverage
pnpm run test -- --coverage
# Run with coverage — see the Coverage section above
pnpm run test:coverage

# Watch mode
pnpm run test -- --watch
Expand Down
15 changes: 14 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -114,8 +114,21 @@ jobs:
- name: Typecheck
run: pnpm run typecheck

# Coverage rather than a plain `vitest run`: it is the same suite with the
# same assertions, plus a gate that goes red when coverage falls below
# the thresholds in `vitest.config.ts`. Instrumenting every worker costs
# about 100s on a 240s suite, which is why it is one run here and not a
# second job — and why it stays off for local runs unless asked for.
- name: Test
run: pnpm run test run
run: pnpm run test:coverage

- name: Upload the coverage report
if: always()
uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0
with:
name: coverage
path: coverage/
retention-days: 14

# A separate invocation, not a third project in the run above. These
# specs boot Nuxt with `loadNuxt`, and the component suite already sits
Expand Down
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -62,3 +62,6 @@ CLAUDE.md
# Python bytecode from .github/scripts (py_compile, and any local run)
__pycache__/
*.pyc

# Coverage reports (pnpm run test:coverage)
/coverage
8 changes: 8 additions & 0 deletions docs/content/docs/1.getting-started/4.contribution.md
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,14 @@ Before submitting a PR, ensure that you run the tests:
pnpm run test
```

::tip
Coverage is gated in CI. `pnpm run test:coverage` runs the same suite
instrumented, and the
[testing guide](https://github.com/bitrix24/b24ui/blob/main/.github/contributing/testing.md#coverage)
explains what the gate does and does not catch. It only works as a whole run —
a `--project` or a path filter is red by construction.
::

::tip
If you have to update the snapshots, run `pnpm run test:update` — the whole
suite, with no path. A targeted `vitest run -u path/to/One.spec.ts` rewrites
Expand Down
14 changes: 8 additions & 6 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -145,12 +145,13 @@
"test:workflows": "test/workflows/run.sh",
"test:module": "vitest run --config vitest.module.config.ts",
"test:update": "vitest run -u",
"test:smoke": "node test/smoke/run.mjs"
"test:smoke": "node test/smoke/run.mjs",
"test:coverage": "vitest run --coverage"
},
"dependencies": {
"@floating-ui/dom": "^1.8.0",
"@bitrix24/b24icons-nuxt": "^2.0.7",
"@bitrix24/b24icons-vue": "^2.0.7",
"@floating-ui/dom": "^1.8.0",
"@internationalized/date": "^3.12.3",
"@internationalized/number": "^3.6.7",
"@nuxt/kit": "^4.5.2",
Expand Down Expand Up @@ -214,28 +215,29 @@
"vue-component-type-helpers": "^3.3.11"
},
"devDependencies": {
"typescript": "^6.0.3",
"@conventional-commits/parser": "0.4.1",
"@nuxt/eslint-config": "^1.17.0",
"@nuxt/module-builder": "^1.0.3",
"@nuxt/test-utils": "^4.1.0",
"@tanstack/table-core": "^8.21.3",
"@types/canvas-confetti": "^1.9.0",
"@types/node": "^25.3.0",
"@vitejs/plugin-vue": "^6.0.8",
"@vitest/coverage-v8": "4.1.11",
"@vue/test-utils": "^2.4.11",
"ai": "^7.0.77",
"embla-carousel": "^8.6.0",
"eslint": "^10.9.0",
"happy-dom": "^20.11.6",
"nuxt": "^4.5.2",
"playwright-core": "1.56.1",
"typescript": "^6.0.3",
"unbuild": "^3.6.1",
"vitest": "^4.1.11",
"vitest-axe": "^0.1.0",
"vitest-environment-nuxt": "^2.0.0",
"vue": "^3.5.41",
"vue-tsc": "^3.3.11",
"@types/canvas-confetti": "^1.9.0",
"@types/node": "^25.3.0"
"vue-tsc": "^3.3.11"
},
"peerDependencies": {
"@inertiajs/vue3": "^2.0.7 || ^3.0.0",
Expand Down
Loading
Loading