Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
5dcb7ec
feat(coverage): emit function records in LCOV and optional text summary
Chemaclass May 4, 2026
8282bc4
test(coverage): cover HTML per-line test attribution tooltip
Chemaclass May 4, 2026
3458073
test(coverage): add HTML report generation tests
Chemaclass May 4, 2026
21e3631
test(coverage): pin subshell tracking behavior
Chemaclass May 4, 2026
5ba609c
test(coverage): cover parallel data aggregation
Chemaclass May 4, 2026
7f808d0
feat(coverage): list uncovered executable lines in text report
Chemaclass May 4, 2026
46dbad5
docs(coverage): document new env vars and pin subshell contract
Chemaclass May 4, 2026
3b026de
test(coverage): skip subshell tests under parallel mode and on Git Bash
Chemaclass May 4, 2026
7688aa8
docs(adr): branch coverage MVP design
Chemaclass May 4, 2026
56c9e7b
feat(coverage): branch extractor and hit computation
Chemaclass May 4, 2026
ace7b50
test(coverage): cover BRDA/BRF/BRH emission in LCOV report
Chemaclass May 4, 2026
60652d2
docs(coverage): document branch coverage MVP
Chemaclass May 4, 2026
21adf54
refactor(coverage): extract helpers in branch logic, verify Bash 3.2
Chemaclass May 4, 2026
967489b
docs(coverage): expand branch coverage section with worked example
Chemaclass May 4, 2026
06eceba
refactor(coverage): split branch parser into per-construct handlers
Chemaclass May 4, 2026
fee2da3
feat(coverage): branch coverage MVP via static branch-point detection…
Chemaclass May 4, 2026
c0593b1
refactor(coverage): tighten LCOV record parsing and branch hit walk
Chemaclass May 4, 2026
cfdee38
refactor(coverage): drop subshell capture in branch hit walk
Chemaclass May 4, 2026
1377743
refactor(coverage): full Bash 3.0+ compliance
Chemaclass May 4, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,10 @@
- `--show-output` displays captured test output on assertion failures (#637)
- npm registry distribution: `npm install -g bashunit` (#244)
- `bashunit::env::supports_color` and `bashunit::io::clear_screen` helpers (#247)
- LCOV reports now include `FN`, `FNDA`, `FNF` and `FNH` function records, consumed by `genhtml`, Codecov and Coveralls
- LCOV reports now include `BRDA`, `BRF` and `BRH` branch records for `if`/`elif`/`else` chains and `case` patterns (see `adrs/adr-007-branch-coverage-mvp.md`)
- `BASHUNIT_COVERAGE_SHOW_FUNCTIONS=true` adds a per-function coverage block to the text report
- `BASHUNIT_COVERAGE_SHOW_UNCOVERED=true` adds an "Uncovered Lines" block to the text report, with consecutive line numbers compressed into ranges

### Changed
- Docs moved into their own npm workspace under `docs/` (use `cd docs && npm ci` or `make docs/install`)
Expand Down
90 changes: 90 additions & 0 deletions adrs/adr-007-branch-coverage-mvp.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
# Branch Coverage MVP via Static Branch-Point Detection

* Status: accepted
* Date: 2026-05-04

## Context and Problem Statement

Coverage today reports line-level execution only. Standard tooling (genhtml, Codecov, Coveralls) consumes branch records via the LCOV `BRDA`/`BRF`/`BRH` fields, which let reviewers see whether `else`/`elif` arms and individual `case` patterns were exercised. Adding true branch coverage to a Bash framework is non-trivial because:

1. Bash exposes no native instrumentation comparable to gcov branch counters.
2. The DEBUG trap fires on commands, not on branch decisions.
3. `BASH_COMMAND` reflects the *next* command, not the boolean outcome of a conditional.

We need a path that yields useful, mostly-correct branch metrics in LCOV reports without breaking Bash 3.0+ compatibility or the cost profile of the existing line tracker.

## Decision Drivers

* Bash 3.0+ compatibility (no associative arrays, no `[[`, no Bash 4-only features).
* Reuse existing line-hit data; do not double the runtime cost of coverage.
* LCOV output must be consumable by genhtml, Codecov and Coveralls without custom processing.
* Implementation must fit in `src/coverage.sh` and remain testable with the existing unit-test patterns.
* Behavior must be predictable enough to pin in tests; "best-effort heuristic" outputs are not acceptable.

## Considered Options

1. **Static branch-point detection plus line-hit inference** — parse the source file for branch-introducing constructs (`if`/`elif`/`else`, `case` patterns), compute the line range owned by each outcome, then mark the outcome as "taken" iff any line inside its range was hit.
2. **Runtime decision tracing via `BASH_COMMAND`** — record the actual command being executed in the DEBUG trap and reconstruct decisions taken (`if X` followed by execution of either then-block or else-block).
3. **Patch-based instrumentation** — preprocess source files to insert hit recorders inside each branch arm, run tests against the instrumented copy, post-process the data file.

## Decision Outcome

Chosen option: **Option 1 (static branch-point detection plus line-hit inference)**.

It reuses the existing line-hit data file with no DEBUG-trap changes. Bash 3.0+ compatibility is preserved because the parser is a single pass over the source with brace counting, identical in shape to the existing `extract_functions` walker. The output maps cleanly to LCOV `BRDA` records, and the contract ("an arm is taken iff any executable line inside it was hit") is precise enough to write unit tests against.

### Positive Consequences

* Zero runtime cost beyond the existing line tracker. Branch records are computed during report generation, not during test execution.
* Reuses `is_executable_line` and `get_all_line_hits`, which already tolerate Bash 3.0 limitations.
* LCOV output remains a single file, consumed unchanged by downstream tools.

### Negative Consequences

* Branch detection is line-presence based, not outcome based. A `then` arm whose only statement is a comment-line will register as `not taken` even if the conditional fired (because there are no executable lines inside). This is documented as a known limitation.
* Implicit `else` (when an `if/elif` chain has no explicit `else`) is reported only when at least one explicit arm exists; the synthetic "fall-through" outcome is omitted from this MVP and may be added in a follow-up.
* Compound conditionals (`if A && B`) are reported as a single binary decision, not per sub-expression.

## Pros and Cons of the Options

### Option 1: Static + line-hit inference (chosen)

* Good, because reuses existing data and code paths.
* Good, because matches the implementation pattern of `extract_functions` already shipping in the codebase.
* Good, because output is deterministic and easy to test.
* Bad, because cannot distinguish "arm executed but produced no executable lines" from "arm not executed".

### Option 2: Runtime DEBUG-trap decision tracing

* Good, because reflects actual runtime behavior.
* Bad, because `BASH_COMMAND` semantics across Bash 3.x and 5.x diverge for `((...))`, `[[...]]` and pipelines, requiring per-version logic.
* Bad, because increases per-line overhead; the existing tracker already has measurable cost.
* Bad, because subshell context loss (already documented for line coverage) extends to branches taken inside `$(...)`.

### Option 3: Source-rewrite instrumentation

* Good, because most accurate signal possible.
* Bad, because requires either running tests against a rewritten source tree or hooking `source` to redirect to instrumented copies — both invasive and brittle.
* Bad, because debugging stack traces and line numbers no longer match the user's source.
* Bad, because doubles the code surface and breaks the "DEBUG-trap only" simplicity model.

## Scope of MVP

Included:

* `if`/`elif`/`else` chains: each arm is one outcome.
* `case` statements: each pattern is one outcome.
* LCOV `BRDA:<line>,<block>,<branch>,<taken>` lines.
* `BRF:<count>` and `BRH:<count>` per file.

Deferred (potential follow-ups):

* Synthetic "implicit-else" outcomes for `if/elif` chains without an explicit `else`.
* Per-sub-expression decisions inside `if A && B`.
* `&&` / `||` short-circuit branches outside `if`.
* Loop-entry decisions (`while`/`until`).

## Links

* Builds on the function extractor introduced in `src/coverage.sh` (see `bashunit::coverage::extract_functions`).
* LCOV format reference: <https://manpages.debian.org/unstable/lcov/geninfo.1.en.html>
146 changes: 145 additions & 1 deletion docs/coverage.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,10 @@ BASHUNIT_COVERAGE_MIN=80
# Color thresholds for console output
BASHUNIT_COVERAGE_THRESHOLD_LOW=50 # Red below this
BASHUNIT_COVERAGE_THRESHOLD_HIGH=80 # Green above this, yellow between

# Optional text-report blocks (off by default, opt-in for verbose runs)
BASHUNIT_COVERAGE_SHOW_FUNCTIONS=true # Print per-function coverage
BASHUNIT_COVERAGE_SHOW_UNCOVERED=true # Print missed line ranges per file
```

## Examples
Expand Down Expand Up @@ -274,6 +278,13 @@ end_of_record
|-------|-------------|---------|
| `TN:` | Test Name (usually empty) | `TN:` |
| `SF:` | Source File path | `SF:/home/user/project/src/math.sh` |
| `FN:` | Function: `start_line,name` | `FN:5,multiply` |
| `FNDA:` | Function call data: `count,name` (1 if any line in body was hit, else 0) | `FNDA:1,add` |
| `FNF:` | Functions Found | `FNF:2` |
| `FNH:` | Functions Hit | `FNH:1` |
| `BRDA:` | Branch data: `decision_line,block,arm,taken` | `BRDA:12,0,1,1` |
| `BRF:` | Branches Found | `BRF:6` |
| `BRH:` | Branches Hit | `BRH:4` |
| `DA:` | Line Data: `line_number,hit_count` | `DA:15,3` (line 15 hit 3 times) |
| `LF:` | Lines Found (total executable lines) | `LF:25` |
| `LH:` | Lines Hit (lines with hits > 0) | `LH:20` |
Expand Down Expand Up @@ -347,6 +358,131 @@ These lines are not counted toward coverage:
- Control flow keywords (`then`, `else`, `fi`, `do`, `done`, `esac`, `in`)
- Case statement patterns (`--option)`, `*)`) and terminators (`;;`, `;&`, `;;&`)

## Branch Coverage

Beyond line and function coverage, bashunit emits **branch coverage** records in the LCOV report so reviewers can see whether each `else`/`elif` arm and each `case` pattern was exercised. Branch records are produced automatically; no extra flags are needed.

### What Counts as a Branch

| Construct | Arms |
|-----------|------|
| `if X; then ... fi` | 1 (the `then` body) |
| `if X; then ... else ... fi` | 2 (`then` + `else`) |
| `if X; then ... elif Y; then ... else ... fi` | 3 (one per arm) |
| `case X in a) ... ;; b) ... ;; *) ... ;; esac` | one per pattern |

An arm is reported as **taken** iff at least one executable line inside its range was hit by tests.

### Verbose Output Helpers

Two opt-in environment variables enrich the text report when investigating coverage gaps:

::: code-group
```bash [Per-function block]
BASHUNIT_COVERAGE_SHOW_FUNCTIONS=true bashunit tests/ --coverage
```
```bash [Uncovered lines block]
BASHUNIT_COVERAGE_SHOW_UNCOVERED=true bashunit tests/ --coverage
```
```bash [Both]
BASHUNIT_COVERAGE_SHOW_FUNCTIONS=true \
BASHUNIT_COVERAGE_SHOW_UNCOVERED=true \
bashunit tests/ --coverage
```
:::

The default text report stays compact; opt in only when triaging.

### Worked Example

Given `src/route.sh`:

```bash
#!/usr/bin/env bash
function route() {
if [ "$1" = "GET" ]; then
echo "fetch"
elif [ "$1" = "POST" ]; then
echo "create"
else
echo "405"
fi
}
```

If tests only call `route GET`, the LCOV record looks like:

```
TN:
SF:/path/to/src/route.sh
FN:2,route
FNDA:1,route
FNF:1
FNH:1
BRDA:3,0,0,1
BRDA:3,0,1,0
BRDA:3,0,2,0
BRF:3
BRH:1
DA:3,1
DA:4,1
DA:5,0
DA:6,0
DA:7,0
DA:8,0
LF:6
LH:2
end_of_record
```

**Reading the branch records:**
- `BRDA:3,0,0,1`: decision on line 3, block 0, arm 0 (`then`/GET), taken.
- `BRDA:3,0,1,0`: same decision, arm 1 (`elif`/POST), not taken.
- `BRDA:3,0,2,0`: same decision, arm 2 (`else`/405), not taken.
- `BRF:3` `BRH:1`: 3 branches found, 1 taken.

### Visualizing with genhtml

LCOV's `genhtml` renders branch coverage alongside line and function coverage:

::: code-group
```bash [Generate]
bashunit tests/ --coverage
genhtml --branch-coverage coverage/lcov.info -o coverage/html
```
:::

The resulting site shows a red/green diamond next to each branch decision, mirroring `gcov`'s C/C++ output.

### CI Integration

Codecov and Coveralls pick up the new records without configuration. To require branch coverage in PR gates:

::: code-group
```yaml [Codecov]
coverage:
status:
project:
default:
target: 80%
patch:
default:
target: 80%
threshold: 0%
flags:
- branch
```
:::

### Limitations

- An arm whose body has no executable lines (only comments or braces) registers as not-taken even when the conditional fired.
- Implicit `else` (an `if`/`elif` chain without an explicit `else`) reports only the explicit arms; the synthetic fall-through outcome is omitted.
- Compound conditionals (`if A && B`) are reported as a single binary decision, not per sub-expression.
- `&&`/`||` short-circuit branches outside `if` and loop-entry decisions (`while`/`until`) are not tracked.

See `adrs/adr-007-branch-coverage-mvp.md` for the design rationale and the rejected alternatives.

## Limitations

### External Commands
Expand All @@ -355,4 +491,12 @@ Coverage only tracks Bash code. External commands (like `grep`, `sed`, etc.) are

### Subshell Behavior

Due to Bash's process model, some subshell contexts may not have full coverage tracking. The DEBUG trap is inherited into subshells, but complex nested scenarios may have edge cases.
Due to Bash's process model, hits produced inside a subshell are written to the subshell's in-memory buffer, which is discarded when the subshell exits. The pinned behavior is:

- `$( ... )` command substitution: the outer line is recorded; commands inside the substitution are not.
- `( ... )` explicit subshells: the same applies; only the outer line is tracked.
- Pipelines (`a | b`): each stage is recorded as a single hit on its source line.
- Process substitution `< <( ... )`: the consumer side is fully tracked; producer lines are not.
- Functions invoked from `$( ... )`: the call site and surrounding lines are hit, but the function body lines are lost when called inside a subshell.

These contracts are pinned by `tests/unit/coverage_subshell_test.sh`.
Loading
Loading