BenchZero

BenchZero is a small automatic test station written in C17. It demonstrates how a test program can control a device under test (DUT), collect measurements, evaluate requirements, detect injected faults, and produce a final pass/fail report.

The project runs entirely on one computer:

┌──────────────────────────┐       TCP        ┌──────────────────────────┐
│ BenchZero Tester         │ ───────────────▶ │ DUT Simulator            │
│                          │                  │                          │
│ • controls the DUT       │ ◀─────────────── │ • maintains device state │
│ • collects measurements  │    responses     │ • returns measurements   │
│ • evaluates limits       │                  │ • exposes status bits    │
│ • prints the report      │                  │ • injects faults         │
└──────────────────────────┘                  └──────────────────────────┘

What it tests

A normal test session:

1. Connects to the DUT simulator and verifies the protocol version.
2. Powers on the DUT and places it in active mode.
3. Measures supply voltage and checks the 4.750–5.250 V requirement.
4. Measures temperature and checks the −20–80 °C requirement.
5. Reads the status register and verifies that bits in mask 0x000F are clear.
6. Runs the built-in self-test.
7. Powers off the DUT and prints the overall result.

Example report:

BENCHZERO AUTOMATIC TEST REPORT
==================================================

[PASS] Connected to DUT
[PASS] DUT powered on
[PASS] Supply voltage
       Actual: 5.020 V
       Required: 4.750 V to 5.250 V
[PASS] Temperature
       Actual: 38.500 C
       Required: -20.000 C to 80.000 C
[PASS] Status register
       Actual: 0x0000
       Disallowed mask: 0x000F
[PASS] Built-in self-test

FINAL RESULT: PASS

Running BenchZero

The all-in-one script configures CMake, compiles the project, runs the unit tests, starts the simulator, executes the tester, and cleans up the server:

./scripts/build_and_run.sh

It uses Ninja when available and falls back to Make. Build artifacts are written to build-app/.

To run a failing scenario, pass a fault name:

./scripts/build_and_run.sh overtemperature

The endpoint can be changed without editing the script:

BENCHZERO_HOST=127.0.0.1 \
BENCHZERO_PORT=9100 \
./scripts/build_and_run.sh

Running the processes separately

After building, start the simulator:

./build-app/dut_simulator \
  --host 127.0.0.1 \
  --port 9000 \
  --fault none

Then run the tester in another terminal:

./build-app/benchzero_tester \
  --host 127.0.0.1 \
  --port 9000

Both programs support --help. Add --debug to print protocol traffic.

Running with Docker

BenchZero can run without a host C toolchain by using Docker. The image uses a multi-stage Alpine build: CMake and the compiler remain in the builder stage, while the final image contains only the two executables and their runtime environment.

The simplest containerized run starts the simulator and tester as separate Compose services:

./scripts/run_docker.sh

The containers communicate over Compose's private network. The simulator is not published to the host, and the tester waits for its health check before starting. The script returns the tester's exit code and removes both containers.

Run an injected-fault scenario by passing its name:

./scripts/run_docker.sh overtemperature

Docker Compose can also be invoked directly:

docker compose up --build --abort-on-container-exit --exit-code-from tester
docker compose down

To build only the image:

docker build -t benchzero:local .

The default image command runs the DUT on 0.0.0.0:9000.

Fault injection

Faults are deterministic: selecting the same fault produces the same behavior on every run.

Fault	Simulated behavior	Expected result
none	Nominal voltage, temperature, status, and self-test	PASS
undervoltage	4.100 V and undervoltage status bit	FAIL
overvoltage	5.900 V and overvoltage status bit	FAIL
overtemperature	105 °C and overtemperature status bit	FAIL
self-test	Failed built-in self-test and status bit	FAIL
stuck-status-bit	Disallowed status bit remains asserted	FAIL
intermittent-timeout	Every third counted DUT operation times out	ERROR

FAIL means the test ran and found a requirement violation. ERROR means the tester could not obtain all required results.

Architecture

BenchZero is split into a reusable core library and two executables:

Component	Responsibility
dut_simulator	TCP server and owner of per-connection DUT state
benchzero_tester	Test sequence, response parsing, and final report
dut.c	DUT state transitions, measurements, and fault behavior
protocol.c	Newline-delimited command dispatch and response formatting
socket_utils.c	POSIX TCP setup, complete writes, and line-oriented reads
test_report.c	Limit evaluation, status masks, totals, and report output

The simulator handles one tester at a time without threads. It creates a fresh DUT state for each connection and returns to listening after the client sends QUIT or disconnects.

The protocol is plain text with one command and one response per line. The socket layer accounts for TCP partial reads, partial writes, interrupted system calls, disconnects, and oversized commands.

Protocol at a glance

Tester                         DUT Simulator
  │──── HELLO ─────────────────────▶│
  │◀─── HELLO BENCHZERO_DUT 1 ─────│
  │──── POWER_ON ──────────────────▶│
  │◀─── OK ─────────────────────────│
  │──── SET_MODE ACTIVE ───────────▶│
  │◀─── OK ─────────────────────────│
  │──── READ_VOLTAGE ──────────────▶│
  │◀─── VOLTAGE 5.020 ──────────────│
  │──── READ_TEMPERATURE ──────────▶│
  │◀─── TEMPERATURE 38.500 ─────────│
  │──── READ_STATUS ───────────────▶│
  │◀─── STATUS 0x0000 ──────────────│
  │──── RUN_SELF_TEST ─────────────▶│
  │◀─── SELF_TEST PASS ─────────────│
  │──── QUIT ──────────────────────▶│
  │◀─── BYE ────────────────────────│

See the generated protocol documentation for the complete command and error reference.

Process exit codes

Code	Meaning
0	Every required test passed
1	At least one completed test failed
2	Communication or test-execution error
3	Invalid command-line configuration

Test suite

The tests/ directory contains a dependency-free C test harness and focused suites for:

• DUT initialization, state transitions, pointer validation, and every fault
• Protocol lifecycle, malformed commands, domain errors, and buffer truncation
• Inclusive measurement limits, status masks, capacity, ownership, and rendering
• Socket writes, LF/CRLF framing, oversized-line recovery, and disconnects
• End-to-end tester/simulator behavior for all seven fault selections

Run everything through CTest:

ctest --test-dir build --output-on-failure

Run only the fast in-process tests:

ctest --test-dir build -L unit --output-on-failure

Run only the process-level TCP scenarios:

ctest --test-dir build -L integration --output-on-failure

Documentation

The source and public APIs are documented with Doxygen. Generate and open the HTML documentation with:

./scripts/install_and_build_docs.sh

This installs Doxygen through Homebrew if necessary, generates the documentation under build-docs/docs/html/, and opens it in the default browser. To generate without opening a browser:

./scripts/install_and_build_docs.sh --no-open

The documentation includes:

• A UML component and domain diagram
• A high-level happy-path control-flow diagram
• State, ownership, and error-boundary descriptions
• The complete text protocol
• Fault and testing behavior
• Public and private C API reference

Building manually

BenchZero requires a C17 compiler and CMake. Ninja is optional.

cmake -S . -B build -DCMAKE_BUILD_TYPE=Debug
cmake --build build --parallel
ctest --test-dir build --output-on-failure

Enable AddressSanitizer and UndefinedBehaviorSanitizer with:

cmake -S . -B build-sanitized \
  -DCMAKE_BUILD_TYPE=Debug \
  -DBENCHZERO_ENABLE_SANITIZERS=ON
cmake --build build-sanitized --parallel
ctest --test-dir build-sanitized --output-on-failure

Repository layout

.
├── include/benchzero/   Public C API
├── src/                 Core implementation and executable entry points
├── tests/               Dependency-free unit tests
├── scripts/             Build, demo, and documentation helpers
├── docs/                Doxygen configuration and architecture pages
└── CMakeLists.txt       Build configuration

Project scope

BenchZero intentionally stays small. It uses fixed-size storage, explicit result codes, a readable protocol, and no third-party runtime libraries. It is a hardware-adjacent C project rather than a model of a production military or industrial automatic test system.

The current implementation targets POSIX sockets on macOS and Linux. It does not provide authentication, encryption, multi-client concurrency, real-time guarantees, or native Windows socket support.