Skip to content

About

Twenty bench labs on one parts bin and four Linux hosts, every pairing checked against headers, voltages and silicon. One expansion board per host, one instrument per lab, and an acceptance test written to be failable.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

116 Commits

Folders and files

Repository files navigation

Embedded Linux Top 20: the audited kit cookbook

Twenty bench labs on one parts bin and four Linux hosts, with every pairing checked against headers, voltages and silicon before it was written down. One 40-pin HAT per host. One instrument per lab. An acceptance test you can fail.

81 figures, 20 labs. Each lab stands on its own: what it alone owns, what the earlier draft of this material got wrong, the kit it takes from the bin, a system architecture figure, a schematic with a wiring table, a bench layout, a UML view, a data-flow sketch, numbered steps with real commands, an acceptance test, the practices that keep it honest, its pitfalls and its sources.

Read it. The whole volume is in chapters/ as Markdown with its figures beside it. Start with About this edition, or take a lab from the table below.

Or check a pin. BOARDS.md gathers what every board in the bin claims on the header, with each fact marked as read from the vendor, measured on this bench with a date, or still open. It carries the decisions taken before any wiring, and an account of the three times the indicator pins moved, including the move whose stated reason later turned out to be false.

Or read how it went wrong. journal/ is the working record, one file per lab that has one. The chapters tell you what is true; these tell you what was believed first, and what it took to stop believing it. Four claims in this volume turned out to be false and all four had looked exactly like the ones that were right, so the route matters as much as the conclusion.

Or search it. PITFALLS.md is every pitfall in the volume, generated from the labs and sorted across them rather than within them. It carries its own count, and this page deliberately does not repeat it: a number written into a sentence here is a number nothing updates when the labs move, and this one had drifted six behind before anybody noticed. It is for the other direction of reading: a bench is misbehaving, and the lab that discussed the symptom is not the lab being built. Putting 5 V on an accelerometer is a pitfall in P01 and again in P19, and feeding 5 V into SYS_3V3 is one in P09 and again in P17.

The sources are not here, and that is deliberate. A chapter is a .md file, and the .tex that generates it stays on the authoring machine along with the 81 TikZ figure sources. GitHub renders Markdown and renders neither LaTeX nor a PDF, so publishing the sources would offer a reader raw markup instead of a chapter. What is published is the Markdown edition, the SVG of every figure, the labs' own software, and the reference pages. One consequence is worth knowing: lint.py, checkcode.py and mdbuild.py have nothing to read in a fresh clone, which is why prepublish.py exists and why the workflow no longer tries to run them.

Or build it. The PDF and a single self-contained HTML file come from the same source, which lives on the authoring machine, and both stay local:

python build.py --chapter 5

That writes lab-05-mcc-118-100-ks-s-voltage-recorder.pdf and a matching self-contained .html with its four figures inlined.

Contents Read it · What has been run · The working record · Every pitfall · What every board claims · The three ways to damage this kit · What this is and is not · The rules the labs obey · The twenty labs · Building · Checks · Layout · Requirements · Licence

What this is and is not

This is an audited cookbook, not a tested codebase. It began as two documents: a kit-only twenty-lab split, and a longer architectural draft that reused the same parts bin and proposed pairings that cannot be built from it. This edition keeps the ambition of the second and throws out its impossible wiring, lab by lab, with the correction printed beside the lab it belongs to rather than hidden in a changelog. Thirteen of the twenty carry such a correction.

What each lab commits to is its acceptance test, written to be checkable on hardware and to be failable. "A continuous ping is a fail" is a real line in one of them. A lab that cannot reach its test because the bench lacks something says so in place of a number, because a blank is honest and an unmeasured number is a claim the first careful reader will check.

The parts constrain the labs. Every lab was written against a fixed inventory, so none opens with a shopping list. Where a lab would be easier with an instrument that is not here, it names the substitute.

This is the companion to a volume of a different kind. The twenty projects in embedded-linux-projects are software engineering depth: the Yocto Project, kernel drivers, real time, over-the-air updates, trusted execution. These twenty are bench integration discipline, and the two share a parts bin without sharing a lab.

The rules the labs obey

Six rules decide what a lab may do, and every lab in the table was checked against them. They are in full in About this edition.

  1. No new parts. Antennas are in scope. A data SIM is assumed for four labs only, and without one those labs stop at bring-up, which is still a pass.
  2. HAT exclusivity. Six boards each own the 40-pin header. Fit one. The official display does not consume the header.
  3. The NanoPi is 24-pin. It cannot accept any 40-pin HAT.
  4. 3.3 V logic. Every host pin is 3.3 V, and the breadboard supply is a labelled rail splitter rather than a licence to mix.
  5. One instrument per lab. Six sensor boards are six different instruments, never interchangeable.
  6. Linux owns the system. A microcontroller speaks over USB, a serial line or a bus to a host that owns storage, time, the network and the interface. A bare-metal sketch is allowed only where a Linux host still records it.

The twenty labs

The numbering is the cookbook's own, so the labs appear here in reading order and keep their P-numbers. The parts are the cellular group, the instruments, the sensor shields on the microcontroller, the hosts and displays, and the rails, sidecars and fleet that close it.

Written is not run. Every lab below ends in an acceptance test that can be failed, and STATUS.md says which of them have actually been compiled, wired and passed, with the board revision and the date. That page holds the count, and this one deliberately does not repeat it: a number written into a sentence here is a number nothing updates when the table moves. Read that page before trusting any step here.

Three ways in, rather than twenty equal doors. The appendix carries the full order for someone starting from bare boards and intending to build all twenty. These three are subsets of that order, for a reader who owns part of the bin or has one weekend, and they keep its relative sequence.

Route Order What it needs
Cellular P01, P03, P02, then P13 A Pi 4, a Pi 3B+ and a Pi 3, one modem HAT each, and antennas. A data SIM is assumed; without one each lab stops at AT bring-up, which is still a pass. P13 is last because it wants two Pis and two modems seated at once
One weekend, one Pi 4, no SIM P15, then P16, P12, P05, P04 A Pi 4, the POW-BB, the official display and keyboard, the MCC 118, an ESP32 and the STWIN.box. P15 first because P16 and P05 both draw on the labelled rails
Nucleo P07, P08, then P18 A Nucleo-H7A3ZI-Q, the IKS4A1 and IKS5A1 fitted one at a time, and a Pi to record. None of the three touches the POW-BB, so P15 is not a prerequisite here

P15 appears first in the weekend route and not in the other two for a reason the repository can check rather than assert: inventory.json records which labs use the POW-BB, and they are P05, P10, P15 and P16.

No lab records how long it takes, and that is a gap rather than an omission here. Until a sitting is actually timed, any figure in this table would be invention, and STATUS.md says how few sittings there have been.

# Lab Host and exclusive part Owns
P01 LTE Cat-4 motion-triggered gateway Pi 4, SIM7600E-H, ADXL345 the only Cat-4 default route
P02 NB-IoT PSM/eDRX CoAP field node Pi 3, SIM7020E NB-IoT and the duty cycle
P03 Cat-M MQTT node with GNSS stamp Pi 3B+, SIM7070G Cat-M and the timestamp rule
P04 STWIN.box vibration and ultrasound USB gateway Pi 4, STWIN.box 6 kHz vibration and the microphones
P05 MCC 118 100 kS/s voltage recorder Pi 4, MCC 118 the calibrated analogue path
P06 8x8 ToF occupancy kiosk Pi 3, LCD, Nucleo, 53L8A1 multi-zone ranging
P07 Consumer 9-DoF plus climate Nucleo, IKS4A1 consumer MEMS and humidity
P08 Industrial high-g and dual-scale baro Nucleo, IKS5A1 simultaneous low-g and high-g
P09 NanoPi NEO Air headless hub and honest eMMC logging NanoPi NEO Air, ADXL345 the only non-Raspberry host
P10 PPK2 energy characterisation bench Pi 4, PPK2, ESP32 microamp figures
P11 Explorer700 field console Pi 3B+, Explorer700 the real-time clock and the overlay
P12 Official DSI operator glass and broker Pi 4, display, keyboard the panel and the broker
P13 Dual-radio signalling failover two Pis, two modems multi-homing with a source tag
P14 USB-TTL provisioning jig Pi 3, serial cable, two ESP boards the cable as a factory tool
P15 Mixed-voltage discipline the breadboard supply, any host the electrical safety lab
P16 ESP32 Wi-Fi/BLE sidecar Pi 4, ESP32 the sidecar pattern
P17 ESP8266 AT modem on the NanoPi NanoPi, ESP8266 the air-gap radio
P18 Nucleo USB-CDC recorder Pi 4, bare Nucleo high-rate gadget traffic
P19 SPI LCD tilt meter Pi 3, LCD, ADXL345 the display as a field instrument
P20 Four-host fleet health all four hosts orchestration, no new physics

The appendix carries the lab order and teardown, the command reference, both header pinouts, which lab owns which part, and a glossary.

Work in the order the appendix gives if the boards are bare. The rails lab is first and is not optional, and the fleet file is last so that it reflects a known sitting.

The labs' software

A lab's host side can be written and tested long before its hardware is wired, and fourteen have been, in twelve directories. Two pairs share a program. P01 and P09 share one because their chapters say the difference between the two publishers is a bus number. P07 and P08 share one because P08's third acceptance item is an item about P07: the numbers live in a different log file, and a check that spans two labs has to be written once. They live in this repository beside the chapters they belong to, because a reader who has just read chapter 4 should not have to go looking.

Path What it is
p04/ the host half of the vibration and ultrasound gateway, in C, with its own README
p04/src/ the capture format, the transform, the classification, the invariants
p04/tests/ the suite, six groups, no framework and no device
p18/ the host half of the USB-CDC recorder, in C, with its own README
p18/src/ the line reader, the capture loop, the report and the gate
p18/tests/ the suite, six groups, no framework and no device
p05/ the file half of the calibrated voltage recorder, in C, with its own README
p05/src/ the capture, the lab book, and eight items judged in four states
p05/tests/ the suite, four groups, no framework and no voltmeter
p06/ the host half of the time-of-flight occupancy kiosk, in C, with its own README
p06/src/ the frame contract, the measured wall, the mapping and the clipping
p06/tests/ the suite, six groups, and a glass file with a guard area
p01/ the shock event publisher, in C, which P09 runs with one argument changed
p01/src/ the sign extension, the dead band, the cool-off and the wear budget
p01/tests/ the suite, five groups, no framework and no sensor
p02/ the host half of the NB-IoT power-saving node, in C, with its own README
p02/src/ the two timer bitmaps, the duty cycle, the conversation
p02/tests/ the suite, six groups, no framework and no SIM
p03/ the host half of the Cat-M node with the GNSS stamp, in C, with its own README
p03/src/ the fix, the bearer, the one door that transmits, the audit
p03/tests/ the suite, five groups, no framework and no sky
p13/ the supervisor of the dual-radio failover, in C, with its own README
p13/src/ two states, one timer, and a clock that is an argument
p13/tests/ the suite, five groups, no framework and no clock
p19/ the offline tilt meter, in C, two formulas and a drawing
p19/tests/ the suite, four groups, no framework and no panel
p15/ the semaphore, in C, three output lines and a plan
p15/tests/ the suite, six groups, no framework and no chip
p10/ the energy bench, in C, two numbers a state and one comparison
p10/tests/ the suite, six groups, no framework and no instrument
p20/ the fleet sweep's judge, in C, which is not the sweep
p20/tests/ the suite, five groups, no framework and no host
p07/ the log checker for P07 and P08, in C, and the rule that two streams stay two
p07/tests/ the suite, five groups, no framework and no shield
lib/ what more than one of them needed: the AT engine, the framebuffer, the sensor, the bus, the fixed-point formatter, sequence accounting and the line reader
make -C p04 check     the suite: no hardware, no serial port, no network
make -C p04 demo      the lab's own demonstration, on synthetic data
make -C p18 check     the same, for the recorder
make -C p18 demo      a clean stream and a holed one, through the same commands
make -C p06 check     the same, for the kiosk
make -C p06 demo      measure a wall, then judge a hand against it
make -C p02 check     the same, for the power-saving node
make -C p02 demo      the timers decoded and four scenes replayed
make -C p03 check     the same, for the Cat-M node
make -C p03 demo      the scenes replayed, and the ordering audited
make -C p13 check     the same, for the failover supervisor
make -C p13 demo      five runs that did not happen, judged
make -C p05 check     the same, for the voltage recorder
make -C p05 demo      six files the board did not produce, judged
make -C p01 check     the same, for the shock publisher of P01 and P09
make -C p01 demo      five benches that were never knocked, and the budget
make -C p19 check     the same, for the tilt meter
make -C p19 demo      the acceptance test with nothing to rotate
make -C p15 check     the same, for the semaphore
make -C p15 demo      the blink plan and the draw, with no chip to drive
make -C p10 check     the same, for the energy bench
make -C p10 demo      a run that passes, and the run the lab exists to refuse
make -C p20 check     the same, for the fleet judge
make -C p20 demo      one sweep that is an artifact, and five that are not
make -C p07 check     the same, for P07 and P08's log checker
make -C p07 demo      three captures that pass, four that do not, and the pair

It is C because the lab is embedded Linux. The deliverable of a lab is the program, not a description of one. No allocation on the data path, library code that returns a status rather than exiting, and samples that stay int16_t until the one place where they become floats.

No chapter in this volume prints Python any more. The last of it went on Saturday 17 October 2026, and checkcode.py now says out loud that its Python half has nothing to check rather than printing a quiet zero that reads like a pass. The half stays, because a chapter may print Python again for orchestration, and the day it does that check should parse it.

Python here builds the book and nothing else. build.py, mdbuild.py and lint.py turn the sources into chapters, figures and the reading editions. They do not touch a bus, a device or a lab, and no lab depends on them.

Most of these labs are the same shape: something arrives over a line and a decision is made from it. Six are written, and lib/ holds what more than one of them turned out to need rather than what was guessed at in advance.

It has three files, and each went in the day a second lab needed it. The sequence accounting moved out of P04 when P18 wanted the same four numbers, and the comparison in it was not a copy, because what P04 had reads a counter wrapping as a loss of four billion samples. The line reader moved out of P18 when P06 wanted it, for a rule both chapters already stated in prose and neither could enforce: a line too long for the buffer is refused whole, since the remainder of a split line is a sample nobody sent in one lab and a partial frame on the glass in the other. The AT engine was written for P02 and P03 together, which are the only two labs that drive a radio by typing at it, and it is the one piece here that was shaped by two sets of requirements rather than by one and a later surprise.

That engine is also why those two labs can be finished before the HATs are seated. It reads a recorded conversation as readily as a serial port, and it checks each command against the one the recording expects, so replaying a session tests the order of the conversation and not only its arithmetic. P03's whole claim is an ordering, and that is how it is tested with no antenna in the room.

One of the seven is a different kind of program, and worth a sentence. P05 is the only lab here that cannot reach a pass on this bench at all: its eighth acceptance item asks that an instrument other than the board agree with the board's own reading, and there is no voltmeter in the room. So its verdict has three outcomes rather than two, and a third exit status to carry the one in the middle: every item this bench can settle passed, and one cannot be settled here by any amount of bench time. That distinction is the program's reason for existing, and it makes a cheap multimeter the single purchase that unblocks the most acceptance tests in the volume.

Three rules hold across all of them. No default threshold anywhere, because a threshold without the baseline it was measured against is not a measurement. No invented wire format, so where a vendor owns the frame layout the program says so and stops. And transport before signal, because a spectrum computed over a stream with holes in it is a picture of the holes.

Nothing here has been run against hardware, and nothing has been compiled on the authoring laptop, which has no compiler. The workflow is the first compile. Each lab's README says so in its own words.

Building

python build.py --chapter 5      one lab, PDF and self-contained HTML
python build.py --chapters       all twenty, one file each
python build.py                  the whole volume, PDF and one HTML file
python mdbuild.py                the Markdown edition, chapters and figures

Built output is not committed. The PDF and the HTML are reading editions, and they are rebuilt from this source rather than carried in it.

Checks

python prepublish.py             everything the workflow can no longer see
python tests/test_inventory.py   the parts checker, broken seven ways

The linter refuses dashes, non-ASCII inside a code block, a code line too long to print, and a lab missing any part of its skeleton or any of its four figures.

It also reads inventory.json, which is rules 2 to 5 of the front matter written as data rather than prose: every part with the header it fits, whether it claims that header exclusively, its logic voltage, and the labs that use it. From that the linter refuses a second board claiming one host's header, a 40-pin HAT on the NanoPi's 24-pin header, a voltage mismatch, two of the six instruments in one lab, and the two halves of the file disagreeing about which lab uses what. The suite breaks the file seven ways to prove the checker is awake, and five of those seven are pairings the front matter already lists as errors found in the 198-page blueprint.

What it cannot decide is anything spatial, and P19 is the standing example. The panel claims the header once and the sensor claims nothing, so the file passes, and the lab is still blocked because the panel sits physically over the pins the sensor needs. A rule engine is not a bench. The workflow in .github/workflows/ runs it on every push, checks that every lab has its four figures and its lab line, and checks that the Markdown edition is in step with the source rather than behind it.

It also refuses a STATUS.md that has lost a lab or grown a state. The four states are written, checked, run and passed, and a row whose state is anything else fails the run, because the usual way a status page rots is a new word invented in passing to avoid writing down one of the four.

Repository layout

Path What it is
chapters/ the Markdown edition, one file per lab, generated
figures/ the SVG of every figure. Its TikZ source is not published
build.py figures to SVG, the PDF, and the single-file HTML
mdbuild.py the Markdown edition
journal/ the working record, one file per lab that has one
prepublish.py the four checks the workflow lost, run before pushing
lint.py house-style check, and the parts check
inventory.json the bill of materials: parts, headers, voltages, exclusive claims
tests/test_inventory.py the parts checker broken seven ways, no framework
STATUS.md which labs have been written, checked, run and passed
SOURCE.md the cookbook this edition illustrates, transcribed
AUTHORING.md the contract every lab follows
CONTENTS.md the lab table, generated, which the table above follows
PITFALLS.md every pitfall in the volume, generated, alphabetical across labs
BOARDS.md what each board's own documentation claims, what was measured, what is open
build/ scratch output, ignored, safe to delete

Lab files use a p prefix so that a cross-reference or a copied figure can never silently resolve against a sibling volume's files. The volume's identity lives in exactly one DOC block in build.py, which refuses to run if the folder name stops matching it.

Requirements

MiKTeX or TeX Live with pdflatex, latex, dvisvgm, circuitikz, tcolorbox and listings; Python 3.10 or newer. No Python package outside the standard library is needed.

Licence

MIT, see LICENSE. Each lab's Sources section records where its prior art came from, because a reader following a vendor wiki or a licence-gated driver needs to know what that commits them to before writing code around it.

About

Twenty bench labs on one parts bin and four Linux hosts, every pairing checked against headers, voltages and silicon. One expansion board per host, one instrument per lab, and an acceptance test written to be failable.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages