An update overwrites the live boot image, so the moment it commits there
is no previous kernel left. That is the wrong shape for the failure that
actually happens.
The failure this is for
The likely failure is not a card dying mid-write — the write is verified
by a read-back, which catches that. It is a kernel that installs
perfectly, boots, and then misbehaves: a driver that hangs, a filesystem
change that cannot mount the card, a regression that only shows up
against real hardware. A verify says nothing about any of those, and the
image that worked yesterday has already been overwritten by the one that
does not work today.
Recovering from that today means pulling the card and writing a
known-good image onto it from a build tree. That has happened at least
once here, from a filesystem change that could not mount the volume it
had just been ported to.
The scheme
Write the kernel to whichever slot is not running, then move the
kernel= line in config.txt:
- Slot A is
kernel7.img / kernel8.img — deliberately the name the
firmware loads when config.txt names none. A lost or corrupt
config.txt therefore falls back to an image that is bootable by
construction rather than to nothing.
- Slot B is
kernel7b.img / kernel8b.img.
- Which is live is read from
config.txt at apply time, not tracked
in state anywhere: no kernel= line means slot A. That stays correct
after a card is swapped, imaged, or edited by hand.
- Order: write the inactive slot, verify it, then rewrite
config.txt. That write is the commit — it is one cluster, and it is
the only irreversible step.
The previous kernel survives without a copy being made, which is what
makes this cheaper than the backup file it replaces rather than more
expensive.
Recovery is a jumper, not a console
The obvious way to roll back is rpi-loader sd-read config.txt, edit one
line, sd-write. That is fine at the bench and useless in the field: it
needs a UART attached to a board that is not booting, which is harder
than swapping the card, not easier. A recovery path that costs more than
the thing it replaces will not get used.
The firmware can make the choice instead. config.txt has a GPIO
conditional filter, evaluated by start.elf while parsing the file —
before any kernel is loaded — so it can steer kernel= from the state of
a pin:
[gpio4=1]
# settings here apply if GPIO 4 is high
[gpio2=0]
# settings here apply if GPIO 2 is low
That makes recovery: power off, fit a jumper, power on. No console, no
card reader, no build tree.
The jumper is a second selector layered on the first — kernel= stays
the remote commit, because OTA has to work with nobody at the board. So
the commit writes both the live slot and the fallback:
# installer: the slot just installed
kernel=kernel8b.img
# installer: the slot that was live before it
[gpio5=0]
kernel=kernel8.img
[all]
The jumper therefore does not mean "slot A", it means boot the kernel
that worked yesterday. That keeps the alternating slots above intact,
instead of freezing slot A as a golden image that is never updated and
rots. The missing-config.txt fallback is unchanged: unfiltered slot A
is the name the firmware loads on its own, so a lost config also lands on
the previous kernel.
Choosing the pin
Rely on the power-on pull, not a gpio= directive in the same file —
that directive's ordering against filter evaluation is not something to
bet a recovery path on. GPIO 0–8 default to pull-up and 9–27 to
pull-down, so use a pull-up pin jumpered to ground: open is normal boot,
shunt is recovery, and a lost or broken jumper fails toward normal.
GPIO5 is header pin 29 and pin 30 is ground, physically adjacent — a
single 2-pin shunt, the same part as a mode jumper on any dev board.
GPIO6 (pin 31) is the same story if 5 is taken.
Two things to get right:
- The application must never drive that pin low as an output. GPIO
state survives a watchdog reset, so an app that muxes the pin and then
reboots itself would silently come back in recovery. The pin is
reserved.
- Duplicate
kernel= resolution order. The block above assumes the
last assignment wins. That is the usual config.txt behaviour and it
is the one load-bearing assumption here, so it gets confirmed on a real
board with a two-line config before the installer is written around it.
The running kernel can read the same pin, which is worth doing once the
rest works: a board that knows it is in recovery can say so — over the
network, or on the ACT LED — and can refuse to auto-apply the update that
just failed.
[tryboot], [partition=N] and bootvar0 would give a nicer version of
this, with automatic revert on the next power cycle, but they are
features of the Pi 4/5 EEPROM bootloader. The GPIO filter is the portable
one and works on the Pi 2 and 3 as well.
config.txt has to be shared, not owned
A bundle can carry config.txt — firmware options get added over time,
and a board that cannot be told disable_splash=1 without a card reader
is the situation this whole update path exists to remove. But the commit
above writes that same file.
The installer owns the slot lines and the project owns the rest. The
commit rewrites or inserts the unfiltered kernel= and the [gpio5=0]
block, into whichever config.txt is on the card after the bundle's
own copy has landed. A project's config.txt in its repository should
carry neither; if it does, the commit overwrites them, because the card
is the only thing that knows which slot is live.
That is a real parsing constraint rather than a line match: the installer
reads and rewrites the kernel= in the unfiltered region, and must not
mistake the one inside the filter block for it. Bounded, but it has to be
written down, because getting it backwards points the recovery jumper at
the image that does not boot.
One gap that needs closing with it: unlike the kernel, config.txt has
no second slot, so a bundle shipping a broken one has overwritten the
only copy. Back it up to config.bak before writing. It is about a
kilobyte — the reason a kernel backup was not worth making does not apply
at this size — and it restores the one-line-edit recovery for the file
most likely to be edited casually.
The Raspberry Pi firmware can be slotted too
config.txt supports start_file= and fixup_file=, which is the same
mechanism as kernel= for the two files that most need it: a start.elf
and a fixup.dat from different releases do not boot, and they are
replaced as a pair. Writing them as start2.elf/fixup2.dat and
pointing those two lines at them gives the previous pair the same free
survival, with the same fallback when config.txt goes missing.
If that is done, the recovery block gets the previous pair too —
otherwise the jumper goes back to yesterday's kernel under today's
firmware, which is only half a rollback.
Worth doing only after the kernel slot works, since it is the same code
with different names.
bootcode.bin has no equivalent and never will — the ROM loads it by
name, so there is no slot and no line to redirect. It is ~50 KB, it
changes almost never, and on the Pi 4 it is in SPI EEPROM and not on the
card at all, which is what makes that acceptable rather than merely
unavoidable.
Where it goes
rpi-loader-ota, behind the apply feature, beside the write/verify
path. It is not application code: every consumer wants it, none would
build it alone, and getting the ordering wrong is what leaves a board
unbootable. That half of the crate does not exist yet, so this is blocked
on it.
The pin number cannot be hard-coded there — a board that already uses
GPIO5 needs a different one — so it is a parameter of the apply call,
alongside an option to write no filter block at all for a board with no
free pin.
Not in scope
Automatic rollback. Choosing a slot by counting failed boots needs
state that survives a hang and something that runs before the kernel to
act on it, and there is none. The selection stays deliberate: a remote
kernel= edit when the board still talks, a jumper when it does not.
References
config.txt
— the rendered documentation for everything below
- Conditional filters
— the GPIO filter, and the [tryboot]/[partition]/bootvar0 filters
that are Pi 4/5 only
- Boot options
— kernel, start_file/fixup_file ("a matched pair – using
unmatched files will stop the board from booting"), os_prefix
- GPIO on Raspberry Pi
— 40-pin header numbering: GPIO5 on pin 29, ground on pin 30
- BCM2835 ARM Peripherals
— the alternate-function table carries the power-on pull state per pin
(GPIO 0–8 up, 9–27 down)
An update overwrites the live boot image, so the moment it commits there
is no previous kernel left. That is the wrong shape for the failure that
actually happens.
The failure this is for
The likely failure is not a card dying mid-write — the write is verified
by a read-back, which catches that. It is a kernel that installs
perfectly, boots, and then misbehaves: a driver that hangs, a filesystem
change that cannot mount the card, a regression that only shows up
against real hardware. A verify says nothing about any of those, and the
image that worked yesterday has already been overwritten by the one that
does not work today.
Recovering from that today means pulling the card and writing a
known-good image onto it from a build tree. That has happened at least
once here, from a filesystem change that could not mount the volume it
had just been ported to.
The scheme
Write the kernel to whichever slot is not running, then move the
kernel=line inconfig.txt:kernel7.img/kernel8.img— deliberately the name thefirmware loads when
config.txtnames none. A lost or corruptconfig.txttherefore falls back to an image that is bootable byconstruction rather than to nothing.
kernel7b.img/kernel8b.img.config.txtat apply time, not trackedin state anywhere: no
kernel=line means slot A. That stays correctafter a card is swapped, imaged, or edited by hand.
config.txt. That write is the commit — it is one cluster, and it isthe only irreversible step.
The previous kernel survives without a copy being made, which is what
makes this cheaper than the backup file it replaces rather than more
expensive.
Recovery is a jumper, not a console
The obvious way to roll back is
rpi-loader sd-read config.txt, edit oneline,
sd-write. That is fine at the bench and useless in the field: itneeds a UART attached to a board that is not booting, which is harder
than swapping the card, not easier. A recovery path that costs more than
the thing it replaces will not get used.
The firmware can make the choice instead.
config.txthas a GPIOconditional filter, evaluated by
start.elfwhile parsing the file —before any kernel is loaded — so it can steer
kernel=from the state ofa pin:
That makes recovery: power off, fit a jumper, power on. No console, no
card reader, no build tree.
The jumper is a second selector layered on the first —
kernel=staysthe remote commit, because OTA has to work with nobody at the board. So
the commit writes both the live slot and the fallback:
The jumper therefore does not mean "slot A", it means boot the kernel
that worked yesterday. That keeps the alternating slots above intact,
instead of freezing slot A as a golden image that is never updated and
rots. The missing-
config.txtfallback is unchanged: unfiltered slot Ais the name the firmware loads on its own, so a lost config also lands on
the previous kernel.
Choosing the pin
Rely on the power-on pull, not a
gpio=directive in the same file —that directive's ordering against filter evaluation is not something to
bet a recovery path on. GPIO 0–8 default to pull-up and 9–27 to
pull-down, so use a pull-up pin jumpered to ground: open is normal boot,
shunt is recovery, and a lost or broken jumper fails toward normal.
GPIO5 is header pin 29 and pin 30 is ground, physically adjacent — a
single 2-pin shunt, the same part as a mode jumper on any dev board.
GPIO6 (pin 31) is the same story if 5 is taken.
Two things to get right:
state survives a watchdog reset, so an app that muxes the pin and then
reboots itself would silently come back in recovery. The pin is
reserved.
kernel=resolution order. The block above assumes thelast assignment wins. That is the usual
config.txtbehaviour and itis the one load-bearing assumption here, so it gets confirmed on a real
board with a two-line config before the installer is written around it.
The running kernel can read the same pin, which is worth doing once the
rest works: a board that knows it is in recovery can say so — over the
network, or on the ACT LED — and can refuse to auto-apply the update that
just failed.
[tryboot],[partition=N]andbootvar0would give a nicer version ofthis, with automatic revert on the next power cycle, but they are
features of the Pi 4/5 EEPROM bootloader. The GPIO filter is the portable
one and works on the Pi 2 and 3 as well.
config.txthas to be shared, not ownedA bundle can carry
config.txt— firmware options get added over time,and a board that cannot be told
disable_splash=1without a card readeris the situation this whole update path exists to remove. But the commit
above writes that same file.
The installer owns the slot lines and the project owns the rest. The
commit rewrites or inserts the unfiltered
kernel=and the[gpio5=0]block, into whichever
config.txtis on the card after the bundle'sown copy has landed. A project's
config.txtin its repository shouldcarry neither; if it does, the commit overwrites them, because the card
is the only thing that knows which slot is live.
That is a real parsing constraint rather than a line match: the installer
reads and rewrites the
kernel=in the unfiltered region, and must notmistake the one inside the filter block for it. Bounded, but it has to be
written down, because getting it backwards points the recovery jumper at
the image that does not boot.
One gap that needs closing with it: unlike the kernel,
config.txthasno second slot, so a bundle shipping a broken one has overwritten the
only copy. Back it up to
config.bakbefore writing. It is about akilobyte — the reason a kernel backup was not worth making does not apply
at this size — and it restores the one-line-edit recovery for the file
most likely to be edited casually.
The Raspberry Pi firmware can be slotted too
config.txtsupportsstart_file=andfixup_file=, which is the samemechanism as
kernel=for the two files that most need it: astart.elfand a
fixup.datfrom different releases do not boot, and they arereplaced as a pair. Writing them as
start2.elf/fixup2.datandpointing those two lines at them gives the previous pair the same free
survival, with the same fallback when
config.txtgoes missing.If that is done, the recovery block gets the previous pair too —
otherwise the jumper goes back to yesterday's kernel under today's
firmware, which is only half a rollback.
Worth doing only after the kernel slot works, since it is the same code
with different names.
bootcode.binhas no equivalent and never will — the ROM loads it byname, so there is no slot and no line to redirect. It is ~50 KB, it
changes almost never, and on the Pi 4 it is in SPI EEPROM and not on the
card at all, which is what makes that acceptable rather than merely
unavoidable.
Where it goes
rpi-loader-ota, behind theapplyfeature, beside the write/verifypath. It is not application code: every consumer wants it, none would
build it alone, and getting the ordering wrong is what leaves a board
unbootable. That half of the crate does not exist yet, so this is blocked
on it.
The pin number cannot be hard-coded there — a board that already uses
GPIO5 needs a different one — so it is a parameter of the apply call,
alongside an option to write no filter block at all for a board with no
free pin.
Not in scope
Automatic rollback. Choosing a slot by counting failed boots needs
state that survives a hang and something that runs before the kernel to
act on it, and there is none. The selection stays deliberate: a remote
kernel=edit when the board still talks, a jumper when it does not.References
config.txt— the rendered documentation for everything below
— the GPIO filter, and the
[tryboot]/[partition]/bootvar0filtersthat are Pi 4/5 only
—
kernel,start_file/fixup_file("a matched pair – usingunmatched files will stop the board from booting"),
os_prefix— 40-pin header numbering: GPIO5 on pin 29, ground on pin 30
— the alternate-function table carries the power-on pull state per pin
(GPIO 0–8 up, 9–27 down)