firmware: move {ARMBIAN/MAINLINE}_FIRMWARE_{SOURCE/BRANCH} to main-config & pin to tag/sha1 - #10654
Conversation
…nfig & pin to tag/sha1 - single spot to set where to source firmware - armbian's still does a branch:master lookup everytime (so it auto-bumps) - mainline is now pinned to specific release (tag)'s sha1, for faster lookups - if needed new firmware lands upstream, bump the tag/sha1 manually - this should avoid the multiple terabytes of armbian-firmware-full churn
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Team Run ID: 📒 Files selected for processing (4)
💤 Files with no reviewable changes (1)
Included review availability: Your plan provides up to 8 included reviews per hour; 7 remain after this review. 📝 WalkthroughWalkthroughThe change centralizes firmware source and revision settings in global configuration. Firmware preparation and Debian packaging now use these values instead of local defaults and hardcoded revisions. ChangesFirmware source configuration
Estimated code review effort: 2 (Simple) | ~10 minutes Merge Risk: ⚪ Minimal · up to Firmware source and revision settings are centralized while mainline firmware is pinned to a specific revision. The supplied changes consistently use the shared settings, with no remaining merge-blocking risk identified. 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
github-actions
Bot
added
size/small
| # Mainline firmware git version | ||
| # used to be 'branch:master', but that caused a lot of churn. Bump this when needed. Use the latest published tag's SHA1. | ||
| # Important: use the tag's ref commit, not the the sha1 for the signed tag itself. | ||
| declare -g -r MAINLINE_FIRMWARE_BRANCH="${MAINLINE_FIRMWARE_BRANCH:-"commit:2135b2f7714a3a514c989b9728f51f36144cab6f"}" # ref: 'tag:20260810' |
There was a problem hiding this comment.
That's pretty well hidden there.
At some point we may need a dedicated file where all commit hash fixed things are present. Just thinking of drivers_network.sh.
There was a problem hiding this comment.
Agree. This is something that will likely be forgotten. Perhaps query tags and use latest tag?
Seems like a middle ground of the problem.
There was a problem hiding this comment.
Hmm. I didn't consider main-config a hidden place, but maybe if we move it to config/sources/common.conf or something?
Ref auto-updating: I was trying to be "kinder" to upstream git servers; listing tags / picking-the-newest is quite the opposite. The check runs for every board build and for every matrix-prepare. Maybe a GHA job (eg "dependabot" or similar, as we already have for Python's requirements.txt) could be enough?
There was a problem hiding this comment.
Maybe: linux-firmware does more or less a tag per month (see https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/refs/tags) -- could do a check if current YYYYMM > used-tag then WARN or something.
There was a problem hiding this comment.
does more or less a tag per month
That is what I meant. Would probably cover it best - no need to tackle by hand and still relatively solid.
There was a problem hiding this comment.
Picking this up — I'd rather resolve the newest tag than warn about a stale one, because listing tags turns out to be much cheaper than it sounds.
git ls-remote --tags --refs <url> '[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]' against linux-firmware:
- 0.16s wall
- with protocol v2 the refspec is pushed to the server (
ls-refsref-prefix), so it answers with the dated tags only — not the repo's whole ref advertisement
So the cost to upstream is one filtered round-trip per build process, which is roughly what resolving branch:main cost before. The pattern is what makes the difference; an unfiltered ls-remote --tags would indeed be the opposite of kind.
Helper, next to git_ls_remote_tag_commit_sha1 in lib/functions/general/git.sh:
# Ask the remote for the newest tag matching a glob, newest by version sort. One round-trip, and
# with protocol v2 the remote does the filtering, so it answers with a handful of refs instead of
# its whole ref advertisement.
# Echoes the tag name, or nothing if the remote has no tag matching the pattern.
# <url> <pattern>
function git_ls_remote_newest_tag() {
declare url="${1}" pattern="${2}"
declare ls_remote_output
# '|| true': finding no matching tag is a normal answer here, not an error; the caller decides.
ls_remote_output="$(git_ls_remote_logged "newest tag matching '${pattern}'" --tags --refs "${url}" "${pattern}" || true)"
echo -n "$(awk '{ sub(/^refs\/tags\//, "", $2); print $2 }' <<< "${ls_remote_output}" | sort -V | tail -1)"
}and in main-config.sh, in place of the hardcoded ref:
# Mainline firmware git ref.
# 'branch:main' meant terabytes of armbian-firmware-full churn; a hardcoded sha1 means somebody
# has to remember to bump it. Track the newest dated release tag instead - linux-firmware cuts a
# YYYYMMDD tag every few weeks. One ls-remote, filtered by refspec, so with protocol v2 the
# remote answers with a handful of refs rather than its whole advertisement (~0.2s).
# Override by setting MAINLINE_FIRMWARE_BRANCH to any 'tag:'/'branch:'/'commit:' ref.
if [[ -z "${MAINLINE_FIRMWARE_BRANCH}" ]]; then
declare newest_firmware_tag=""
if [[ "${OFFLINE_WORK}" != "yes" ]]; then
newest_firmware_tag="$(git_ls_remote_newest_tag "${MAINLINE_FIRMWARE_SOURCE}" '[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]')"
fi
if [[ -n "${newest_firmware_tag}" ]]; then
declare -g MAINLINE_FIRMWARE_BRANCH="tag:${newest_firmware_tag}"
else
# OFFLINE_WORK, or the remote had no dated tag: last known good release.
declare -g MAINLINE_FIRMWARE_BRANCH="commit:2135b2f7714a3a514c989b9728f51f36144cab6f" # ref: 'tag:20260810'
fi
display_alert "Mainline firmware ref" "${MAINLINE_FIRMWARE_BRANCH}" "info"
fi
declare -g -r MAINLINE_FIRMWARE_BRANCHThe sha1 stays as the OFFLINE_WORK fallback, so offline builds keep working and the pin never silently disappears — it just stops being the thing anyone has to remember to bump.
Tested against the live remote: returns 20260916 (current newest), and returns empty with rc=0 when nothing matches, so it is safe under errexit. bash -n and shellcheck clean.
Two things still open from above, both yours to call:
- Where it lives — happy with
config/sources/common.confifmain-configfeels hidden; the code is the same either way. ARMBIAN_FIRMWARE_GIT_SOURCE/ARMBIAN_FIRMWARE_GIT_BRANCH— this PR drops them forARMBIAN_FIRMWARE_{SOURCE,BRANCH}, and the value shape changes too (master->branch:master). Anyone setting the old names inuserpatchessilently falls back to upstreamarmbian/firmwareinstead of their fork, with no warning. Worth a compat shim, or at least adisplay_alertwhen the old names are set?
There was a problem hiding this comment.
Nice. What if we do a split:
- We use the code above for a GHA job that runs daily and determines the latest tag. We write the tag and unpeeled commit in vars to
config/sources/linux-firmware.confdirectly (overwrite, no need to parse, etc). If result if different, commit & PR. - In armbian/build we simply source
config/sources/linux-firmware.confand use it.
That way
- normal operations do not hit the upstream at all (only once, if ever, to fetch the repo).
- we get auto-updating in reasonable timeframe.
This could also be expanded to "lookup what is the latest release from GitHub releases" that we've been hitting GH api quotes for (aic8800, others, mostly extensions).
3 participants
Summary by CodeRabbit