Skip to content

fix(lxc): parse cgroup2-native limit forms, not just our own emission - #144

Merged
colonelpanik merged 2 commits into
colonelpanik:mainfrom
livingstaccato:fix/lxc-resource-parse
Aug 9, 2026
Merged

fix(lxc): parse cgroup2-native limit forms, not just our own emission#144
colonelpanik merged 2 commits into
colonelpanik:mainfrom
livingstaccato:fix/lxc-resource-parse

Conversation

@livingstaccato

Copy link
Copy Markdown
Contributor

Container cgroup limits are read back from the on-disk lxc config by parseResourceConfig (internal/lxc/lxc.go), which inverted only litevirt's own ResourceConfig emission — <quota> <period> with quota/1000, and <MiB>M. But that config file is root-editable and cgroup2 accepts forms litevirt never writes, so a legally-edited config made the container's limits unreadable or wildly wrong:

  • a bare-integer memory.max (cgroup2's native unit is bytes) was read as MiB — a hand-written 256 MiB cap became a 268435456 MiB charge, which drove effective host capacity to a huge negative and refused unrelated creates on an idle host;
  • memory.max = max (unlimited) failed the whole parse, blinding the runtime-inventory probe for that container.

The parse now accepts the full cgroup2/lxc vocabulary: max → unlimited (both keys); a bare memory integer is bytes, rounded up to MiB so a charge never undercounts a cap; K/M/G/T suffixes (either case) scale; and the cpu quota is scaled by the actual period (defaulting to cgroup2's 100000 when omitted) so a hand-written period keeps its ratio. litevirt's own emission round-trips unchanged, and genuinely unparseable values still error.

All misread forms are pinned red-first in TestParseResourceConfig_CgroupNativeForms. Verified live on a 4-node lab (red→green: a raw-bytes memory.max refused a 64 MiB create on an idle host before, admits after; max parses as unlimited instead of erroring).

@livingstaccato

Copy link
Copy Markdown
Contributor Author

Heads up — I'm still working this one through (rebasing/verifying against the freshly-merged main and re-checking on the lab), so please hold off merging until I confirm it's final. I'll post when it's ready for review. Thanks!

@livingstaccato
livingstaccato force-pushed the fix/lxc-resource-parse branch from 9cc96b2 to cabd74c Compare August 7, 2026 21:25
parseResourceConfig inverted only litevirt's own ResourceConfig output —
'<quota> <period>' with quota/1000, and '<MiB>M'. The lxc config file is
root-editable, and cgroup2 accepts forms litevirt never writes: a
hand-written raw-bytes memory.max was read as MiB (a 256 MiB cap charged
as 268435456 MiB — conservative, but absurd), and cgroup2's 'max'
(unlimited) failed the whole parse, so a legally-edited config made the
container's limits unreadable.

The parse now accepts the full vocabulary: 'max' reads as unlimited for
both keys; a bare memory integer is BYTES (the cgroup2 native unit),
rounded UP to MiB so a charge never undercounts a cap; K/M/G/T suffixes
in either case scale; and the cpu quota is scaled by the ACTUAL period
(defaulting to cgroup2's 100000 when absent) so a hand-written period
keeps its ratio instead of silently mis-scaling. litevirt's own emission
round-trips unchanged, and genuinely unparseable values still error.

Observed live on the lab (2026-08-05): a hand-written raw-bytes
memory.max produced the massive overcount this closes. All misread forms
were captured red in TestParseResourceConfig_CgroupNativeForms before
the fix.
@livingstaccato
livingstaccato force-pushed the fix/lxc-resource-parse branch from acf8411 to fec9bba Compare August 8, 2026 18:53
@livingstaccato

livingstaccato commented Aug 8, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto upstream/main (de655ea) to bring this current — it now sits directly on top of #146 (pki cert-serial re-recording) and #147 (CI release idempotency). Rebase only: the fix is unchanged (still one commit, same diff), no content edits, no merge commit.

Verified green on the rebased tip: go build ./..., go vet ./..., full go test ./..., and make ci-guards (schema-bump, ledger-drift, writecheck, stmtshapecheck — 336 statements registered, docs-truth).

Drafted and posted by Claude Code.

Three ways a root-written cpu.max still resolved to a cap litevirt could not
justify, all in the same expression:

The ceiling was (quota*100 + period - 1) / period, but the only guard bounded
quota*100. Adding the period could still overflow int64 and wrap the numerator
NEGATIVE, and the division turned that back into a plausible answer instead of
an error. This is not confined to exotic input: at the cgroup2 default period
of 100000, a quota just under the guard's ceiling produced -92233720368546, and
at larger periods it produced 0 — the codebase's UNLIMITED sentinel, i.e. a
malformed cap read as no cap. The ceiling is now taken by remainder, which
cannot overflow for any pair the guard admits, and the result is bounded to
what an int holds on every platform so a 32-bit build cannot truncate a large
limit back down into a small or zero one.

A zero quota passed the negative check and divided down to 0, unlimited again,
for a value the kernel itself rejects (it enforces a minimum bandwidth). Zero
now fails with the negatives.

"max" returned before the period was parsed at all, so "max 0", "max -1" and
"max banana" were accepted as unlimited despite this branch's stated contract
that a non-positive period is rejected. The period is validated first now,
whatever the quota says; a well-formed "max" is still unlimited.

Tests assert the property rather than a list of inputs — over a sweep of quotas
and periods, a successful parse must yield a strictly positive limit — plus the
specific triggers and the representable counterpart (the largest admitted quota
over the largest period ceilings to 1, which the old expression reported as 0).
Reverting the parser leaves 24 assertions failing.

The single consumer, ContainerLimits in runtime_inventory.go, already routes a
parse error to fail(), marking the observation partial. Newly-rejected input
therefore lands on "this host's limits are unreliable", not on a false zero.
@livingstaccato

livingstaccato commented Aug 8, 2026

Copy link
Copy Markdown
Contributor Author

Adversarial review remediation — 485b7d1

An independent adversarial pass flagged four parser findings on this branch. Three were real and are fixed; one is closed as a non-defect with evidence. Every fix was mutation-verified — regression test written first, observed failing, then re-run against the restored original implementation. Reverting the parser leaves 24 assertions failing.

Verification on 485b7d1: go build ./... && go vet ./..., go test ./..., and BASE_REF=upstream/main make ci-guards all pass.

Fixed — ceiling arithmetic could overflow into a false unlimited, or a negative (the review understated this one)

The ceiling was (quota*100 + period - 1) / period, but the only guard bounded quota*100. Adding the period could still overflow int64 and wrap the numerator negative, and the division turned that back into a plausible answer rather than an error.

The review reached this only through period = MaxInt64 and described the result only as 0. Both are narrower than the real behaviour — a property sweep shows it fires at the cgroup2 default period:

quota period old result
92233720368547757 100000 -92233720368546
92233720368547758 9 -1024819115206086200
92233720368547758 MaxInt64 0 (i.e. unlimited)

So the trigger is not exotic, and the result is not always the unlimited sentinel — it can be negative, which is not a cap in any sense.

The ceiling is now taken by remainder (num/period, ++ if num%period != 0), which cannot overflow for any pair the multiplication guard admits, and the result is bounded to MaxInt32 so a 32-bit build cannot truncate a large limit back into a small or zero one.

One correction to the review's own arithmetic: 92233720368547758 over MaxInt64 is representable. Its true ceiling is 1, and the old expression's 0 was the bug — that case is now pinned as an expected value rather than an expected error.

Fixed — a zero CPU quota read as unlimited

0 100000 passed the negative check and divided down to 0 — the codebase's unlimited sentinel — for a value the kernel itself rejects, since it enforces a minimum bandwidth. Zero now fails with the negatives.

Fixedmax skipped period validation entirely

The max branch returned before the period was ever parsed, so max 0, max -1 and max banana were accepted as unlimited despite this branch's stated contract that a non-positive period is rejected.

The period is validated first now, whatever the quota says. A well-formed max is still unlimited.

Closed, not a defect — bare 0 memory cap reads as unlimited

The parse is exactly as the review describes, but the impact runs the other way.

runtime_inventory.go sets w.Uncapped = mem == 0, so a zero-byte cap is classified uncapped and trips the rogue gate — a false alarm, not the silent overcommit the Medium rating implied, and the direction this subsystem is meant to fail in. Charging 0 MiB for a container that can allocate nothing is also correct, not an undercount.

Erroring out or changing the return type would add a new failure mode to a parser whose whole job is reading configs litevirt did not write, in exchange for suppressing a rare false positive on a cgroup value that renders the container unable to allocate a single page. Not worth the blast radius.

Test approach and blast radius

Tests assert the property rather than the two named inputs: across a swept grid of quotas × periods, a successful parse must yield a strictly positive limit. Specific triggers and the representable counterpart are pinned alongside it.

Blast radius. ContainerLimits at runtime_inventory.go:158 is the only consumer, and it already routes a parse error to fail(), marking the observation partial. Newly-rejected input lands on "this host's limits are unreliable" rather than on a false zero, so no 0 = unlimited site sees a changed value — only fewer bogus ones.

Drafted and posted by Claude Code.

@livingstaccato

livingstaccato commented Aug 8, 2026

Copy link
Copy Markdown
Contributor Author

@colonelpanik — ready for review whenever you have time.

This branch is done from our side for now: the adversarial-review remediation above is the last change we plan to make, CI is green on 485b7d1 (Build & Test, Schema version guard), and the branch is rebased on current upstream/main and mergeable. No further pushes are queued, so it will not move under you mid-review.

Happy to take it further if anything in the remediation — particularly the decision to close the zero-byte memory cap as a non-defect rather than patch it — reads differently to you.

Drafted and posted by Claude Code.

@livingstaccato

Copy link
Copy Markdown
Contributor Author

I agree with the CPU remediation at 485b7d1. The remainder-based ceiling fixes the overflow property, the MaxInt64-period example correctly returns 1, and validating the period before max plus rejecting quota 0 closes the remaining false-sentinel paths.

One disposition remains: numeric memory.max = 0 is a Low-severity representation defect rather than a non-defect. It does not create silent overcommit. parseMemoryMax returns the same integer for finite zero and unlimited, then runtime_inventory.go sets w.Uncapped = mem == 0. A running, runtime-only container with a finite zero-byte cap therefore trips the uncapped gate and blocks new admission. The failure direction is alarm, and charging 0 MiB is correct, so I would keep this non-blocking for this PR.

The long-term boundary should use an explicit finite/unlimited type so numeric zero and max remain distinct. I would not turn numeric zero into a parse error.

@colonelpanik
colonelpanik merged commit 9856d49 into colonelpanik:main Aug 9, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants