Add a checked registry for engine environment variables - #800
Conversation
df4af78 to
5eb550c
Compare
|
Rebased on current
Six variables in two days, from three different engines, none of them mine. That is the drift rate this exists to catch, measured rather than argued — and it is the reason a registry has to be checked by a machine instead of maintained by intention. Types are taken from the call sites rather than the names: the three read through One judgement call worth flaggingTwo of the six belong to
Stale numbers in the header commentThe comment said "212 environment variables from 187 scattered getenv() call sites" — both already wrong, in precisely the way the table would have been without this check. The count is now 220. The The PR title still says 214; it is 220 now. Happy to retitle, or leave it since the count moves with Verified: |
|
Same blocker as #849, from the same cause — the branch carries the identical 16 compiled objects:
The registry itself I want to read properly rather than wave through — 214 environment variables is most of the engine's real interface, and a check that rejects an unknown name is the kind of thing that is excellent until the day someone's working |
|
Two separate things here, and I want to check the second rather than assume. 1. The objects clear themselves now. #868 is on 2. The body still opens with Asking because I have got this wrong before in the other direction. I merged #804 and #806 without noticing the same marker in the body and had to say so publicly on #801. I would rather be the person who asks a redundant question than the one who does that twice. If it is ready, rebase and say so and I will review it properly. On the substance I want to read it carefully rather than wave it through: 214 environment variables is most of the engine's real interface, and a registry that rejects unknown names is excellent right up until the day it rejects someone's working |
5eb550c to
f9bc14c
Compare
#868 closed most of this: c/COLI_V4_UNIT_*.o covers the twelve objects the amalgamated deepseek_v4 Makefile leaves next to the sources. Four more go somewhere else. c/Makefile:895: V4_OWN_DIR = build/ownership so `make check` also writes build/ownership/COLI_V4_UNIT_{RUNTIME,CONFIG, ST,NATIVE_QUANT}.o, which no rule matched. On current dev a clean checkout plus `make check` still leaves `?? c/build/` in git status -- one `git add -A` away from the same accident #868 was written to stop, and the one I made on #849 and #800. Ignoring c/build/ wholesale rather than the four names: nothing under it is tracked, and it is a build output directory, so a rule per object would need editing every time the ownership suite grows. Worth stating because it caught me on #849: an ignore rule does not untrack what is already committed. #868 made the twelve invisible in git status while leaving them in the tree; git rm --cached is what removed them. Same shape as #835. This prevents the next one, it does not clean up an existing one.
f9bc14c to
d8ca421
Compare
|
Ready for review. Rebased onto current |
d8ca421 to
0d4e648
Compare
0d4e648 to
dae908a
Compare
Two failure modes, both silent today:
COLI_PREFIL_CHUNK=512 ./colibri ... # typo
K3_BITS=8 ./colibri ... # wrong engine
Neither prints anything. The run proceeds on the default, produces entirely
plausible output, and the number that gets written into a benchmark table is
for a configuration nobody set. Anyone who has tuned this engine has lost
an afternoon to one of these.
The cause is that nothing in the tree knew which variables exist: 214 of
them, read from 187 scattered getenv() sites, with no list anywhere. So:
c/coli_env.h -- one table: name, value shape, which engines read it, and
whether it is deprecated. coli_env_check() runs once at startup in all
four engines and reports what it cannot use, with a "did you mean X?"
from an edit-distance match:
[env] unknown variable COLI_PREFIL_CHUNK -- did you mean COLI_PREFILL_CHUNK?
[env] K3_BITS is not read by colibri (it belongs to kimi_k3) -- it will have no effect
It warns, it does not exit: an unrecognised variable has never stopped a
run and making it fatal would break scripts that export a knob for
whichever engine they may launch. COLI_ENV_STRICT=1 makes it fatal for
those who want the guarantee. COLI_ENV_DUMP=1 prints every variable the
engine reads with its current value, which answers "is my export actually
arriving" without a debugger.
Variables we do not own are never touched -- only COLI_*, K3_* and INK_*
are checked, so an unrelated EDITOR in the environment is not our
business.
This is the shape vLLM arrived at (vllm/envs.py): one declaration point, one
prefix convention, validation at the boundary. Worth recording that vLLM
carries 284 of these to our 214 -- the count was never the problem in either
project, the missing registry was.
WHAT THIS DELIBERATELY DOES NOT DO
It does not rewrite the 187 getenv() call sites. That is mechanical but
large, and one mistyped default in the middle of it silently changes engine
behaviour; it belongs in its own commits, engine by engine, with this table
as the precondition for doing it safely.
So the table could drift from the code the moment it lands -- which is what
`make -C c check-env` (and a CI job) prevents. It fails on a getenv() with
no row, a row nothing reads, a duplicate, or a table that is not sorted
(coli_env_find binary-searches it, so an unsorted table silently stops
finding valid names). All three failure modes were verified by breaking the
table on purpose and confirming the check catches each one. It caught its
own first bug during development: COLI_ENV_DUMP was added to the code and
not to the table.
Only TEMP is marked deprecated -- it is the one the docs already call a
deprecated alias, and the one that actually bites, since $TEMP is the
temp-DIRECTORY path on Windows and under the ROCm runtime (JustVugg#509). SNAP,
SNAP_MIRROR and PROMPT are fallbacks but remain the mainstream spelling
(README, CONTRIBUTING's oracle command and CI all use SNAP=), so flagging
them would have printed a warning on every existing run and every CI job for
no benefit. Verified: today's usage is completely silent.
All four engines build warning-free and parse under every backend
configuration (default, CUDA, Vulkan, Metal, ANS); make test-c passes; the
Python suite is 288 tests OK (18 skipped).
Rebasing on dev turned check-env red, which is the point of it: - read by the code but missing from coli_env.h: COLI_MTP_GUARD_PCT - read by the code but missing from coli_env.h: COLI_MTP_GUARD_WINDOW - read by the code but missing from coli_env.h: COLI_V4_EXPERT_PREFETCH - read by the code but missing from coli_env.h: CUDA_EXPERT_LOAD_BALANCE - read by the code but missing from coli_env.h: INK_METAL_SHARED - read by the code but missing from coli_env.h: V4_PREFIX_LOG Types come from the call sites, not from the names: the three read through atoi() are CE_INT, INK_METAL_SHARED and COLI_V4_EXPERT_PREFETCH are tested as switches so CE_BOOL, and V4_PREFIX_LOG is presence-checked like the INK_PREFIX_LOG row already in the table, so CE_STR. Two of them belong to deepseek_v4.c, which JustVugg#165 added and which has no engine bit here, so add CE_DSV4. It is deliberately NOT folded into CE_ALL: that flag means the four engines sharing route_trace.h, rans.h and omp_tune.h, and deepseek_v4.c includes none of the three -- putting it in CE_ALL would claim every shared-header knob is read by V4 too. Also refresh the header comment, which had gone stale in the same way the table would have: 212 -> 220. The "187 scattered getenv() call sites" figure is removed rather than corrected -- nothing verifies it, so it can only rot again. `make check-env` prints the live count and is the one number in that comment that cannot go stale. check-env: 220 variables, registry matches the sources.
Rebasing again turned check-env red again, which is the argument for the
PR restating itself: COLI_V4_{AUTOPIN,DIRECT,MARKOV_BLOCK,MARKOV_KEEP,
MARKOV_SPEC,PREWARM,SAVE_USAGE}, K3_CUDA, V4_{DRAFT,MTP,MTP_DRAFT,MTP_GB,
MTP_MIN,MTP_PARTIAL_KEEP,NGRAM,NGRAM_PARTIAL_KEEP} -- most of them from
JustVugg#839's MTP/DSpark work, landed since the last rebase.
Types from the call sites: the ones tested as atoi()!=0 or atoi()==0
switches are CE_BOOL, V4_MTP_GB parses with atof() and defaults to 0.45
so CE_FLOAT, the rest read a number with atoi() so CE_INT. K3_CUDA is
CE_KIMI; every V4_/COLI_V4_ row is CE_DSV4.
Header count 220 -> 236.
dae908a to
c8425d3
Compare
ZacharyZcR
changed the title
2 participants
The problem
Environment-variable mistakes are currently silent:
Both runs proceed with defaults and can produce plausible but incorrectly labelled benchmark results. The tree had no authoritative list of the variables consumed across its engines.
What this adds
c/coli_env.hprovides one checked registry containing each variable's name, value shape, owning engine, and deprecation state.coli_env_check()runs once at engine startup:The default remains warning-only to preserve scripts that export knobs for multiple engines.
COLI_ENV_STRICT=1makes these errors fatal, whileCOLI_ENV_DUMP=1prints the variables read by the selected engine and their current values.Only project-owned prefixes and legacy project variables are checked; unrelated environment variables are ignored.
Drift prevention
This PR deliberately does not rewrite every existing
getenv()call site. That mechanical migration is larger and can silently change defaults, so it should be handled separately, engine by engine.make -C c check-envand its CI job keep the registry aligned with the source. The check rejects:getenv()with no registry rowcoli_env_find()binary searchThe registry now covers the current GLM, DeepSeek V4, Inkling, Kimi K3, OLMoE, and Qwen3.6 code paths. The exact count is intentionally left out of the title because it changes as engines evolve; the checker reports the live value.
Compatibility and deprecation
Unknown variables warn rather than exit unless strict mode is enabled. This preserves the existing runtime contract requested in review.
Only
TEMPis marked deprecated.SNAP,SNAP_MIRROR, andPROMPTremain accepted without warnings because they are active spellings in documentation and CI.Compiled objects are not part of this branch.
Verification after rebasing onto current
devmake -C c check-env: registry matches sources, 338 variablesmake -C c test-c: passedWindows UCRT64 and sanitizer jobs were still running when this description was refreshed; no current check had a terminal failure.
GPU runtime execution was not performed locally; GPU backend coverage is compile/syntax validation only.
Suggested follow-ups (not in this PR)
getenv()sites with registry lookups one engine at a time.