[anth-volk]

Fixes #33

Summary

A Microcosm build on Modal has no Hugging Face credential, so its emitter could never authenticate to the collector. This PR is the emitter's half of the accepted design: the operator's workstation obtains a telemetry-only collector token at launch, and the emitter in the container uses it.

1. microcosm-emitter telemetry-token --lifetime-seconds N (on the workstation). It exchanges this machine's Hugging Face credential at the emitter's own resolved collector origin, with the body {"scope": "telemetry:write", "lifetime_seconds": N}. It prints exactly one JSON object on standard output, {"access_token", "expires_at", "collector_origin"}, and nothing else. expires_at is an ISO 8601 UTC time counted from before the request, so it never overstates the token's real expiry. It prints the token only on standard output and never prints the Hugging Face credential; an unexpected error exits 7 and prints only the error's type.
2. A given token replaces the Hugging Face exchange. With MICROCOSM_COLLECTOR_TOKEN, MICROCOSM_COLLECTOR_TOKEN_EXPIRES_AT and MICROCOSM_COLLECTOR_ORIGIN set, CollectorSession returns that token and never reads or exchanges a Hugging Face credential. A session for another scope (the Hugging Face module's artifacts:submit session) has no credential (missing), because the token only writes telemetry. The token is sent only to the collector origin it was obtained from (see "The Orrery module and a given token").
3. The origin variable is only checked. The session compares MICROCOSM_COLLECTOR_ORIGIN with the origin the module settings resolved (unchanged: PRODUCTION_COLLECTOR_URL, a trusted collector_origin, or a loopback development URL). A different origin, a missing variable, an expiry without a time zone, or a token that is not visible ASCII without spaces raises GivenCollectorTokenError. That stops the whole emitter, not only telemetry: whichever of the telemetry and Orrery modules the host starts first prints The Microcosm emitter cannot start: <reason>. It exits before accepting any request from the build. (naming the variable, never its value), before opening its database. The host then closes any module it had started and exits with code 1 before accepting a request, so no module of that emitter, the Hugging Face module included, serves the build; the interface's start_emitter raises EmitterStartupError (child_exit). An emitter that loads only the Hugging Face module starts, with collector submissions off.
4. Expiry and 401 leave runs pending. A given token cannot be renewed where it is used. From 30 seconds before expires_at (TOKEN_REFRESH_MARGIN_SECONDS, when an exchanged token is renewed; the collector allows no clock leeway), or after a 401 on registration or events, it is never sent again; each run stays pending with the new reason collector_token_expired or collector_token_rejected, instead of local_only, which is permanent. One line on standard error says so, and the module's drain at shutdown ends at once instead of holding the build's exit for the drain period. A later emitter with a Hugging Face login of the same user delivers the run; with 0.11.0 or later, its successful registration also clears the reason.
5. microcosm-emitter version also prints collector_origin: the origin the emitter sends to without a module setting. The hello answer is unchanged.

The command's name

telemetry-token names what the command prints and the only thing the token allows. The design called it delegate; I didn't use that, because it describes an intent rather than the output, and it suggests the emitter grants authority when the collector does. It also reads like version: a noun for what is printed.

Exit codes of telemetry-token

Exit code	Meaning
0	The token is printed
2	Usage error (a lifetime below 1, or a --development-collector-url that is not loopback); nothing was sent
3	No Hugging Face credential, or a login saved by hf auth login that this user cannot read; nothing was sent
4	The collector refused the credential (401, or 403 for a user outside the PolicyEngine organization), or the credential contains characters other than visible ASCII, such as a space or a line break, which cannot be sent as a bearer token, so it was not sent
5	The collector was unreachable, its answer was not valid HTTP, or its answer had no non-empty access_token and positive integer expires_in
6	The collector answered 422: its strict exchange body has no lifetime_seconds, so it predates tokens for other machines
7	New. An unexpected error; only the error's type is printed

A usage error prints argparse's usage text. Every other failure prints one line on standard error, without the token, the Hugging Face credential, the collector's answer body or an exception's text. An unsendable credential and an unreadable saved login each have their own fixed message. exchange_huggingface_credential checks the credential before sending it, and _http_post still refuses any bearer value that is not visible ASCII without spaces with a fixed message, so http.client never quotes one in its own error.

One exchange for the session and the command

exchange_huggingface_credential in core.auth is the only code that posts to the exchange, maps its statuses and checks a 200 answer; CollectorSession and telemetry-token both call it. Both now use the command's check: a non-empty access_token and a positive integer expires_in. For the session that is stricter than before (it defaulted a missing lifetime to an hour and read one sent as text); every collector release sends an integer expires_in. A collector answer that is not valid HTTP (http.client.HTTPException) counts as no answer.

The exchange returns ExchangedToken, whose token and lifetime are always set, or ExchangeFailure, with the reason and the HTTP status. The session maps each failure reason to its own answer through one table, _FAILED_EXCHANGE_ANSWERS, rather than passing the exchange's result on as its own. Session answers are typed: SessionCredential, a named pair of a SessionResult literal and the token, which unpacks as before and leaves the token out of its repr. The repository runs no type checker (its quality check is Ruff only), so these types were checked by reading.

An unsendable Hugging Face credential counts as refused. A credential that is not visible ASCII, such as one with a pasted space, a tab, a line break or a non-ASCII character, is never sent: the exchange answers unsendable, and so does the session. Such a value cannot be a bearer token (RFC 6750), so the collector would refuse it: before this PR the emitter sent one with a space and the collector answered 401, and one with a line break made http.client raise an error quoting the whole header value. Each caller handles it as a refused credential:

• The workstation's telemetry module makes the run local_only with huggingface_credential_rejected and keeps recording events, instead of stopping on a ValueError. Its warning, printed once, says the credential contains characters that cannot be sent as a bearer token, and where to check, rather than that it was not accepted as a PolicyEngine organization member.
• Collector submissions end the hand-over failed with CREDENTIAL_REJECTED.
• The Orrery module's jobs stay queued with credential_unsendable, and are retried as for credential_rejected.

A given collector token with such characters is still refused at start.

The Orrery module and a given token

The Orrery module receives its credential through the collector session (shared_session), and sends it to its publication_origin setting, which by default is another service, not the collector. With a given token, the session hands the token only to a caller whose destination is the collector origin it was obtained from, so Orrery sends the given token only to the emitter's resolved collector origin: with any other publication origin, its default included, the session answers withheld_from_destination, and Orrery's jobs stay queued with credential_withheld_from_destination, so a receipt says the token was withheld on purpose rather than looking like a machine without a login. Today that loses nothing, since that service cannot yet authorize a publication with a collector token.

A 401 ends a given token only for the module that received it. Modules still share one session, and so one exchanged token and its renewal. With a given token there is nothing to renew, so shared_session gives each module a session of its own: a 401 that Orrery receives from its publication origin or from the storage it uploads to no longer stops telemetry.

Where the pending reason is stored

In the run's existing local_only_reason column, while upload_state stays pending. A new column or a new upload state would need a spool migration, and released emitters that share the spool database refuse a revision they don't know. Released emitters only look at upload_state, so they deliver these runs normally.

Environment variables

All three are new. None of them chooses where the emitter sends: the destination is still resolved only from PRODUCTION_COLLECTOR_URL, the trusted collector_origin setting or a loopback development_collector_url, and MICROCOSM_COLLECTOR_ORIGIN is only compared with it.

Variable	Who sets it	Who reads it	Value
MICROCOSM_COLLECTOR_TOKEN	The Microcosm Modal runner, from telemetry-token's access_token (tracker task 6.4 · 3, not written yet)	core.auth.given_collector_token, through every CollectorSession in the emitter host. The telemetry module sends the token; the Orrery module only if it publishes to the collector origin; the Hugging Face module's submission session never	The token; never printed or logged
MICROCOSM_COLLECTOR_TOKEN_EXPIRES_AT	The same runner, from expires_at	The same	ISO 8601 time with a time zone
MICROCOSM_COLLECTOR_ORIGIN	The same runner, from collector_origin	The same; compared with the resolved origin only	An origin

Provider and wiring: unverified until the runner exists. The design passes them as a Modal secret created for each launch (modal.Secret.from_dict), exempted from the runner's credential removal, and inherited by the build and its emitter child process. Nothing in this repository sets them. The tests' credential isolation fixture now clears them, as it clears HF_TOKEN.

Depends on

• Collector task 4.3 · 1 (microcosm-collector, lifetime_seconds on the exchange, in progress) must be deployed before telemetry-token works against the deployed collector. Until then the deployed collector answers 422 to the new field and the command exits 6. The command is tested only against the loopback fake collector in tests/support/fake_collector.py, which now accepts lifetime_seconds with telemetry:write only, caps it at 24 hours, answers 422 when told to behave like an older collector, and can answer every exchange with a chosen 200 body.
• The collector's own token-expiry check and its 401 for an expired or re-signed token are assumed, not tested here.

Overlaps

Open #5, #6 and #7 (stacked) change local-only handling in telemetry/service/collector.py. This PR doesn't change their code paths: it adds two branches to _collector_token (for an unsendable credential and for an unusable given token), one call in _register's 201 path, and the new _keep_pending. Whichever merges second merges main in; the conflicts should be limited to those spots.

Tests

Red, then green, for each behaviour (Python 3.14, macOS):

Behaviour	Red commit and result	Green commit and result
version reports the collector origin	5b72282: 1 failed (no collector_origin)	f88eb20: passed
telemetry-token and its exit codes	83661c7: 7 failed (invalid choice: 'telemetry-token')	c420d4f: 7 passed
A given token replaces the Hugging Face exchange; an unusable one stops telemetry at start	24717bd: 4 failed (the Hugging Face credential was read; no ValueError)	a068d13: passed
Expiry and 401 leave runs pending for a later emitter; the drain ends at once	666c8c8: 4 failed (expired token still sent; no reason recorded; drain waited 5 s)	5446fae: passed
The session checks a 200 exchange answer as the command does (empty token, no, zero or text lifetime)	4ff2e43: 4 failed (the session answered ok; the command already exited 5)	21a0839: 4 passed
No failure prints a credential or carries it in its text; usage errors exit 2	ac205df: 2 failed (the command raised ValueError: Invalid header value quoting the credential; a given token with a line break passed the start check); the two usage cases passed	40e6660: passed
A given token counts as expired 30 s early	34a9647: 1 failed (a token with 10 s left was sent)	bf4c7dc: 3 passed
Orrery sends a given token only to the collector, and a 401 ends it only for Orrery	e85d6a7: 1 failed (Orrery got the token for its default publication origin; with that assertion removed, telemetry answered token_rejected after Orrery's 401)	5a7a450: passed
The start error says the whole emitter stops, whichever module starts first	7d4d810: 4 failed (the line said "Microcosm telemetry cannot start"; the Orrery module printed nothing)	832f6ea: 4 passed
Round 2. An unsendable Hugging Face credential is handled like a refused one: the run becomes local_only, nothing is sent, delivery keeps running	59a8e5a: 1 failed, 1 passed (ValueError from _http_post's check, raised through CollectorDelivery.flush_once)	17a9b53: 2 passed
Round 2. telemetry-token: an unsendable credential exits 4 with its own message, an unreadable saved login 3, an answer that is not HTTP 5, and an unexpected error still 7	9a5d25d: 3 failed, 14 passed (all three exited 7, with KeyError, PermissionError and BadStatusLine)	ce0c1e2: 17 passed
Round 2. Orrery's session says a given token is withheld from its publication origin	0bb0492: 1 failed ('missing' != 'withheld_from_destination')	ff9e209: passed
Round 3. The telemetry warning for an unsendable credential gives its own cause	fd1e0b5: 1 failed, 1 passed (the warning said the credential "was not accepted as a PolicyEngine organization member")	45f0149: 2 passed

Commits without a behaviour test: 7b7c052 sets the three variables through one test helper (tests/support/given_token.py); 6191d15 types the session's answers and removes the session's unused environment parameter; b080f9e words the start error; 25d40c4 updates the README, the changelog fragment and docstrings. In round 2: 25551bb splits the exchange's result into success and failure types; 96bee3e tests an expiry without a time zone and the token's absence from the session's repr (both held already); ca37144 corrects the changelog and the README. In round 3: f44f9b7 tests that collector submissions refuse an unsendable credential without sending it, and that ExchangedToken's repr leaves the token out (both held already; each test fails with the code it covers removed); a6cf212 says what an unsendable credential is and why it is refused, in the command's message, the comments, the README and the changelog, which also name the new warning and credential_unsendable.

At a6cf212: uv run --no-sync pytest -m "not collector" passed in full (809 passed, 9 deselected), and tools/check-quality.sh and tools/check-changelog.sh passed. All 10 CI checks pass on a6cf212. The collector job's pinned collector predates scopes and lifetime_seconds, so this PR adds no collector-job test.

Follow-up

#37 tracks two problems older than this PR, which it leaves as they are: a saved login that cannot be read, or a malformed collector answer during delivery, still stops the workstation telemetry module; and the exchange accepts a collector token that contains a space.

Release

The fragment is added, so merging releases 0.11.0 of both distributions to PyPI.

🤖 Generated with Claude Code