diff --git a/AGENTS.md b/AGENTS.md index 63eb761..c99ce88 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -106,7 +106,10 @@ may write to stdout, which must carry a request id into an error, that every telemetry marker is disclosed in README.md, that every end-to-end test file that runs the binary either sets `MAPBOX_CLI_NO_TELEMETRY=1` or points `MAPBOX_INTERNAL_TELEMETRY_URL` at loopback (so CI can't fire a real event by -accident), and that the prose is American English. +accident), that every such file points `MAPBOX_INTERNAL_CLI_TOKEN_URL` and +`MAPBOX_INTERNAL_TOKENS_URL` at loopback (so CI can't fetch or check the +CLI's token), and that the prose is American +English. They read `src/` recursively and name a file by its path under it (`output/mod.rs`), so a module that moves into a directory stays covered — diff --git a/CHANGELOG.md b/CHANGELOG.md index b43f8ad..2a71156 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,6 +19,20 @@ that may never merge. They are not releases and are not listed here. ## Unreleased +- The CLI's own token, used for telemetry when you have no token of your + own, is now fetched from `https://cli.mapbox.com/config/token.json` and + kept in `~/.mapbox/cli-token.json` instead of being built into the binary, + so it can be rotated without a release. What changes for an existing + setup: + - `MAPBOX_CLI_BUNDLED_TOKEN` is no longer read at build time. A build + from source fetches the token like a release does. + - Keeping the token creates `~/.mapbox` if it is missing, even on a run + whose only network use is telemetry. + - When an API refuses the token, the Tokens API is asked why. Only a + revoked, invalid, expired or malformed token is replaced, at once. + Three fetches in a row that find no usable token, or three refusals + in a row of a valid one, pause that for an hour. + - `tilesets query` renders as a numbered list under `-o text` again, as documented. It had printed pretty JSON since 0.2.0, when the command moved from `tilequery get` and the renderer kept matching the old name. diff --git a/README.md b/README.md index 7f0637c..f757e93 100644 --- a/README.md +++ b/README.md @@ -302,10 +302,10 @@ It never includes a file path or free text you typed. It carries a random ID that is replaced on the first run a day after it was created. It is sent in the background, so a command never waits for it, and dropped if it can't be delivered. It is sent with your own token (`--token`, `MAPBOX_ACCESS_TOKEN` -or your login) when you have one, and with a token built into the CLI +or your login) when you have one, and with the CLI's own public token otherwise; Mapbox Events keeps the token an event was sent with, and the -account it belongs to, alongside the event. A build from source has no -built-in token, so with no token of your own the event is dropped. +account it belongs to, alongside the event. With no token of your own and +no way to fetch the CLI's, the event is dropped. `MAPBOX_CLI_NO_TELEMETRY=1` or `mapbox config set telemetry off` turns it off. ### Diagnostics and settings @@ -479,8 +479,16 @@ and `--dry-run` show it alongside everything else on the request. Requests the CLI makes for itself rather than for your commands use your own token whenever you have one: `--token` or `MAPBOX_ACCESS_TOKEN`, then your login. Only when you have none does it use the CLI's token: -`MAPBOX_CLI_TOKEN`, or one built into the binary. The CLI's token is only -used for what `src/cli_token.rs` lists for it, today just telemetry. +`MAPBOX_CLI_TOKEN`, or a public token fetched from +`https://cli.mapbox.com/config/token.json` and kept in +`~/.mapbox/cli-token.json`. It is replaced only when the Tokens API says it +is revoked, invalid, expired or malformed, not when one API refuses it. A +refusal of a valid token, or one the Tokens API cannot explain, counts; the +third in a row pauses telemetry's sending with it for an hour, as three +fetches in a row that find no usable token pause fetching. The token is +restricted to `https://cli.mapbox.com/`, so every request with it carries +that as its `Referer`. The CLI's token is only used for what +`src/cli_token.rs` lists for it, today just telemetry. ### Proxies diff --git a/build.rs b/build.rs index e22d60c..41451f3 100644 --- a/build.rs +++ b/build.rs @@ -41,8 +41,6 @@ use std::path::Path; fn main() { - check_bundled_token(); - let manifest = Path::new("../internal/openapi-command-config/PINNED_SOURCE"); // The vendored specs and their provenance are ordinary build inputs now, @@ -93,20 +91,6 @@ fn main() { warn_if_stale(committed_at); } -/// `src/cli_token.rs` compiles `MAPBOX_CLI_BUNDLED_TOKEN` into the binary, -/// where `strings` can read it. Only a public `pk.` token may go there, so -/// anything else fails the build rather than shipping a secret. -fn check_bundled_token() { - println!("cargo:rerun-if-env-changed=MAPBOX_CLI_BUNDLED_TOKEN"); - let Ok(token) = std::env::var("MAPBOX_CLI_BUNDLED_TOKEN") else { - return; - }; - let token = token.trim(); - if !token.is_empty() && !token.starts_with("pk.") { - panic!("MAPBOX_CLI_BUNDLED_TOKEN must be a public pk. token"); - } -} - /// New API surface lands in `openapi-specs` at something closer to a monthly /// rate, and the maintainer-only sync that regenerates `openapi/` from it /// runs weekly, so two weeks without one is already long enough to be worth a diff --git a/docs/architecture.md b/docs/architecture.md index b2f858f..3ef0fe0 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -209,7 +209,9 @@ for the same `ps`-visibility reason Diagram 3 gives. Step ②'s token follows `src/cli_token.rs`'s fallback order: the user's own token first (flag, environment, or stored login), and only when none of -those exist, a `pk.` token compiled into the binary at build time -(`MAPBOX_CLI_BUNDLED_TOKEN`) as a last resort so usage from a user who has -never logged in is still counted. A `cargo build` from source has no bundled -token, so with no login either, the event is simply dropped. +those exist, the CLI's `pk.` token as a last resort so usage from a user who +has never logged in is still counted. Every build fetches that token from +`https://cli.mapbox.com/config/token.json` and keeps it in +`~/.mapbox/cli-token.json` until the Tokens API says it is dead +(`src/cli_token/fetch.rs`). With no login and no way to fetch it, the event +is simply dropped. diff --git a/src/auth.rs b/src/auth.rs index f497e2f..5960e04 100644 --- a/src/auth.rs +++ b/src/auth.rs @@ -36,7 +36,7 @@ const TOKEN_ENDPOINT: &str = "https://api.mapbox.com/oauth/2.1/token"; const REGISTRATION_ENDPOINT: &str = "https://api.mapbox.com/oauth/register"; /// Where `--verify` sends the token. Reached with the token as a query /// parameter, so every failure path out of it has to redact. -const VALIDATION_ENDPOINT: &str = "https://api.mapbox.com/tokens/v2"; +pub(crate) const VALIDATION_ENDPOINT: &str = "https://api.mapbox.com/tokens/v2"; // Every scope here must be one the Accounts API's dynamic client // registration (DCR) will actually grant — a scope it doesn't recognize is @@ -1317,36 +1317,72 @@ fn verify_token(token: &str, debug: bool, timeout: Option) -> Result"); } - - let response = crate::http::send( - crate::http::client()? - .get(VALIDATION_ENDPOINT) - .query(&[("access_token", token)]) - // The one request `auth` makes to a Mapbox API rather than to the - // authorization server, so it is the one `--timeout` has to reach. - // The other three — refresh, registration, code exchange — carry a - // few hundred bytes each and keep the client's own budget. - .timeout(crate::http::budget(timeout, crate::http::Payload::Bounded)), + // The one request `auth` makes to a Mapbox API rather than to the + // authorization server, so it is the one `--timeout` has to reach. The + // other three — refresh, registration, code exchange — carry a few + // hundred bytes each and keep the client's own budget. + check_token( + token, + crate::http::budget(timeout, crate::http::Payload::Bounded), ) - // `reqwest::Error`'s `Display` appends the URL, and the token is in it. - .map_err(|e| crate::executor::transport_failure("Token check failed", e))?; - - let status = response.status(); - // Before `text()` consumes the response — see `executor::request_id`. - let request_id = crate::executor::request_id(response.headers()); - let body = response - .text() - .map_err(|e| crate::executor::transport_failure("Failed to read the token check", e))?; +} +/// The request behind [`verify_token`]: the endpoint's answer, +/// whose `code` is the verdict. +fn check_token(token: &str, timeout: Duration) -> Result { + let (status, request_id, body) = send_token_check(VALIDATION_ENDPOINT, token, None, timeout)?; if !status.is_success() { return Err(CliError::http(status.as_u16(), &body) .with_request_id(request_id) .into()); } - serde_json::from_str(&body).context("Invalid JSON from the token check endpoint") } +/// The verdict's `code` alone, read whatever the status, or `None` when there +/// was no answer with one. How [`crate::cli_token`] tells a dead token from +/// one an API merely refuses. Observed answering `200` for every verdict, but +/// a dead token's `code` must not be lost if that ever changes, or the token +/// would never be replaced. +/// +/// `referer` is sent along, for a token restricted to a URL. +pub(crate) fn token_code( + url: &str, + token: &str, + referer: Option<&str>, + timeout: Duration, +) -> Option { + let (_, _, body) = send_token_check(url, token, referer, timeout).ok()?; + let answer: Value = serde_json::from_str(&body).ok()?; + Some(answer.get("code")?.as_str()?.to_owned()) +} + +fn send_token_check( + url: &str, + token: &str, + referer: Option<&str>, + timeout: Duration, +) -> Result<(reqwest::StatusCode, Option, String)> { + let mut request = crate::http::client()? + .get(url) + .query(&[("access_token", token)]) + .timeout(timeout); + if let Some(referer) = referer { + request = request.header(reqwest::header::REFERER, referer); + } + let response = crate::http::send(request) + // `reqwest::Error`'s `Display` appends the URL, and the token is in it. + .map_err(|e| crate::executor::transport_failure("Token check failed", e))?; + + let status = response.status(); + // Before `text()` consumes the response — see `executor::request_id`. + let request_id = crate::executor::request_id(response.headers()); + let body = response + .text() + .map_err(|e| crate::executor::transport_failure("Failed to read the token check", e))?; + Ok((status, request_id, body)) +} + /// A verdict from [`verify_token`] that is not `TokenValid`. /// /// Each code becomes its own error code: telling a revoked token from a typo diff --git a/src/cli_token.rs b/src/cli_token.rs index f4648ac..7c2c37e 100644 --- a/src/cli_token.rs +++ b/src/cli_token.rs @@ -11,10 +11,9 @@ //! credentials lock, only when it is about to expire. //! 3. `MAPBOX_CLI_TOKEN` at run time, the CLI's token for someone with none //! of their own, such as a developer pointing at staging. -//! 4. The token compiled in from `MAPBOX_CLI_BUNDLED_TOKEN` at build time. A -//! plain `cargo build` has none. It is a different variable from the -//! run-time one so that a token exported in a dev shell is not baked into -//! every local build. +//! 4. The token published on the release channel, fetched and kept on disk +//! until the Tokens API calls it dead. See [`fetch`] for when it is +//! fetched again. //! //! A `401` moves on to the next token; any other answer is the caller's. //! [`send`]'s callers never see the token, so every one of them gets the same @@ -26,22 +25,34 @@ //! billed to the account that owns it. So it is decided by what the request //! is for, not where it goes: a request the CLI makes for itself passes //! [`send`] an order that ends in [`Fallback::CliToken`], and every such order -//! is defined in this file (today only [`TELEMETRY`]); a command must be in -//! [`CLI_TOKEN_COMMANDS`]. Where the request is sent does not matter, so a URL -//! overridden from the environment keeps the same fallback. +//! is defined in this file (today only [`TELEMETRY`]). No command uses it. +//! Where the request is sent does not matter, so a URL overridden from the +//! environment keeps the same fallback. //! -//! # The bundled token is public +//! # The token is public //! -//! Anything compiled into a distributed binary can be pulled out with -//! `strings`. The bundled token must therefore be a dedicated `pk.` token with -//! the minimum scopes, and nothing may ever treat it as a secret. `build.rs` +//! It is served from a public URL, so it must be a dedicated `pk.` token with +//! the minimum scopes, and nothing may ever treat it as a secret. [`fetch`] //! refuses anything that is not `pk.`. +use std::cell::RefCell; + use reqwest::blocking::{RequestBuilder, Response}; use reqwest::StatusCode; use crate::{auth, http}; +mod fetch; + +use fetch::Refusal; + +/// The `Referer` every request with the CLI's token carries. The published +/// token is restricted to this URL, so that a web page elsewhere cannot use +/// it; without the header, the API answers `401`. It keeps nothing out of a +/// program that sets the header itself, which is all the restriction can do +/// for a token anyone can read. +pub(crate) const REFERER: &str = "https://cli.mapbox.com/"; + /// One place [`send`] may take a token from. A caller passes them in the /// order to try. #[derive(Clone, Copy, Debug, PartialEq, Eq)] @@ -50,11 +61,14 @@ pub(crate) enum Fallback { UserToken, /// The stored login, refreshed only when it is about to expire. Login, - /// `MAPBOX_CLI_TOKEN` at run time, then the bundled token. Billed to the + /// `MAPBOX_CLI_TOKEN` at run time, then the fetched token. Billed to the /// CLI's account, so it only ever comes after every token of the user's. CliToken, } +/// Telemetry's name as a caller of the CLI's token, for its cooldown. +pub(crate) const TELEMETRY_CALLER: &str = "telemetry"; + /// Telemetry's order. The CLI's own usage data, so there is no user's /// account to bill when the user has no token. pub(crate) const TELEMETRY: &[Fallback] = @@ -65,33 +79,6 @@ pub(crate) const TELEMETRY: &[Fallback] = #[cfg(test)] const ORDERS: &[&[Fallback]] = &[TELEMETRY]; -/// Commands that may run on the CLI's token when the user has none, by the -/// name `--schema` reports, each with the reason the CLI's account should -/// carry their use. Command names are a compatibility promise, so an entry -/// does not quietly move to another API when a spec changes its URL. -const CLI_TOKEN_COMMANDS: &[(&str, &str)] = &[]; - -/// The CLI's token for a command's request, when the user has none of their -/// own and the command is in [`CLI_TOKEN_COMMANDS`]. -/// -/// Only for a user with no token at all: one whose token is rejected sees that -/// rejection rather than having it hidden behind the CLI's token. -pub(crate) fn for_command(op: &crate::spec::Operation) -> Option { - let command = op.command(); - if CLI_TOKEN_COMMANDS.iter().any(|(name, _)| *name == command) { - cli_token(std::env::var("MAPBOX_CLI_TOKEN").ok().as_deref()) - } else { - None - } -} - -/// `MAPBOX_CLI_TOKEN` at run time, then the bundled token. -fn cli_token(override_token: Option<&str>) -> Option { - usable(override_token) - .or_else(|| usable(option_env!("MAPBOX_CLI_BUNDLED_TOKEN"))) - .map(str::to_owned) -} - /// The token the user gave this run, ranked as a command ranks it: with /// `--use-login` only a typed `--token` counts, otherwise the environment /// does too, `MapboxAccessToken` included, since that is the user's token @@ -120,23 +107,66 @@ pub(crate) fn user_token(matches: &clap::ArgMatches) -> Option { /// request could not be sent at all; otherwise the first answer that was not /// a `401`, or the last `401` when every token was rejected. pub(crate) fn send( + caller: &str, order: &[Fallback], user_token: Option<&str>, profile: Option<&str>, request: impl Fn() -> RequestBuilder, ) -> Option> { let override_token = std::env::var("MAPBOX_CLI_TOKEN").ok(); - resolve( + let fetched = RefCell::new(None); + let dead = RefCell::new(None); + let mut attempt = |token: &str| { + let is_cli_token = fetched.borrow().as_deref() == Some(token) + || usable(override_token.as_deref()) == Some(token); + let request = request().query(&[("access_token", token)]); + let request = if is_cli_token { + request.header(reqwest::header::REFERER, REFERER) + } else { + request + }; + let response = http::send(request); + // Only the fetched token's answers are kept: the user's tokens, and + // `MAPBOX_CLI_TOKEN`, are never checked or counted. + if let (Ok(r), true) = (&response, fetched.borrow().as_deref() == Some(token)) { + if r.status().is_success() { + fetch::accepted(caller, token); + } else if is_refusal(r.status()) && fetch::refused(caller, token) == Refusal::Dead { + *dead.borrow_mut() = Some(token.to_owned()); + } + } + response + }; + let fetch = || { + // A caller resting from the CLI's token sends nothing with it. + let token = (!fetch::cooling(caller)).then(fetch::token)??; + *fetched.borrow_mut() = Some(token.clone()); + Some(token) + }; + let answer = resolve( order, Sources { user: user_token, override_token: override_token.as_deref(), - bundled: option_env!("MAPBOX_CLI_BUNDLED_TOKEN"), }, || login_token(profile), - |token| http::send(request().query(&[("access_token", token)])), + fetch, + &mut attempt, |response| response.status() == StatusCode::UNAUTHORIZED, - ) + ); + // A dead token is replaced at once, and the request sent once more. + match dead.take() { + Some(dead) if matches!(&answer, Some(Ok(r)) if is_refusal(r.status())) => { + match fetch::token() { + Some(fresh) if fresh != dead => { + *fetched.borrow_mut() = Some(fresh.clone()); + Some(attempt(&fresh)) + } + _ => answer, + } + } + _ => answer, + } } /// Whether [`send`] has a token it could try in `order`, judged without the @@ -144,6 +174,7 @@ pub(crate) fn send( /// find nothing to send with. A login counts by being there: it may still turn /// out expired and unrefreshable, and then nothing is sent. pub(crate) fn has_token_for( + caller: &str, order: &[Fallback], user_token: Option<&str>, profile: Option<&str>, @@ -152,7 +183,8 @@ pub(crate) fn has_token_for( Fallback::UserToken => usable(user_token).is_some(), Fallback::Login => auth::load_credentials_readonly(profile).is_some(), Fallback::CliToken => { - cli_token(std::env::var("MAPBOX_CLI_TOKEN").ok().as_deref()).is_some() + usable(std::env::var("MAPBOX_CLI_TOKEN").ok().as_deref()).is_some() + || (!fetch::cooling(caller) && fetch::may_have_token()) } }) } @@ -173,39 +205,53 @@ fn login_token(profile: Option<&str>) -> Option { struct Sources<'a> { user: Option<&'a str>, override_token: Option<&'a str>, - bundled: Option<&'a str>, } -/// The order and the fallback, with the login and the send passed in so each -/// path can be tested without the network or the disk. +enum Source { + User, + Login, + Override, + Fetched, +} + +/// The order and the fallback, with the login, the fetched token and the +/// send passed in so each path can be tested without the network or the +/// disk. /// -/// Tries each [`Fallback`] in `order`. The login is loaded only when its turn -/// comes, and a token that was already rejected is not tried again. A -/// failed send stops here: no other token fixes a request that never -/// arrived. +/// Tries each [`Fallback`] in `order`. The login and the fetched token are +/// loaded only when their turn comes, and a token that was already rejected +/// is not tried again. A failed send stops here: no other token fixes a +/// request that never arrived. fn resolve( order: &[Fallback], sources: Sources<'_>, login: impl FnOnce() -> Option, + fetched: impl FnOnce() -> Option, mut attempt: impl FnMut(&str) -> Result, rejected: impl Fn(&R) -> bool, ) -> Option> { let mut login = Some(login); + let mut fetched = Some(fetched); let mut tried: Vec = Vec::new(); let mut last = None; for fallback in order { - let tokens: Vec = match fallback { - Fallback::UserToken => usable(sources.user) - .map(str::to_owned) - .into_iter() - .collect(), - Fallback::Login => login.take().and_then(|load| load()).into_iter().collect(), - Fallback::CliToken => [sources.override_token, sources.bundled] - .into_iter() - .filter_map(|t| usable(t).map(str::to_owned)) - .collect(), + // One source per step, each loaded only when the one before it was + // rejected, so a developer's override never costs a fetch. + let steps: &[Source] = match fallback { + Fallback::UserToken => &[Source::User], + Fallback::Login => &[Source::Login], + Fallback::CliToken => &[Source::Override, Source::Fetched], }; - for token in tokens { + for source in steps { + let token = match source { + Source::User => usable(sources.user).map(str::to_owned), + Source::Login => login.take().and_then(|load| load()), + Source::Override => usable(sources.override_token).map(str::to_owned), + Source::Fetched => fetched.take().and_then(|load| load()), + }; + let Some(token) = token else { + continue; + }; if tried.contains(&token) { continue; } @@ -223,6 +269,11 @@ fn resolve( last.map(Ok) } +/// A `401` or `403`: the answers [`fetch::refused`] can explain. +pub(crate) fn is_refusal(status: StatusCode) -> bool { + status == StatusCode::UNAUTHORIZED || status == StatusCode::FORBIDDEN +} + fn usable(token: Option<&str>) -> Option<&str> { token.map(str::trim).filter(|t| !t.is_empty()) } @@ -235,7 +286,7 @@ mod tests { /// Runs [`resolve`] against a fake API that accepts only `accepted`, and /// returns the answer, every token sent, and whether the login was loaded. fn run( - sources: Sources<'_>, + sources: (Sources<'_>, Option<&str>), login: Option<&str>, accepted: &[&str], ) -> (Option, Vec, bool) { @@ -244,7 +295,7 @@ mod tests { fn run_in( order: &[Fallback], - sources: Sources<'_>, + (sources, fetched): (Sources<'_>, Option<&str>), login: Option<&str>, accepted: &[&str], ) -> (Option, Vec, bool) { @@ -257,6 +308,7 @@ mod tests { loaded.set(true); login.map(str::to_owned) }, + || fetched.map(str::to_owned), |token| { sent.borrow_mut().push(token.to_owned()); Ok::<_, ()>(if accepted.contains(&token) { 200 } else { 401 }) @@ -266,22 +318,25 @@ mod tests { (answer.map(|a| a.unwrap()), sent.into_inner(), loaded.get()) } + /// The user's token, `MAPBOX_CLI_TOKEN`, and what a fetch would return. fn sources<'a>( user: Option<&'a str>, override_token: Option<&'a str>, - bundled: Option<&'a str>, - ) -> Sources<'a> { - Sources { - user, - override_token, - bundled, - } + fetched: Option<&'a str>, + ) -> (Sources<'a>, Option<&'a str>) { + ( + Sources { + user, + override_token, + }, + fetched, + ) } #[test] fn an_accepted_user_token_is_all_that_is_sent() { let (answer, sent, loaded) = run( - sources(Some("pk.typed"), Some("pk.override"), Some("pk.bundled")), + sources(Some("pk.typed"), Some("pk.override"), Some("pk.fetched")), Some("sk.login"), &["pk.typed"], ); @@ -314,7 +369,7 @@ mod tests { #[test] fn the_clis_token_is_sent_only_once_every_user_token_is_rejected() { let (answer, sent, _) = run( - sources(Some("pk.typed"), Some("pk.override"), Some("pk.bundled")), + sources(Some("pk.typed"), Some("pk.override"), Some("pk.fetched")), Some("sk.login"), &["pk.override"], ); @@ -326,9 +381,9 @@ mod tests { fn the_clis_token_is_never_sent_by_an_order_without_it() { let (answer, sent, _) = run_in( &[Fallback::UserToken, Fallback::Login], - sources(Some("pk.typed"), Some("pk.override"), Some("pk.bundled")), + sources(Some("pk.typed"), Some("pk.override"), Some("pk.fetched")), None, - &["pk.override", "pk.bundled"], + &["pk.override", "pk.fetched"], ); assert_eq!(answer, Some(401)); assert_eq!(sent, ["pk.typed"]); @@ -355,11 +410,11 @@ mod tests { } #[test] - fn override_comes_before_bundled() { + fn override_comes_before_fetched() { let (answer, sent, _) = run( - sources(None, Some("pk.override"), Some("pk.bundled")), + sources(None, Some("pk.override"), Some("pk.fetched")), None, - &["pk.override", "pk.bundled"], + &["pk.override", "pk.fetched"], ); assert_eq!(answer, Some(200)); assert_eq!(sent, ["pk.override"]); @@ -369,23 +424,23 @@ mod tests { fn a_rejected_token_is_not_sent_twice() { // The login and the typed token can be the same token. let (answer, sent, _) = run( - sources(Some("sk.same"), None, Some("pk.bundled")), + sources(Some("sk.same"), None, Some("pk.fetched")), Some("sk.same"), - &["pk.bundled"], + &["pk.fetched"], ); assert_eq!(answer, Some(200)); - assert_eq!(sent, ["sk.same", "pk.bundled"]); + assert_eq!(sent, ["sk.same", "pk.fetched"]); } #[test] fn every_token_rejected_returns_the_last_answer() { let (answer, sent, _) = run( - sources(Some("pk.typed"), Some("pk.override"), Some("pk.bundled")), + sources(Some("pk.typed"), Some("pk.override"), Some("pk.fetched")), Some("sk.login"), &[], ); assert_eq!(answer, Some(401)); - assert_eq!(sent, ["pk.typed", "sk.login", "pk.override", "pk.bundled"]); + assert_eq!(sent, ["pk.typed", "sk.login", "pk.override", "pk.fetched"]); } #[test] @@ -393,8 +448,9 @@ mod tests { let mut sent = Vec::new(); let answer = resolve( TELEMETRY, - sources(Some("pk.typed"), Some("pk.override"), Some("pk.bundled")), + sources(Some("pk.typed"), Some("pk.override"), None).0, || Some("sk.login".to_owned()), + || Some("pk.fetched".to_owned()), |token| { sent.push(token.to_owned()); Err::("unreachable") @@ -408,12 +464,12 @@ mod tests { #[test] fn blank_tokens_are_skipped_and_the_rest_trimmed() { let (answer, sent, _) = run( - sources(Some(" "), Some(""), Some(" pk.bundled\n")), + sources(Some(" "), Some(""), Some("pk.fetched")), None, - &["pk.bundled"], + &["pk.fetched"], ); assert_eq!(answer, Some(200)); - assert_eq!(sent, ["pk.bundled"]); + assert_eq!(sent, ["pk.fetched"]); } #[test] @@ -462,33 +518,31 @@ mod tests { } #[test] - fn every_cli_token_command_exists_and_says_why() { - let specs = crate::spec::effective_services().expect("the bundled specs parse"); - let commands: Vec = specs - .iter() - .flat_map(|svc| &svc.operations) - .map(|op| op.command()) - .collect(); - for (name, reason) in CLI_TOKEN_COMMANDS { - assert!( - commands.iter().any(|c| c == name), - "`{name}` is in CLI_TOKEN_COMMANDS but is not a command" - ); - assert!(!reason.trim().is_empty(), "`{name}` needs a reason"); - } + fn the_override_is_trimmed_before_it_is_sent() { + let (answer, sent, _) = run( + sources(None, Some(" pk.override\n"), None), + None, + &["pk.override"], + ); + assert_eq!(answer, Some(200)); + assert_eq!(sent, ["pk.override"]); } #[test] - fn the_runtime_override_comes_before_the_bundled_token() { - assert_eq!( - cli_token(Some(" pk.override ")).as_deref(), - Some("pk.override") - ); - assert_eq!( - cli_token(Some(" ")).as_deref(), - option_env!("MAPBOX_CLI_BUNDLED_TOKEN") - .map(str::trim) - .filter(|t| !t.is_empty()) + fn an_accepted_override_never_fetches() { + let fetched = Cell::new(false); + let answer = resolve( + &[Fallback::CliToken], + sources(None, Some("pk.override"), None).0, + || None, + || { + fetched.set(true); + Some("pk.fetched".to_owned()) + }, + |_| Ok::<_, ()>(200), + |status| *status == 401, ); + assert_eq!(answer, Some(Ok(200))); + assert!(!fetched.get()); } } diff --git a/src/cli_token/fetch.rs b/src/cli_token/fetch.rs new file mode 100644 index 0000000..6ae04d3 --- /dev/null +++ b/src/cli_token/fetch.rs @@ -0,0 +1,687 @@ +//! Where the CLI's token comes from: fetched from the release channel, and +//! kept on disk until the Tokens API calls it dead. +//! +//! The token is published at `https://cli.mapbox.com/config/token.json` as +//! `{"token": "pk...."}`, so rotating it needs no release. It is fetched +//! when there is none on disk or the one there is dead, and saved to +//! `/cli-token.json`. +//! +//! # When the endpoint has nothing new +//! +//! A `401` or `403` alone does not say the token is dead: the API may only be +//! refusing it for one endpoint. So [`refused`] asks the Tokens API about +//! the token itself, and only a revoked, invalid, expired or malformed one is +//! replaced. A fetch right after a +//! rotation can still return the revoked token: the edge caches the file for +//! a few minutes, and the secret may not be republished yet. So the dead +//! token served again is checked once more, since a token called invalid for +//! something fixed in the Console stays the same string; it is taken back if +//! the Tokens API now calls it valid. When fetches answer with nothing usable +//! three times running, the next waits an hour ([`backoff`]). A fetch that +//! never got an answer +//! does not count: that is this machine being offline, not the endpoint +//! being stale, and it is asked again on the next run. +//! +//! # Where it is fetched from +//! +//! Every build fetches from production, a local `cargo build` included, +//! since the token is a production token whatever the channel. [`URL_ENV`] +//! points a build that is not a production release somewhere else, which is +//! how the tests stay off the network: every end-to-end test sets it +//! (`tests/source_guards.rs` holds that). +//! +//! The state file is what makes all of this hold across runs, so writing it +//! creates the config directory when there is none — a run that never needs +//! the CLI's token, or only fails to reach the endpoint, still leaves no +//! trace. See [`write_state`]. + +use std::collections::BTreeMap; +use std::path::PathBuf; +use std::time::{Duration, SystemTime, UNIX_EPOCH}; + +use serde::{Deserialize, Serialize}; + +use crate::{auth, http}; + +const PRODUCTION_URL: &str = "https://cli.mapbox.com/config/token.json"; + +/// Asked about a refused token's own state; see [`refused`]. +const TOKENS_URL_ENV: &str = "MAPBOX_INTERNAL_TOKENS_URL"; + +/// Overrides [`PRODUCTION_URL`] in a build that is not a production release. +/// A release ignores it, so an injected variable cannot choose the token a +/// release sends. +pub(crate) const URL_ENV: &str = "MAPBOX_INTERNAL_CLI_TOKEN_URL"; + +const STATE_FILE: &str = "cli-token.json"; + +/// A small JSON file on a CDN; nobody should wait longer for it. +const FETCH_TIMEOUT: Duration = Duration::from_secs(5); + +const HOUR: u64 = 60 * 60; +/// Misses in a row before an hour's wait. +const TRIES_BEFORE_WAITING: u32 = 3; + +/// Long enough for any `pk.` token; anything longer is not one. +const MAX_TOKEN_LEN: usize = 512; + +/// What a fetch got back, before [`acquire`] judges it. +#[derive(Debug)] +pub(crate) enum Fetched { + /// The body's `token`, unchecked. + Token(String), + /// An answer with no token in it: a bad status or a bad body. + Nothing, + /// No answer at all. + Failed, +} + +/// What is kept on disk between runs. +#[derive(Debug, Default, Clone, PartialEq, Eq, Serialize, Deserialize)] +struct State { + #[serde(default, skip_serializing_if = "Option::is_none")] + token: Option, + /// Whether the Tokens API called `token` dead. A `401` or `403` alone + /// never sets it: the API may only be refusing it for one endpoint. + /// Cleared if the endpoint serves it again and it now checks valid. + #[serde(default)] + rejected: bool, + /// Who is resting from using `token`, by the name [`cooling`] takes: + /// an API refused it while the Tokens API called it valid. Cleared with + /// a new token, which that API may accept. + #[serde(default, skip_serializing_if = "BTreeMap::is_empty")] + cooldowns: BTreeMap, + /// Whether it was written with an endpoint overridden ([`overridden`]). + /// A dev build pointed at staging and an installed release share this + /// file, and neither is to trust the other's token or verdicts. + #[serde(default)] + overridden: bool, + /// Consecutive fetches that found nothing usable. + #[serde(default)] + failures: u32, + /// Unix seconds before which the endpoint is not asked. + #[serde(default)] + retry_at: u64, +} + +impl State { + fn usable_token(&self) -> Option<&str> { + self.token + .as_deref() + .filter(|token| !self.rejected && is_plausible(token)) + } + + fn fail(&mut self, now: u64) { + self.failures += 1; + self.retry_at = now + backoff(self.failures); + } +} + +/// One caller's rest from the CLI's token, after an API refused it and the +/// Tokens API could not call it dead, paced as the fetch is ([`backoff`]). +/// Never longer than an hour: a token that is valid but refused is a +/// configuration someone can fix in the Console without the token changing, +/// and a fix should not wait long to be seen. +#[derive(Debug, Default, Clone, PartialEq, Eq, Serialize, Deserialize)] +struct Cooldown { + /// Consecutive refusals. + #[serde(default)] + failures: u32, + /// Unix seconds before which it is not used again. + #[serde(default)] + until: u64, +} + +/// The wait after `failures` consecutive misses: an hour after every third, +/// none otherwise, so a miss that was only bad timing costs nothing. +fn backoff(failures: u32) -> u64 { + if failures.is_multiple_of(TRIES_BEFORE_WAITING) { + HOUR + } else { + 0 + } +} + +/// Whether `caller` — today only `telemetry` — is resting from the +/// CLI's token. While it is, it sends nothing with it, so nothing is +/// refused, and nothing is checked. +pub(crate) fn cooling(caller: &str) -> bool { + read_state().is_some_and(|state| { + state + .cooldowns + .get(caller) + .is_some_and(|cooldown| holds(cooldown.until, now())) + }) +} + +/// Records that `caller`'s request with the saved `token` went through, which +/// ends any cooldown it was counting. +pub(crate) fn accepted(caller: &str, token: &str) { + if let Some(mut state) = read_state().filter(|state| state.token.as_deref() == Some(token)) { + if state.cooldowns.remove(caller).is_some() { + write_state(&state); + } + } +} + +/// The CLI's token: the one on disk while it is not dead, otherwise a fresh +/// one from the endpoint unless it is being waited for. `None` when there is +/// none to be had right now. +pub(crate) fn token() -> Option { + let mut state = read_state().unwrap_or_default(); + let before = state.clone(); + let token = acquire(&mut state, now(), || get(&endpoint()), check); + if state != before { + save_fetch(&before, state); + } + token +} + +/// Writes what a fetch decided, unless another run decided first. The fetch +/// can take seconds: if the token, its verdict or the count changed on disk +/// meanwhile, that run's answer stands. Cooldowns another run added are +/// kept, unless this fetch brought a new token, which ends them all. +fn save_fetch(before: &State, state: State) { + if let Some(state) = merged(before, state, read_state().unwrap_or_default()) { + write_state(&state); + } +} + +/// [`save_fetch`]'s decision: what to write, given the state the fetch +/// started from, what it decided, and what is on disk now. +fn merged(before: &State, mut state: State, disk: State) -> Option { + let same_fetch = disk.token == before.token + && disk.rejected == before.rejected + && disk.failures == before.failures + && disk.retry_at == before.retry_at; + if !same_fetch { + return None; + } + if state.token == before.token { + state.cooldowns = disk.cooldowns; + } + Some(state) +} + +/// Whether [`token`] could have one, judged without the network. +pub(crate) fn may_have_token() -> bool { + let state = read_state().unwrap_or_default(); + state.usable_token().is_some() || !holds(state.retry_at, now()) +} + +/// What a `401` or `403` to the CLI's token meant. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) enum Refusal { + /// The token itself is revoked, invalid, expired or malformed. It is + /// marked, so the next [`token`] fetches a new one. + Dead, + /// The token is fine; this API does not accept it. + Forbidden, + /// The Tokens API could not say. The token is kept, and the caller rests + /// as for [`Refusal::Forbidden`]: sending it again at once would most + /// likely be refused again, and cost another check. + Unknown, + /// Not checked: `token` is not the one on disk. Another run saved a new + /// one meanwhile, or the state cannot be written at all, in which case + /// nothing is marked or paced and each run fetches as if for the first + /// time (see [`write_state`]). + NotFetched, +} + +/// Asks the Tokens API, at once, why `caller`'s request with the fetched +/// `token` was refused. A dead token is marked, so the next [`token`] +/// fetches a new one; otherwise `caller`'s refusals are counted, and the +/// third in a row rests it for an hour. +pub(crate) fn refused(caller: &str, token: &str) -> Refusal { + let now = now(); + if read_state() + .filter(|state| state.token.as_deref() == Some(token)) + .is_none() + { + return Refusal::NotFetched; + } + let refusal = check(token); + // Read again: the check can take seconds, and another run may have + // saved a new token meanwhile, which this must not write over. + if let Some(mut state) = read_state().filter(|state| state.token.as_deref() == Some(token)) { + // `verdict` never answers `NotFetched`; that was returned above. + if refusal == Refusal::Dead { + state.rejected = true; + } else { + let cooldown = state.cooldowns.entry(caller.to_owned()).or_default(); + cooldown.failures += 1; + cooldown.until = now + backoff(cooldown.failures); + } + write_state(&state); + } + refusal +} + +/// What the Tokens API says about `token`. Never [`Refusal::NotFetched`]. +fn check(token: &str) -> Refusal { + verdict(auth::token_code(&tokens_url(), token, Some(super::REFERER), FETCH_TIMEOUT).as_deref()) +} + +fn tokens_url() -> String { + http::url_override(TOKENS_URL_ENV).unwrap_or_else(|| auth::VALIDATION_ENDPOINT.to_owned()) +} + +/// Whether either endpoint is overridden from the environment, as only a +/// build that is not a production release allows. +fn overridden() -> bool { + http::url_override(URL_ENV).is_some() || http::url_override(TOKENS_URL_ENV).is_some() +} + +fn verdict(code: Option<&str>) -> Refusal { + match code { + Some("TokenValid") => Refusal::Forbidden, + Some("TokenRevoked" | "TokenInvalid" | "TokenExpired" | "TokenMalformed") => Refusal::Dead, + _ => Refusal::Unknown, + } +} + +/// The decision, with the clock, the endpoint and the Tokens API passed in. +fn acquire( + state: &mut State, + now: u64, + get: impl FnOnce() -> Fetched, + check: impl FnOnce(&str) -> Refusal, +) -> Option { + if let Some(token) = state.usable_token() { + return Some(token.to_owned()); + } + if holds(state.retry_at, now) { + return None; + } + match get() { + Fetched::Failed => None, + Fetched::Token(token) + if is_plausible(&token) + && (!(state.rejected && state.token.as_deref() == Some(token.as_str())) + || check(&token) == Refusal::Forbidden) => + { + *state = State { + token: Some(token.clone()), + ..State::default() + }; + Some(token) + } + // The rejected token again and still not valid, something that is + // not a `pk.` token, or no token at all. + Fetched::Token(_) | Fetched::Nothing => { + state.fail(now); + None + } + } +} + +/// Whether a wait until `until` still holds at `now`: the endpoint's, or a +/// caller's cooldown. One further out than the longest wait, an hour, is a +/// clock that went backwards or an edited file, and is not honored. +fn holds(until: u64, now: u64) -> bool { + now < until && until <= now + HOUR +} + +/// Whether `value` looks like a public token and nothing else. Checked on +/// what the endpoint returns and on what is read back from disk, since both +/// end up in a request. +fn is_plausible(value: &str) -> bool { + value.starts_with("pk.") + && value.len() <= MAX_TOKEN_LEN + && value + .bytes() + .all(|b| b.is_ascii_alphanumeric() || matches!(b, b'.' | b'-' | b'_')) +} + +fn endpoint() -> String { + http::url_override(URL_ENV).unwrap_or_else(|| PRODUCTION_URL.to_owned()) +} + +fn get(url: &str) -> Fetched { + let Ok(client) = http::client() else { + return Fetched::Failed; + }; + let Ok(response) = http::send(client.get(url).timeout(FETCH_TIMEOUT)) else { + return Fetched::Failed; + }; + if !response.status().is_success() { + return Fetched::Nothing; + } + response + .json::() + .ok() + .and_then(|body| Some(body.get("token")?.as_str()?.trim().to_owned())) + .map_or(Fetched::Nothing, Fetched::Token) +} + +fn state_path() -> Option { + Some(auth::config_dir_path()?.join(STATE_FILE)) +} + +/// The state on disk, or `None` when there is no readable one. A malformed +/// file costs one fetch, which is not worth failing anything over. +fn read_state() -> Option { + let text = std::fs::read_to_string(state_path()?).ok()?; + let state: State = serde_json::from_str(&text).ok()?; + (state.overridden == overridden()).then_some(state) +} + +/// Best-effort: a read-only home directory costs a fetch per run, not an +/// error. +/// +/// Creates the config directory when there is none, `0700` like any other +/// first write to it, because without this file a dead token is never marked +/// and nothing backs off. An existing one is written into as it is, so its +/// permissions are not changed by a run that only sent telemetry. A plain +/// file where the directory belongs leaves nothing to write into, and each +/// run then fetches as if for the first time. +fn write_state(state: &State) { + let state = &State { + overridden: overridden(), + ..state.clone() + }; + let Some(dir) = auth::config_dir_path() else { + return; + }; + let dir = if dir.is_dir() { + dir + } else if dir.exists() { + return; + } else { + match auth::config_dir() { + Ok(dir) => dir, + Err(_) => return, + } + }; + if let Ok(text) = serde_json::to_string(state) { + let _ = auth::write_private(&dir.join(STATE_FILE), &text); + } +} + +fn now() -> u64 { + SystemTime::now() + .duration_since(UNIX_EPOCH) + .map(|d| d.as_secs()) + .unwrap_or_default() +} + +#[cfg(test)] +mod tests { + use super::*; + + const NOW: u64 = 1_000_000; + + fn saved(token: &str, rejected: bool) -> State { + State { + token: Some(token.to_owned()), + rejected, + ..State::default() + } + } + + /// Runs [`acquire`] with an endpoint that answers `answer` and a Tokens + /// API that still calls the token dead, and says whether the endpoint was + /// asked. + fn run(state: &mut State, now: u64, answer: Fetched) -> (Option, bool) { + let (result, asked, _) = run_checked(state, now, answer, Refusal::Dead); + (result, asked) + } + + /// [`run`] with the Tokens API answering `verdict`, and also says whether + /// it was asked. + fn run_checked( + state: &mut State, + now: u64, + answer: Fetched, + verdict: Refusal, + ) -> (Option, bool, bool) { + let (mut asked, mut checked) = (false, false); + let result = acquire( + state, + now, + || { + asked = true; + answer + }, + |_| { + checked = true; + verdict + }, + ); + (result, asked, checked) + } + + fn token(value: &str) -> Fetched { + Fetched::Token(value.to_owned()) + } + + #[test] + fn a_saved_token_is_used_without_asking() { + let mut state = saved("pk.saved", false); + let (result, asked) = run(&mut state, NOW, token("pk.new")); + assert_eq!(result, Some("pk.saved".to_owned())); + assert!(!asked); + } + + #[test] + fn no_saved_token_fetches_one_and_keeps_it() { + let mut state = State::default(); + let (result, asked) = run(&mut state, NOW, token("pk.new")); + assert_eq!(result, Some("pk.new".to_owned())); + assert!(asked); + assert_eq!(state, saved("pk.new", false)); + } + + #[test] + fn a_rejected_token_is_replaced_by_a_new_one() { + let mut state = saved("pk.old", true); + let (result, _) = run(&mut state, NOW, token("pk.new")); + assert_eq!(result, Some("pk.new".to_owned())); + assert_eq!(state, saved("pk.new", false)); + } + + #[test] + fn the_rejected_token_again_waits_an_hour_after_every_third_time() { + let mut state = saved("pk.old", true); + let mut now = NOW; + let mut waits = Vec::new(); + for _ in 0..8 { + let (result, asked) = run(&mut state, now, token("pk.old")); + assert!(asked); + assert_eq!(result, None); + waits.push(state.retry_at - now); + now = state.retry_at; + } + assert_eq!( + waits, + [0, 0, HOUR, 0, 0, HOUR, 0, 0], + "an hour after every third failure in a row" + ); + } + + #[test] + fn the_rejected_token_again_is_taken_back_once_it_checks_valid() { + let mut state = State { + failures: 2, + ..saved("pk.old", true) + }; + let (result, _, checked) = + run_checked(&mut state, NOW, token("pk.old"), Refusal::Forbidden); + assert!(checked); + assert_eq!(result, Some("pk.old".to_owned())); + assert_eq!(state, saved("pk.old", false), "the count starts again"); + + for verdict in [Refusal::Dead, Refusal::Unknown] { + let mut state = saved("pk.old", true); + let (result, _, _) = run_checked(&mut state, NOW, token("pk.old"), verdict); + assert_eq!(result, None, "{verdict:?}"); + assert!(state.rejected, "{verdict:?}"); + assert_eq!(state.failures, 1, "{verdict:?}"); + } + } + + #[test] + fn only_the_rejected_token_served_again_is_checked() { + for (mut state, answer) in [ + (State::default(), token("pk.new")), + (saved("pk.old", true), token("pk.new")), + (saved("pk.old", true), Fetched::Nothing), + (saved("pk.old", true), Fetched::Failed), + ] { + let (_, _, checked) = run_checked(&mut state, NOW, answer, Refusal::Forbidden); + assert!(!checked); + } + } + + #[test] + fn nothing_is_fetched_while_waiting() { + let mut state = State { + retry_at: NOW + HOUR, + ..saved("pk.old", true) + }; + let (result, asked) = run(&mut state, NOW + HOUR - 1, token("pk.new")); + assert!(!asked); + assert_eq!(result, None); + let (result, asked) = run(&mut state, NOW + HOUR, token("pk.new")); + assert!(asked); + assert_eq!(result, Some("pk.new".to_owned())); + assert_eq!(state.failures, 0, "a new token starts the count again"); + } + + #[test] + fn an_answer_without_a_usable_token_counts_as_a_failure() { + for answer in [ + Fetched::Nothing, + token("sk.secret"), + token("pk.has space"), + token(""), + ] { + let mut state = State::default(); + let (result, _) = run(&mut state, NOW, answer); + assert_eq!(result, None); + assert_eq!(state.failures, 1); + assert_eq!(state.retry_at, NOW, "the first failure waits for nothing"); + assert_eq!(state.token, None, "nothing unusable is kept"); + } + } + + #[test] + fn no_answer_at_all_is_tried_again_next_run() { + let mut state = saved("pk.old", true); + let (result, _) = run(&mut state, NOW, Fetched::Failed); + assert_eq!(result, None); + assert_eq!(state, saved("pk.old", true), "the schedule is untouched"); + } + + #[test] + fn a_wait_beyond_an_hour_is_not_honored() { + let mut state = State { + retry_at: u64::MAX, + ..saved("pk.old", true) + }; + let (result, asked) = run(&mut state, NOW, token("pk.new")); + assert!(asked); + assert_eq!(result, Some("pk.new".to_owned())); + } + + #[test] + fn a_saved_token_that_is_not_plausible_is_not_used() { + let mut state = saved("sk.edited-in", false); + let (result, asked) = run(&mut state, NOW, token("pk.new")); + assert!(asked); + assert_eq!(result, Some("pk.new".to_owned())); + } + + #[test] + fn a_cooldown_holds_for_its_hour_and_not_beyond() { + let cooldown = Cooldown { + failures: 3, + until: NOW + HOUR, + }; + assert!(holds(cooldown.until, NOW)); + assert!(!holds(cooldown.until, NOW + HOUR)); + let edited = Cooldown { + failures: 3, + until: u64::MAX, + }; + assert!( + !holds(edited.until, NOW), + "a wait beyond an hour is not honored" + ); + } + + #[test] + fn a_fetch_keeps_what_another_run_wrote_meanwhile() { + let resting = || State { + cooldowns: BTreeMap::from([( + "telemetry".to_owned(), + Cooldown { + failures: 3, + until: NOW + HOUR, + }, + )]), + ..saved("pk.old", true) + }; + let before = saved("pk.old", true); + + // A cooldown added meanwhile is kept by a fetch that only failed... + let mut failed = before.clone(); + failed.fail(NOW); + let written = merged(&before, failed.clone(), resting()).expect("written"); + assert_eq!(written.failures, 1); + assert_eq!(written.cooldowns, resting().cooldowns); + + // ...and ended by one that brought a new token. + let fresh = saved("pk.new", false); + let written = merged(&before, fresh.clone(), resting()).expect("written"); + assert_eq!(written, fresh); + + // A token or a count another run saved meanwhile stands. + assert_eq!( + merged(&before, fresh.clone(), saved("pk.other", false)), + None + ); + let mut counted = before.clone(); + counted.fail(NOW); + assert_eq!(merged(&before, failed, counted), None); + } + + #[test] + fn only_a_dead_token_is_replaced() { + for code in [ + "TokenRevoked", + "TokenInvalid", + "TokenExpired", + "TokenMalformed", + ] { + assert_eq!(verdict(Some(code)), Refusal::Dead, "{code}"); + } + assert_eq!(verdict(Some("TokenValid")), Refusal::Forbidden); + assert_eq!(verdict(Some("SomethingNew")), Refusal::Unknown); + assert_eq!(verdict(None), Refusal::Unknown, "the check itself failed"); + } + + #[test] + fn the_state_round_trips_and_tolerates_a_partial_one() { + let state = State { + token: Some("pk.a".to_owned()), + rejected: true, + failures: 2, + retry_at: 7, + cooldowns: BTreeMap::from([( + "telemetry".to_owned(), + Cooldown { + failures: 1, + until: 9, + }, + )]), + overridden: true, + }; + let text = serde_json::to_string(&state).unwrap(); + assert_eq!(serde_json::from_str::(&text).unwrap(), state); + assert_eq!( + serde_json::from_str::(r#"{"token":"pk.a"}"#).unwrap(), + saved("pk.a", false) + ); + } +} diff --git a/src/http.rs b/src/http.rs index 164d4fa..ad222f4 100644 --- a/src/http.rs +++ b/src/http.rs @@ -310,9 +310,44 @@ fn is_mapbox_host(host: &str) -> bool { host == "mapbox.com" || host.ends_with(".mapbox.com") } +/// A URL from the environment variable `name`, unless this build is a +/// production release, which ignores it, so an injected variable cannot point +/// a request, or the token it carries, at a host of its choosing. Builds that +/// are not releases honor it, which is how the tests reach loopback servers. +/// For a request that carries a token; `update_check`'s, which carries none, +/// keeps its override in a release too. +pub(crate) fn url_override(name: &str) -> Option { + override_unless_production(is_production_build(), std::env::var(name).ok()) +} + +fn override_unless_production(production: bool, value: Option) -> Option { + let value = value?; + let value = value.trim(); + (!production && !value.is_empty()).then(|| value.to_owned()) +} + +/// `MAPBOX_CLI_BUILD_ENV` is how the release pipeline marks a production +/// build, read at compile time, so outside a release this is a constant +/// `false`. +pub(crate) fn is_production_build() -> bool { + option_env!("MAPBOX_CLI_BUILD_ENV") == Some("production") +} + #[cfg(test)] mod tests { use super::*; + + #[test] + fn only_a_build_that_is_not_production_takes_a_url_override() { + let custom = || Some(" http://127.0.0.1:1/x ".to_string()); + assert_eq!( + override_unless_production(false, custom()).as_deref(), + Some("http://127.0.0.1:1/x") + ); + assert_eq!(override_unless_production(true, custom()), None); + assert_eq!(override_unless_production(false, None), None); + assert_eq!(override_unless_production(false, Some(" ".into())), None); + } use std::io::{Read, Write}; use std::net::TcpListener; use std::time::Duration; diff --git a/src/main.rs b/src/main.rs index dc87e78..7328ea7 100644 --- a/src/main.rs +++ b/src/main.rs @@ -1481,16 +1481,6 @@ fn run(app: &Command, specs: &[ServiceSpec], matches: &ArgMatches, mode: Mode) - if let Some(token) = &token { run_record::set_resolved_token(matches, use_login, token); } - // After the record above, which knows only the user's own - // sources. `--use-login` already returned when it found nothing. - let token = token.or_else(|| { - let token = cli_token::for_command(op)?; - output::progress( - "No token of your own, so this request uses the CLI's token. \ - Run `mapbox auth login` to use yours.", - ); - Some(token) - }); let username: Option = matches .get_one::("username") .cloned() diff --git a/src/telemetry_sink.rs b/src/telemetry_sink.rs index 8c3bd60..b26fe09 100644 --- a/src/telemetry_sink.rs +++ b/src/telemetry_sink.rs @@ -161,7 +161,7 @@ pub(crate) fn remember_user_token(token: Option) { /// it with, and drops it otherwise. `profile` is the run's `--profile`, whose /// login it may send with. pub(crate) fn deliver(line: &str, profile: Option<&str>) { - let url = send_url(is_production_build(), from_environment(URL_ENV)); + let url = send_url(); // Help, `--version` and usage errors return before the arguments are // parsed, so nothing was remembered; the environment is still the user's, // unless `--use-login` was typed to set it aside. @@ -170,7 +170,12 @@ pub(crate) fn deliver(line: &str, profile: Option<&str>) { None if std::env::args_os().any(|arg| arg == "--use-login") => None, None => auth::environment_token().map(|(_, token)| token), }; - if cli_token::has_token_for(cli_token::TELEMETRY, user_token.as_deref(), profile) { + if cli_token::has_token_for( + cli_token::TELEMETRY_CALLER, + cli_token::TELEMETRY, + user_token.as_deref(), + profile, + ) { HttpSink { url, user_token, @@ -196,7 +201,7 @@ pub fn run_send_child() -> ExitCode { if telemetry::telemetry_allowed() { // Through `send_url` again, so a production child cannot be pointed // elsewhere either. - let url = send_url(is_production_build(), from_environment(URL_ENV)); + let url = send_url(); let mut input = String::new(); let read = std::io::stdin() .take(MAX_EVENT_BYTES) @@ -230,12 +235,16 @@ fn send(url: &str, envelope: &Envelope) { fn post(url: &str, envelope: &Envelope) -> Result<(u16, Duration), String> { let body = serde_json::to_string(&[&envelope.event]).map_err(|err| err.to_string())?; let client = http::client().map_err(|err| err.to_string())?; - let started = Instant::now(); + // Reset by every attempt, so the time is the POST that answered alone, + // not the token fetch or check that may come before it. + let started = std::cell::Cell::new(Instant::now()); let response = cli_token::send( + cli_token::TELEMETRY_CALLER, cli_token::TELEMETRY, envelope.user_token.as_deref(), envelope.profile.as_deref(), || { + started.set(Instant::now()); client .post(url) .header(USER_AGENT, telemetry::PRODUCT_TOKEN) @@ -246,7 +255,7 @@ fn post(url: &str, envelope: &Envelope) -> Result<(u16, Duration), String> { ) .ok_or_else(|| "no token to send with".to_string())? .map_err(|err| http::failure(&err))?; - Ok((response.status().as_u16(), started.elapsed())) + Ok((response.status().as_u16(), started.get().elapsed())) } /// Appends one line to the delivery log, when [`DELIVERY_LOG_ENV`] is set. @@ -279,38 +288,9 @@ fn from_environment(name: &str) -> Option { } /// Where to send: [`ENDPOINT`], or [`URL_ENV`]'s override in a build that is -/// not a production release. A release ignores the override, so an injected -/// variable cannot send a user's token to a host of its choosing. -fn send_url(production: bool, override_url: Option) -> String { - match override_url { - Some(url) if !production => url, - _ => ENDPOINT.to_string(), - } -} - -/// `MAPBOX_CLI_BUILD_ENV` is how the release pipeline marks a production -/// build; see `update_check`. -const fn is_production_build() -> bool { - match option_env!("MAPBOX_CLI_BUILD_ENV") { - Some(env) => same(env, "production"), - None => false, - } -} - -/// `==` on `&str`, which is not available in a `const` context. -const fn same(a: &str, b: &str) -> bool { - let (a, b) = (a.as_bytes(), b.as_bytes()); - if a.len() != b.len() { - return false; - } - let mut i = 0; - while i < a.len() { - if a[i] != b[i] { - return false; - } - i += 1; - } - true +/// not a production release (see [`http::url_override`]). +fn send_url() -> String { + http::url_override(URL_ENV).unwrap_or_else(|| ENDPOINT.to_string()) } /// The contents of a state file, trimmed, if it is there. @@ -371,17 +351,3 @@ fn dir() -> Option { } dated_jsonl::private_dir(DIR) } - -#[cfg(test)] -mod tests { - use super::*; - - #[test] - fn only_a_build_that_is_not_production_takes_the_url_override() { - let custom = || Some("http://127.0.0.1:1/events/v2".to_string()); - assert_eq!(send_url(false, custom()), "http://127.0.0.1:1/events/v2"); - assert_eq!(send_url(true, custom()), ENDPOINT); - assert_eq!(send_url(false, None), ENDPOINT); - assert_eq!(send_url(true, None), ENDPOINT); - } -} diff --git a/src/update_check.rs b/src/update_check.rs index 8171b42..f2d60bd 100644 --- a/src/update_check.rs +++ b/src/update_check.rs @@ -185,12 +185,9 @@ fn manifest_url() -> Option { if let Some(override_url) = from_environment(URL_ENV) { return Some(override_url); } - match option_env!("MAPBOX_CLI_BUILD_ENV") { - // Only production. Staging and dev sit behind basic auth at the - // edge, and this does not send credentials. - Some("production") => Some(format!("{PRODUCTION_BASE_URL}/latest/manifest.json")), - _ => None, - } + // Only production. Staging and dev sit behind basic auth at the edge, + // and this does not send credentials. + http::is_production_build().then(|| format!("{PRODUCTION_BASE_URL}/latest/manifest.json")) } fn cache_path() -> Option { diff --git a/tests/auth_profiles.rs b/tests/auth_profiles.rs index c5bdf0e..42980b2 100644 --- a/tests/auth_profiles.rs +++ b/tests/auth_profiles.rs @@ -47,7 +47,9 @@ fn command(home: &Path) -> Command { let mut cmd = Command::new(env!("CARGO_BIN_EXE_mapbox")); // Telemetry is not under test here, and would post to Mapbox. cmd.env("MAPBOX_CLI_NO_TELEMETRY", "1") - .env_remove("MAPBOX_CLI_TOKEN"); + .env_remove("MAPBOX_CLI_TOKEN") + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", "http://127.0.0.1:9/") + .env("MAPBOX_INTERNAL_TOKENS_URL", "http://127.0.0.1:9/"); cmd.env_remove("MAPBOX_ACCESS_TOKEN") .env_remove("MapboxAccessToken") .env_remove("MAPBOX_USERNAME") diff --git a/tests/completion.rs b/tests/completion.rs index 886d45f..0331469 100644 --- a/tests/completion.rs +++ b/tests/completion.rs @@ -43,7 +43,9 @@ fn command() -> Command { let mut cmd = Command::new(env!("CARGO_BIN_EXE_mapbox")); // Telemetry is not under test here, and would post to Mapbox. cmd.env("MAPBOX_CLI_NO_TELEMETRY", "1") - .env_remove("MAPBOX_CLI_TOKEN"); + .env_remove("MAPBOX_CLI_TOKEN") + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", "http://127.0.0.1:9/") + .env("MAPBOX_INTERNAL_TOKENS_URL", "http://127.0.0.1:9/"); cmd.env_remove("MAPBOX_ACCESS_TOKEN") .env_remove("MapboxAccessToken") .env_remove("MAPBOX_USERNAME") diff --git a/tests/config.rs b/tests/config.rs index e07a5b0..dc8d42b 100644 --- a/tests/config.rs +++ b/tests/config.rs @@ -25,7 +25,9 @@ fn command(home: &Path) -> Command { let mut cmd = Command::new(env!("CARGO_BIN_EXE_mapbox")); // Telemetry is not under test here, and would post to Mapbox. cmd.env("MAPBOX_CLI_NO_TELEMETRY", "1") - .env_remove("MAPBOX_CLI_TOKEN"); + .env_remove("MAPBOX_CLI_TOKEN") + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", "http://127.0.0.1:9/") + .env("MAPBOX_INTERNAL_TOKENS_URL", "http://127.0.0.1:9/"); cmd.env_remove("MAPBOX_ACCESS_TOKEN") .env_remove("MapboxAccessToken") .env_remove("MAPBOX_USERNAME") diff --git a/tests/diagnostic_log.rs b/tests/diagnostic_log.rs index e8fdf93..a89569e 100644 --- a/tests/diagnostic_log.rs +++ b/tests/diagnostic_log.rs @@ -43,7 +43,9 @@ fn command(home: &Path) -> Command { let mut cmd = Command::new(env!("CARGO_BIN_EXE_mapbox")); // Telemetry is not under test here, and would post to Mapbox. cmd.env("MAPBOX_CLI_NO_TELEMETRY", "1") - .env_remove("MAPBOX_CLI_TOKEN"); + .env_remove("MAPBOX_CLI_TOKEN") + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", "http://127.0.0.1:9/") + .env("MAPBOX_INTERNAL_TOKENS_URL", "http://127.0.0.1:9/"); cmd.env_remove("MAPBOX_ACCESS_TOKEN") .env_remove("MapboxAccessToken") .env_remove("MAPBOX_USERNAME") diff --git a/tests/docs_contract.rs b/tests/docs_contract.rs index c94f4fb..f916eba 100644 --- a/tests/docs_contract.rs +++ b/tests/docs_contract.rs @@ -71,6 +71,8 @@ fn schema() -> Value { let out = Command::new(env!("CARGO_BIN_EXE_mapbox")) .env("MAPBOX_CLI_NO_TELEMETRY", "1") .env_remove("MAPBOX_CLI_TOKEN") + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", "http://127.0.0.1:9/") + .env("MAPBOX_INTERNAL_TOKENS_URL", "http://127.0.0.1:9/") .env_remove("MAPBOX_ACCESS_TOKEN") .env_remove("MapboxAccessToken") .env_remove("MAPBOX_USERNAME") diff --git a/tests/doctor.rs b/tests/doctor.rs index 847f785..1790935 100644 --- a/tests/doctor.rs +++ b/tests/doctor.rs @@ -44,6 +44,8 @@ fn command(home: &Path) -> Command { .env_remove("MAPBOX_CLI_NO_TELEMETRY") // Telemetry stays on, as it is for a user, but goes nowhere. .env_remove("MAPBOX_CLI_TOKEN") + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", "http://127.0.0.1:9/") + .env("MAPBOX_INTERNAL_TOKENS_URL", "http://127.0.0.1:9/") .env("MAPBOX_INTERNAL_TELEMETRY_URL", "http://127.0.0.1:9/") .env_remove("MAPBOX_NO_UPDATE_CHECK") .env_remove("MAPBOX_TIMEOUT") diff --git a/tests/dry_run.rs b/tests/dry_run.rs index 58e28e1..052835f 100644 --- a/tests/dry_run.rs +++ b/tests/dry_run.rs @@ -40,7 +40,9 @@ fn command(config: &Path) -> Command { let mut cmd = Command::new(env!("CARGO_BIN_EXE_mapbox")); // Telemetry is not under test here, and would post to Mapbox. cmd.env("MAPBOX_CLI_NO_TELEMETRY", "1") - .env_remove("MAPBOX_CLI_TOKEN"); + .env_remove("MAPBOX_CLI_TOKEN") + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", "http://127.0.0.1:9/") + .env("MAPBOX_INTERNAL_TOKENS_URL", "http://127.0.0.1:9/"); cmd.env_remove("MAPBOX_ACCESS_TOKEN") .env_remove("MapboxAccessToken") .env_remove("MAPBOX_USERNAME") diff --git a/tests/generate_skills.rs b/tests/generate_skills.rs index 1e01316..7281d40 100644 --- a/tests/generate_skills.rs +++ b/tests/generate_skills.rs @@ -36,7 +36,9 @@ fn command(name: &str) -> Command { let mut cmd = Command::new(env!("CARGO_BIN_EXE_mapbox")); // Telemetry is not under test here, and would post to Mapbox. cmd.env("MAPBOX_CLI_NO_TELEMETRY", "1") - .env_remove("MAPBOX_CLI_TOKEN"); + .env_remove("MAPBOX_CLI_TOKEN") + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", "http://127.0.0.1:9/") + .env("MAPBOX_INTERNAL_TOKENS_URL", "http://127.0.0.1:9/"); cmd.env_remove("MAPBOX_ACCESS_TOKEN") .env_remove("MapboxAccessToken") .env_remove("MAPBOX_USERNAME") diff --git a/tests/history.rs b/tests/history.rs index 0c57cb2..cbf6ac6 100644 --- a/tests/history.rs +++ b/tests/history.rs @@ -43,6 +43,8 @@ fn command(home: &Path) -> Command { .env_remove("MAPBOX_USERNAME") .env_remove("MAPBOX_OUTPUT") .env_remove("MAPBOX_HISTORY") + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", "http://127.0.0.1:9/") + .env("MAPBOX_INTERNAL_TOKENS_URL", "http://127.0.0.1:9/") .env_remove("MAPBOX_LOG") .env_remove("SUDO_USER") .env_remove("NO_PROXY") diff --git a/tests/mcp_proxy.rs b/tests/mcp_proxy.rs index 51f61ce..3eb077f 100644 --- a/tests/mcp_proxy.rs +++ b/tests/mcp_proxy.rs @@ -85,7 +85,9 @@ fn run(stub: &Path, marker: &Path, get_exit: &str, add_exit: &str, args: &[&str] let mut cmd = Command::new(env!("CARGO_BIN_EXE_mapbox")); // Telemetry is not under test here, and would post to Mapbox. cmd.env("MAPBOX_CLI_NO_TELEMETRY", "1") - .env_remove("MAPBOX_CLI_TOKEN"); + .env_remove("MAPBOX_CLI_TOKEN") + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", "http://127.0.0.1:9/") + .env("MAPBOX_INTERNAL_TOKENS_URL", "http://127.0.0.1:9/"); isolate_every_client(&mut cmd); cmd.arg("mcp") .args(args) @@ -250,7 +252,9 @@ fn a_client_not_on_path_is_skipped_and_named() { let mut cmd = Command::new(env!("CARGO_BIN_EXE_mapbox")); // Telemetry is not under test here, and would post to Mapbox. cmd.env("MAPBOX_CLI_NO_TELEMETRY", "1") - .env_remove("MAPBOX_CLI_TOKEN"); + .env_remove("MAPBOX_CLI_TOKEN") + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", "http://127.0.0.1:9/") + .env("MAPBOX_INTERNAL_TOKENS_URL", "http://127.0.0.1:9/"); isolate_every_client(&mut cmd); let output = cmd .args(["mcp", "install"]) @@ -344,7 +348,9 @@ fn run_codex( let mut cmd = Command::new(env!("CARGO_BIN_EXE_mapbox")); // Telemetry is not under test here, and would post to Mapbox. cmd.env("MAPBOX_CLI_NO_TELEMETRY", "1") - .env_remove("MAPBOX_CLI_TOKEN"); + .env_remove("MAPBOX_CLI_TOKEN") + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", "http://127.0.0.1:9/") + .env("MAPBOX_INTERNAL_TOKENS_URL", "http://127.0.0.1:9/"); isolate_every_client(&mut cmd); cmd.arg("mcp") .args(args) @@ -489,7 +495,9 @@ fn run_vscode( let mut cmd = Command::new(env!("CARGO_BIN_EXE_mapbox")); // Telemetry is not under test here, and would post to Mapbox. cmd.env("MAPBOX_CLI_NO_TELEMETRY", "1") - .env_remove("MAPBOX_CLI_TOKEN"); + .env_remove("MAPBOX_CLI_TOKEN") + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", "http://127.0.0.1:9/") + .env("MAPBOX_INTERNAL_TOKENS_URL", "http://127.0.0.1:9/"); isolate_every_client(&mut cmd); cmd.arg("mcp") .args(args) @@ -715,7 +723,9 @@ fn cursor_reads_its_own_settings_json_shape() { let mut cmd = Command::new(env!("CARGO_BIN_EXE_mapbox")); // Telemetry is not under test here, and would post to Mapbox. cmd.env("MAPBOX_CLI_NO_TELEMETRY", "1") - .env_remove("MAPBOX_CLI_TOKEN"); + .env_remove("MAPBOX_CLI_TOKEN") + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", "http://127.0.0.1:9/") + .env("MAPBOX_INTERNAL_TOKENS_URL", "http://127.0.0.1:9/"); isolate_every_client(&mut cmd); let output = cmd .arg("mcp") diff --git a/tests/non_interactive.rs b/tests/non_interactive.rs index d08f5a2..6b2fb1f 100644 --- a/tests/non_interactive.rs +++ b/tests/non_interactive.rs @@ -35,7 +35,9 @@ fn command(home: &Path) -> Command { let mut cmd = Command::new(env!("CARGO_BIN_EXE_mapbox")); // Telemetry is not under test here, and would post to Mapbox. cmd.env("MAPBOX_CLI_NO_TELEMETRY", "1") - .env_remove("MAPBOX_CLI_TOKEN"); + .env_remove("MAPBOX_CLI_TOKEN") + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", "http://127.0.0.1:9/") + .env("MAPBOX_INTERNAL_TOKENS_URL", "http://127.0.0.1:9/"); cmd.env_remove("MAPBOX_ACCESS_TOKEN") .env_remove("MapboxAccessToken") .env_remove("MAPBOX_USERNAME") @@ -351,6 +353,8 @@ fn under_a_pty(home: &Path, out_path: &Path, args: &str, expect: &str) -> Option .env_remove("MAPBOX_OUTPUT") .env_remove("MAPBOX_YES") .env_remove("MAPBOX_CLI_TOKEN") + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", "http://127.0.0.1:9/") + .env("MAPBOX_INTERNAL_TOKENS_URL", "http://127.0.0.1:9/") .env("MAPBOX_CLI_NO_TELEMETRY", "1") .env("HOME", home) .env("MAPBOX_CONFIG_DIR", home.join("config")) diff --git a/tests/output_contract.rs b/tests/output_contract.rs index 475553c..22e66c2 100644 --- a/tests/output_contract.rs +++ b/tests/output_contract.rs @@ -47,7 +47,9 @@ fn command() -> Command { let mut cmd = Command::new(env!("CARGO_BIN_EXE_mapbox")); // Telemetry is not under test here, and would post to Mapbox. cmd.env("MAPBOX_CLI_NO_TELEMETRY", "1") - .env_remove("MAPBOX_CLI_TOKEN"); + .env_remove("MAPBOX_CLI_TOKEN") + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", "http://127.0.0.1:9/") + .env("MAPBOX_INTERNAL_TOKENS_URL", "http://127.0.0.1:9/"); cmd.env_remove("MAPBOX_ACCESS_TOKEN") .env_remove("MapboxAccessToken") .env_remove("MAPBOX_USERNAME") @@ -105,6 +107,8 @@ fn a_file_where_the_credential_directory_goes_fails_before_the_browser() { let out = Command::new(env!("CARGO_BIN_EXE_mapbox")) .env("MAPBOX_CLI_NO_TELEMETRY", "1") .env_remove("MAPBOX_CLI_TOKEN") + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", "http://127.0.0.1:9/") + .env("MAPBOX_INTERNAL_TOKENS_URL", "http://127.0.0.1:9/") .args(["-o", "text", "auth", "login"]) .env_remove("MAPBOX_ACCESS_TOKEN") .env_remove("MapboxAccessToken") @@ -162,6 +166,8 @@ fn a_command_that_is_not_about_credentials_warns_in_one_line() { let out = Command::new(env!("CARGO_BIN_EXE_mapbox")) .env("MAPBOX_CLI_NO_TELEMETRY", "1") .env_remove("MAPBOX_CLI_TOKEN") + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", "http://127.0.0.1:9/") + .env("MAPBOX_INTERNAL_TOKENS_URL", "http://127.0.0.1:9/") .args(["-o", "text", "styles", "list"]) .env_remove("MAPBOX_ACCESS_TOKEN") .env_remove("MapboxAccessToken") @@ -471,6 +477,8 @@ fn without_an_agent_the_help_has_no_hint() { // `env_clear`, which would otherwise clear it too. cmd.env_clear() .env("MAPBOX_CLI_NO_TELEMETRY", "1") + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", "http://127.0.0.1:9/") + .env("MAPBOX_INTERNAL_TOKENS_URL", "http://127.0.0.1:9/") .env("HOME", &home) .env("MAPBOX_CONFIG_DIR", home.join(".mapbox")); // Windows processes expect this one; nothing reads it as an agent. diff --git a/tests/proxy.rs b/tests/proxy.rs index 238baae..87e7dee 100644 --- a/tests/proxy.rs +++ b/tests/proxy.rs @@ -131,7 +131,9 @@ fn mapbox() -> Command { let mut cmd = Command::new(env!("CARGO_BIN_EXE_mapbox")); // Telemetry is not under test here, and would post to Mapbox. cmd.env("MAPBOX_CLI_NO_TELEMETRY", "1") - .env_remove("MAPBOX_CLI_TOKEN"); + .env_remove("MAPBOX_CLI_TOKEN") + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", "http://127.0.0.1:9/") + .env("MAPBOX_INTERNAL_TOKENS_URL", "http://127.0.0.1:9/"); cmd.args(["styles", "list", "--username", "someone", "-o", "json"]) .env_remove("HTTP_PROXY") .env_remove("HTTPS_PROXY") diff --git a/tests/schema_contract.rs b/tests/schema_contract.rs index 6ceb64c..4be552c 100644 --- a/tests/schema_contract.rs +++ b/tests/schema_contract.rs @@ -34,7 +34,9 @@ fn command() -> Command { let mut cmd = Command::new(env!("CARGO_BIN_EXE_mapbox")); // Telemetry is not under test here, and would post to Mapbox. cmd.env("MAPBOX_CLI_NO_TELEMETRY", "1") - .env_remove("MAPBOX_CLI_TOKEN"); + .env_remove("MAPBOX_CLI_TOKEN") + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", "http://127.0.0.1:9/") + .env("MAPBOX_INTERNAL_TOKENS_URL", "http://127.0.0.1:9/"); cmd.env_remove("MAPBOX_ACCESS_TOKEN") .env_remove("MapboxAccessToken") .env_remove("MAPBOX_USERNAME") diff --git a/tests/source_guards.rs b/tests/source_guards.rs index bcdccb1..547068b 100644 --- a/tests/source_guards.rs +++ b/tests/source_guards.rs @@ -567,3 +567,33 @@ fn only_telemetry_and_doctor_read_the_env_opt_out_alone() { persisted setting: {unexpected:?}. Call `telemetry::telemetry_allowed` instead." ); } + +/// Every end-to-end test file that runs the binary keeps the CLI token's +/// fetch and check off the network. +/// +/// Every build fetches the CLI's token from cli.mapbox.com when it needs one, +/// a test build included: telemetry's child does. A refusal of it is checked +/// with the Tokens API at api.mapbox.com. Pointing `MAPBOX_INTERNAL_CLI_TOKEN_URL` and +/// `MAPBOX_INTERNAL_TOKENS_URL` at loopback is the decision this asks for, on +/// every command a file builds. +#[test] +fn every_test_that_runs_the_binary_keeps_the_token_fetch_offline() { + let dir = PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("tests"); + let mut undecided: Vec = files_under(&dir, &["rs"]) + .into_iter() + .filter_map(|(name, path)| { + let text = std::fs::read_to_string(&path).expect("read a test"); + let decided = text.contains(r#".env("MAPBOX_INTERNAL_CLI_TOKEN_URL""#) + && text.contains(r#".env("MAPBOX_INTERNAL_TOKENS_URL""#); + (text.contains("CARGO_BIN_EXE_mapbox") && !decided).then_some(name) + }) + .collect(); + undecided.sort(); + assert!( + undecided.is_empty(), + "these run `mapbox` without setting both MAPBOX_INTERNAL_CLI_TOKEN_URL and \ + MAPBOX_INTERNAL_TOKENS_URL, so a run that needs the CLI's token reaches \ + cli.mapbox.com or api.mapbox.com: {undecided:?}. Point both at loopback \ + on the command they build." + ); +} diff --git a/tests/telemetry_events.rs b/tests/telemetry_events.rs index 97a075b..8ce38a9 100644 --- a/tests/telemetry_events.rs +++ b/tests/telemetry_events.rs @@ -6,9 +6,11 @@ //! user typed, disappears when telemetry is off, and never changes stdout. //! //! `MAPBOX_INTERNAL_TELEMETRY_URL` points the binary at a loopback server, -//! which is where these tests read the event. `cargo test` builds no token -//! in, and a run with no token drops its event, so most runs here are given -//! `MAPBOX_CLI_TOKEN` to send with. +//! which is where these tests read the event. The CLI's token endpoint is +//! pointed at a dead port, and a run with no token drops its event, so most +//! runs here are given `MAPBOX_CLI_TOKEN` to send with. The tests of the +//! fetched token point `MAPBOX_INTERNAL_CLI_TOKEN_URL`, and the Tokens API +//! check behind `MAPBOX_INTERNAL_TOKENS_URL`, at loopback servers. use std::io::{Read, Write}; use std::net::TcpListener; @@ -46,6 +48,8 @@ fn command(home: &Path) -> Command { .env_remove("MAPBOX_CLI_NO_TELEMETRY") .env_remove("MAPBOX_INTERNAL_TELEMETRY_URL") .env_remove("MAPBOX_CLI_TOKEN") + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", "http://127.0.0.1:9/") + .env("MAPBOX_INTERNAL_TOKENS_URL", "http://127.0.0.1:9/") .env_remove("MAPBOX_INTERNAL_TELEMETRY_SEND") .env_remove("MAPBOX_INTERNAL_TELEMETRY_LOG") .env_remove("SUDO_USER") @@ -597,6 +601,20 @@ fn the_clis_token_sends_the_event_for_someone_with_none() { request.head.lines().next().unwrap_or_default(), "POST /events/v2?access_token=pk.cli HTTP/1.1" ); + assert_eq!( + referer(&request.head).as_deref(), + Some("https://cli.mapbox.com/"), + "the CLI's token is restricted to that URL" + ); +} + +/// The `Referer` header a request carried, if any. +fn referer(head: &str) -> Option { + head.lines().find_map(|line| { + let (name, value) = line.split_once(':')?; + name.eq_ignore_ascii_case("referer") + .then(|| value.trim().to_string()) + }) } #[test] @@ -642,7 +660,19 @@ fn the_users_token_comes_before_the_clis() { .output() .expect("run mapbox"); assert!(out.status.success()); - assert_eq!(sent_with(&received), "pk.typed"); + let request = received + .recv_timeout(Duration::from_secs(10)) + .expect("the event was posted"); + assert!( + request.head.contains("access_token=pk.typed "), + "{}", + request.head + ); + assert_eq!( + referer(&request.head), + None, + "the user's token carries no Referer" + ); } /// `--version`, help and usage errors return before the arguments are parsed, @@ -763,15 +793,17 @@ fn sending_never_creates_the_config_directory() { #[test] fn without_a_token_the_event_is_dropped() { let home = scratch("no-token"); + // The token endpoint is unreachable, which only the child finds out. let out = command(&home) .env("MAPBOX_INTERNAL_TELEMETRY_LOG", "1") .args(["config", "list"]) .output() .expect("run mapbox"); assert!(out.status.success()); - let lines = deliveries(&home, 1); - assert_eq!(lines.len(), 1, "{lines:?}"); - assert_eq!(lines[0]["stage"], "no_token"); + let lines = deliveries(&home, 2); + assert_eq!(lines.len(), 2, "{lines:?}"); + assert_eq!(lines[0]["stage"], "queued"); + assert_eq!(lines[1]["error"], "no token to send with"); let kept: Vec = std::fs::read_dir(config_dir(&home).join(".telemetry")) .expect("the telemetry directory") .map(|e| e.expect("an entry").path()) @@ -779,3 +811,468 @@ fn without_a_token_the_event_is_dropped() { .collect(); assert!(kept.is_empty(), "the event was kept on disk: {kept:?}"); } + +/// A server that answers one request with `status` and `body`, and reports +/// that it was asked. +fn answer_once(status: &'static str, body: &'static str) -> (mpsc::Receiver<()>, String) { + let listener = TcpListener::bind("127.0.0.1:0").expect("a loopback port"); + let addr = listener.local_addr().expect("the bound address"); + let (sender, receiver) = mpsc::channel(); + std::thread::spawn(move || { + let Ok((mut stream, _)) = listener.accept() else { + return; + }; + let _ = stream.set_read_timeout(Some(Duration::from_secs(10))); + let mut head = Vec::new(); + let mut byte = [0u8; 1]; + while !head.ends_with(b"\r\n\r\n") { + match stream.read(&mut byte) { + Ok(1) => head.push(byte[0]), + _ => break, + } + } + // Read the body too: closing with it unread resets the connection + // before the client sees the answer. + let head = String::from_utf8_lossy(&head).into_owned(); + let length = head + .lines() + .find_map(|l| { + let (name, value) = l.split_once(':')?; + name.eq_ignore_ascii_case("content-length") + .then(|| value.trim().parse::().ok())? + }) + .unwrap_or(0); + let _ = stream.read_exact(&mut vec![0u8; length]); + let _ = sender.send(()); + let _ = write!( + stream, + "HTTP/1.1 {status}\r\nContent-Type: application/json\r\n\ + Content-Length: {}\r\nConnection: close\r\n\r\n{body}", + body.len() + ); + }); + (receiver, format!("http://{addr}/config/token.json")) +} + +/// A server that answers one request after another with `answers`, in +/// order, and passes on each request's head. +fn answer_each( + answers: &'static [(&'static str, &'static str)], +) -> (mpsc::Receiver, String) { + let listener = TcpListener::bind("127.0.0.1:0").expect("a loopback port"); + let addr = listener.local_addr().expect("the bound address"); + let (sender, receiver) = mpsc::channel(); + std::thread::spawn(move || { + for (status, body) in answers { + let Ok((mut stream, _)) = listener.accept() else { + return; + }; + let _ = stream.set_read_timeout(Some(Duration::from_secs(10))); + let mut head = Vec::new(); + let mut byte = [0u8; 1]; + while !head.ends_with(b"\r\n\r\n") { + match stream.read(&mut byte) { + Ok(1) => head.push(byte[0]), + _ => break, + } + } + let head = String::from_utf8_lossy(&head).into_owned(); + let length = head + .lines() + .find_map(|l| { + let (name, value) = l.split_once(':')?; + name.eq_ignore_ascii_case("content-length") + .then(|| value.trim().parse::().ok())? + }) + .unwrap_or(0); + let _ = stream.read_exact(&mut vec![0u8; length]); + let _ = sender.send(head); + let _ = write!( + stream, + "HTTP/1.1 {status}\r\nContent-Type: application/json\r\n\ + Content-Length: {}\r\nConnection: close\r\n\r\n{body}", + body.len() + ); + } + }); + (receiver, format!("http://{addr}/events/v2")) +} + +/// Waits for the child to have written the CLI token's state, which happens +/// after the run itself has exited. +fn cli_token_state(home: &Path, contains: &str) -> Value { + let path = config_dir(home).join("cli-token.json"); + let deadline = Instant::now() + Duration::from_secs(10); + loop { + if let Ok(text) = std::fs::read_to_string(&path) { + if text.contains(contains) { + return serde_json::from_str(&text).expect("the state is JSON"); + } + } + assert!(Instant::now() < deadline, "no state holding {contains}"); + std::thread::sleep(Duration::from_millis(50)); + } +} + +/// With no token of the user's and no `MAPBOX_CLI_TOKEN`, the event goes +/// with the token fetched from the endpoint, which is kept for the next run. +#[test] +fn the_fetched_token_sends_the_event_and_is_kept() { + let home = scratch("send-fetched"); + let (asked, token_url) = answer_once("200 OK", r#"{"token":"pk.fetched"}"#); + let (received, url) = events_server(true); + let out = command(&home) + .env("MAPBOX_INTERNAL_TELEMETRY_URL", &url) + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", &token_url) + .args(["config", "list"]) + .output() + .expect("run mapbox"); + assert!(out.status.success()); + assert_eq!(sent_with(&received), "pk.fetched"); + asked + .recv_timeout(Duration::from_secs(10)) + .expect("the endpoint was asked"); + assert_eq!(cli_token_state(&home, "pk.fetched")["token"], "pk.fetched"); + + // The endpoint is gone now; the kept token is enough. + let (received, url) = events_server(true); + let out = command(&home) + .env("MAPBOX_INTERNAL_TELEMETRY_URL", &url) + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", &token_url) + .args(["config", "list"]) + .output() + .expect("run mapbox"); + assert!(out.status.success()); + assert_eq!(sent_with(&received), "pk.fetched"); +} + +/// A `401` to the kept token, which the Tokens API says is revoked, marks +/// it; a new one is fetched at once and the event sent again with it. +#[test] +fn a_revoked_token_is_replaced_and_the_event_resent() { + let home = scratch("send-rejected"); + std::fs::write( + config_dir(&home).join("cli-token.json"), + r#"{"token":"pk.old","overridden":true}"#, + ) + .expect("write the state"); + let (heads, events_url) = answer_each(&[ + ("401 Unauthorized", r#"{"message":"Not Authorized"}"#), + ("204 No Content", ""), + ]); + // A `401` around the verdict, which the real endpoint has not been seen + // to send: the `code` still decides. + let (_, tokens_url) = answer_once("401 Unauthorized", r#"{"code":"TokenRevoked"}"#); + let (_, token_url) = answer_once("200 OK", r#"{"token":"pk.new"}"#); + let out = command(&home) + .env("MAPBOX_INTERNAL_TELEMETRY_URL", &events_url) + .env("MAPBOX_INTERNAL_TOKENS_URL", &tokens_url) + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", &token_url) + .args(["config", "list"]) + .output() + .expect("run mapbox"); + assert!(out.status.success()); + let tokens: Vec = (0..2) + .map(|_| { + let head = heads + .recv_timeout(Duration::from_secs(10)) + .expect("an event was posted"); + assert_eq!( + referer(&head).as_deref(), + Some("https://cli.mapbox.com/"), + "{head}" + ); + head.split("access_token=") + .nth(1) + .and_then(|rest| rest.split([' ', '&']).next()) + .unwrap_or_default() + .to_string() + }) + .collect(); + assert_eq!(tokens, ["pk.old", "pk.new"]); + assert_eq!(cli_token_state(&home, "pk.new")["rejected"], false); +} + +/// A `401` to a token the Tokens API calls valid is this API refusing it, +/// not the token being dead, so it is kept; the third such refusal in a row +/// rests telemetry for an hour. +#[test] +fn a_token_an_api_refuses_is_kept() { + let home = scratch("send-forbidden"); + std::fs::write( + config_dir(&home).join("cli-token.json"), + // Refused twice before: this is the third in a row. + r#"{"token":"pk.kept","overridden":true,"cooldowns":{"telemetry":{"failures":2}}}"#, + ) + .expect("write the state"); + let (_, events_url) = answer_once("401 Unauthorized", r#"{"message":"Not Authorized"}"#); + let (checked, tokens_url) = answer_once("200 OK", r#"{"code":"TokenValid"}"#); + let out = command(&home) + .env("MAPBOX_INTERNAL_TELEMETRY_LOG", "1") + .env( + "MAPBOX_INTERNAL_TELEMETRY_URL", + events_url.replace("/config/token.json", "/events/v2"), + ) + .env("MAPBOX_INTERNAL_TOKENS_URL", &tokens_url) + .args(["config", "list"]) + .output() + .expect("run mapbox"); + assert!(out.status.success()); + checked + .recv_timeout(Duration::from_secs(10)) + .expect("the token was checked"); + let lines = deliveries(&home, 2); + assert_eq!(lines[1]["status"], 401, "{lines:?}"); + let state = cli_token_state(&home, "cooldowns"); + assert_eq!(state["token"], "pk.kept"); + assert_eq!(state["rejected"], false); + assert!(state["cooldowns"]["telemetry"]["until"].is_u64(), "{state}"); + + // Resting: the next run sends nothing with it, and checks nothing. + let (sent, events_url) = answer_once("401 Unauthorized", r#"{"message":"Not Authorized"}"#); + let (checked, tokens_url) = answer_once("200 OK", r#"{"code":"TokenValid"}"#); + let out = command(&home) + .env("MAPBOX_INTERNAL_TELEMETRY_LOG", "1") + .env( + "MAPBOX_INTERNAL_TELEMETRY_URL", + events_url.replace("/config/token.json", "/events/v2"), + ) + .env("MAPBOX_INTERNAL_TOKENS_URL", &tokens_url) + .args(["config", "list"]) + .output() + .expect("run mapbox"); + assert!(out.status.success()); + let lines = deliveries(&home, 3); + assert_eq!(lines[2]["stage"], "no_token", "{lines:?}"); + assert!( + sent.recv_timeout(Duration::from_millis(500)).is_err(), + "sent while resting" + ); + assert!( + checked.recv_timeout(Duration::from_millis(100)).is_err(), + "checked while resting" + ); +} + +/// The endpoint serving the rejected token a third time in a row is not +/// asked again for an hour, and the event is dropped meanwhile. +#[test] +fn the_rejected_token_served_again_waits_before_asking() { + let home = scratch("send-still-rejected"); + std::fs::write( + config_dir(&home).join("cli-token.json"), + // Two fetches already found it: this is the third in a row. + r#"{"token":"pk.old","rejected":true,"overridden":true,"failures":2}"#, + ) + .expect("write the state"); + let (_, token_url) = answer_once("200 OK", r#"{"token":"pk.old"}"#); + let out = command(&home) + .env( + "MAPBOX_INTERNAL_TELEMETRY_URL", + "http://127.0.0.1:1/events/v2", + ) + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", &token_url) + .args(["config", "list"]) + .output() + .expect("run mapbox"); + assert!(out.status.success()); + let state = cli_token_state(&home, "retry_at"); + assert_eq!(state["failures"], 3); + + // Nothing listens at the endpoint now, and nothing should try it. + let out = command(&home) + .env("MAPBOX_INTERNAL_TELEMETRY_LOG", "1") + .env( + "MAPBOX_INTERNAL_TELEMETRY_URL", + "http://127.0.0.1:1/events/v2", + ) + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", &token_url) + .args(["config", "list"]) + .output() + .expect("run mapbox"); + assert!(out.status.success()); + let lines = deliveries(&home, 1); + assert_eq!(lines.last().expect("a delivery line")["stage"], "no_token"); +} + +/// A token called invalid for something fixed in the Console stays the same +/// string, so the dead token served again is checked once more, and taken +/// back once it checks valid. +#[test] +fn the_rejected_token_served_again_is_taken_back_once_valid() { + let home = scratch("send-taken-back"); + std::fs::write( + config_dir(&home).join("cli-token.json"), + r#"{"token":"pk.fixed","rejected":true,"overridden":true}"#, + ) + .expect("write the state"); + let (_, token_url) = answer_once("200 OK", r#"{"token":"pk.fixed"}"#); + let (checked, tokens_url) = answer_once("200 OK", r#"{"code":"TokenValid"}"#); + let (received, url) = events_server(true); + let out = command(&home) + .env("MAPBOX_INTERNAL_TELEMETRY_URL", &url) + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", &token_url) + .env("MAPBOX_INTERNAL_TOKENS_URL", &tokens_url) + .args(["config", "list"]) + .output() + .expect("run mapbox"); + assert!(out.status.success()); + checked + .recv_timeout(Duration::from_secs(10)) + .expect("the token was checked"); + assert_eq!(sent_with(&received), "pk.fixed"); + assert_eq!(cli_token_state(&home, "pk.fixed")["rejected"], false); +} + +/// `config set telemetry off` covers the token fetch as +/// `MAPBOX_CLI_NO_TELEMETRY` does: telemetry is its only user. +#[test] +fn telemetry_off_fetches_no_token() { + let home = scratch("send-off-no-fetch"); + let (asked, token_url) = answer_once("200 OK", r#"{"token":"pk.fetched"}"#); + let (received, url) = events_server(true); + for args in [ + &["config", "set", "telemetry", "off"][..], + &["config", "list"], + ] { + let out = command(&home) + .env("MAPBOX_INTERNAL_TELEMETRY_URL", &url) + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", &token_url) + .args(args) + .output() + .expect("run mapbox"); + assert!(out.status.success()); + } + assert!( + asked.recv_timeout(Duration::from_secs(3)).is_err(), + "`telemetry off` still fetched the CLI's token" + ); + assert!(received.recv_timeout(Duration::from_millis(100)).is_err()); + assert!(!config_dir(&home).join("cli-token.json").exists()); +} + +/// With no config directory, a fetched token still has to be kept, or a dead +/// one is never marked and nothing backs off; so the directory is created. +#[test] +fn a_fetched_token_is_kept_even_without_a_config_directory() { + let home = scratch("send-fetched-no-dir"); + std::fs::remove_dir(config_dir(&home)).expect("remove the scratch config dir"); + let (_, token_url) = answer_once("200 OK", r#"{"token":"pk.fetched"}"#); + let (received, url) = events_server(true); + // History would create the directory first, and hide whether keeping + // the token does. + let out = command(&home) + .env("MAPBOX_HISTORY", "0") + .env("MAPBOX_INTERNAL_TELEMETRY_URL", &url) + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", &token_url) + .args(["config", "list"]) + .output() + .expect("run mapbox"); + assert!(out.status.success()); + assert_eq!(sent_with(&received), "pk.fetched"); + assert_eq!(cli_token_state(&home, "pk.fetched")["token"], "pk.fetched"); + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + let mode = std::fs::metadata(config_dir(&home)) + .expect("the config dir") + .permissions() + .mode(); + assert_eq!(mode & 0o777, 0o700); + } +} + +/// An existing config directory is written into as it is. +#[cfg(unix)] +#[test] +fn keeping_a_fetched_token_leaves_the_directory_mode_alone() { + use std::os::unix::fs::PermissionsExt; + + let home = scratch("send-fetched-mode"); + let dir = config_dir(&home); + std::fs::set_permissions(&dir, std::fs::Permissions::from_mode(0o755)).expect("chmod"); + let (_, token_url) = answer_once("200 OK", r#"{"token":"pk.fetched"}"#); + let (received, url) = events_server(true); + let out = command(&home) + .env("MAPBOX_INTERNAL_TELEMETRY_URL", &url) + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", &token_url) + .args(["config", "list"]) + .output() + .expect("run mapbox"); + assert!(out.status.success()); + assert_eq!(sent_with(&received), "pk.fetched"); + cli_token_state(&home, "pk.fetched"); + let mode = std::fs::metadata(&dir) + .expect("the config dir") + .permissions() + .mode(); + assert_eq!( + mode & 0o777, + 0o755, + "the write changed the directory's mode" + ); +} + +/// Only an answer that went through ends a cooldown: a `500` after one +/// expired says nothing about whether the API takes the token now. +#[test] +fn a_failed_send_does_not_end_a_cooldown() { + let home = scratch("send-cooldown-500"); + std::fs::write( + config_dir(&home).join("cli-token.json"), + r#"{"token":"pk.kept","overridden":true,"cooldowns":{"telemetry":{"until":1}}}"#, + ) + .expect("write the state"); + let (sent, events_url) = answer_once("500 Internal Server Error", "{}"); + let out = command(&home) + .env("MAPBOX_INTERNAL_TELEMETRY_LOG", "1") + .env( + "MAPBOX_INTERNAL_TELEMETRY_URL", + events_url.replace("/config/token.json", "/events/v2"), + ) + .args(["config", "list"]) + .output() + .expect("run mapbox"); + assert!(out.status.success()); + sent.recv_timeout(Duration::from_secs(10)) + .expect("the expired cooldown let the event go"); + let lines = deliveries(&home, 2); + assert_eq!(lines[1]["status"], 500, "{lines:?}"); + let state: Value = serde_json::from_str( + &std::fs::read_to_string(config_dir(&home).join("cli-token.json")).expect("the state"), + ) + .expect("the state is JSON"); + assert_eq!(state["cooldowns"]["telemetry"]["until"], 1, "{state}"); +} + +/// A refusal the Tokens API cannot explain still rests telemetry: sending +/// the same token on the next run would most likely be refused again, and +/// cost another check that may take seconds. +#[test] +fn an_unexplained_refusal_starts_a_cooldown_too() { + let home = scratch("send-unknown"); + std::fs::write( + config_dir(&home).join("cli-token.json"), + r#"{"token":"pk.kept","overridden":true}"#, + ) + .expect("write the state"); + let (_, events_url) = answer_once("403 Forbidden", r#"{"message":"Forbidden"}"#); + let (checked, tokens_url) = answer_once("500 Internal Server Error", "{}"); + let out = command(&home) + .env( + "MAPBOX_INTERNAL_TELEMETRY_URL", + events_url.replace("/config/token.json", "/events/v2"), + ) + .env("MAPBOX_INTERNAL_TOKENS_URL", &tokens_url) + .args(["config", "list"]) + .output() + .expect("run mapbox"); + assert!(out.status.success()); + checked + .recv_timeout(Duration::from_secs(10)) + .expect("the token was checked"); + let state = cli_token_state(&home, "cooldowns"); + assert_eq!(state["token"], "pk.kept"); + assert_eq!(state["rejected"], false); + assert!(state["cooldowns"]["telemetry"]["until"].is_u64(), "{state}"); +} diff --git a/tests/tilesets_cli_proxy.rs b/tests/tilesets_cli_proxy.rs index 16a4a66..3d4bce6 100644 --- a/tests/tilesets_cli_proxy.rs +++ b/tests/tilesets_cli_proxy.rs @@ -60,6 +60,8 @@ fn run(stub: &Path, args: &[&str]) -> Output { Command::new(env!("CARGO_BIN_EXE_mapbox")) .env("MAPBOX_CLI_NO_TELEMETRY", "1") .env_remove("MAPBOX_CLI_TOKEN") + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", "http://127.0.0.1:9/") + .env("MAPBOX_INTERNAL_TOKENS_URL", "http://127.0.0.1:9/") .args(args) .env("MAPBOX_TILESETS_CLI", stub) .env_remove("MAPBOX_ACCESS_TOKEN") @@ -275,6 +277,8 @@ fn an_environment_token_is_passed_through_untouched() { let output = Command::new(env!("CARGO_BIN_EXE_mapbox")) .env("MAPBOX_CLI_NO_TELEMETRY", "1") .env_remove("MAPBOX_CLI_TOKEN") + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", "http://127.0.0.1:9/") + .env("MAPBOX_INTERNAL_TOKENS_URL", "http://127.0.0.1:9/") .args(["tilesets-cli", "list", "someone"]) .env("MAPBOX_TILESETS_CLI", &stub) .env("MAPBOX_ACCESS_TOKEN", "pk.from-the-environment") @@ -296,6 +300,8 @@ fn use_login_prefers_the_stored_token_over_the_environment() { let output = Command::new(env!("CARGO_BIN_EXE_mapbox")) .env("MAPBOX_CLI_NO_TELEMETRY", "1") .env_remove("MAPBOX_CLI_TOKEN") + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", "http://127.0.0.1:9/") + .env("MAPBOX_INTERNAL_TOKENS_URL", "http://127.0.0.1:9/") .args(["--use-login", "tilesets-cli", "list", "stored-account"]) .env("MAPBOX_TILESETS_CLI", &stub) .env("MAPBOX_ACCESS_TOKEN", "pk.from-the-environment") @@ -322,6 +328,8 @@ fn without_use_login_the_environment_still_wins() { let output = Command::new(env!("CARGO_BIN_EXE_mapbox")) .env("MAPBOX_CLI_NO_TELEMETRY", "1") .env_remove("MAPBOX_CLI_TOKEN") + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", "http://127.0.0.1:9/") + .env("MAPBOX_INTERNAL_TOKENS_URL", "http://127.0.0.1:9/") .args(["tilesets-cli", "list", "stored-account"]) .env("MAPBOX_TILESETS_CLI", &stub) .env("MAPBOX_ACCESS_TOKEN", &env_token) @@ -351,6 +359,8 @@ fn use_login_without_a_login_is_an_error() { let output = Command::new(env!("CARGO_BIN_EXE_mapbox")) .env("MAPBOX_CLI_NO_TELEMETRY", "1") .env_remove("MAPBOX_CLI_TOKEN") + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", "http://127.0.0.1:9/") + .env("MAPBOX_INTERNAL_TOKENS_URL", "http://127.0.0.1:9/") .args(["--use-login", "tilesets-cli", "list", "someone"]) .env("MAPBOX_TILESETS_CLI", &stub) .env("MAPBOX_ACCESS_TOKEN", "pk.from-the-environment") @@ -384,6 +394,8 @@ fn the_config_dir_override_is_where_a_login_is_read_from() { let output = Command::new(env!("CARGO_BIN_EXE_mapbox")) .env("MAPBOX_CLI_NO_TELEMETRY", "1") .env_remove("MAPBOX_CLI_TOKEN") + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", "http://127.0.0.1:9/") + .env("MAPBOX_INTERNAL_TOKENS_URL", "http://127.0.0.1:9/") .args(["--use-login", "tilesets-cli", "list", "stored-account"]) .env("MAPBOX_TILESETS_CLI", &stub) .env("MAPBOX_ACCESS_TOKEN", "pk.from-the-environment") @@ -415,6 +427,8 @@ fn an_environment_token_for_another_account_is_called_out() { let output = Command::new(env!("CARGO_BIN_EXE_mapbox")) .env("MAPBOX_CLI_NO_TELEMETRY", "1") .env_remove("MAPBOX_CLI_TOKEN") + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", "http://127.0.0.1:9/") + .env("MAPBOX_INTERNAL_TOKENS_URL", "http://127.0.0.1:9/") .args(["tilesets-cli", "list", "me"]) .env("MAPBOX_TILESETS_CLI", &stub) // Parseable, unlike the `pk.from-the-environment` the precedence @@ -439,6 +453,8 @@ fn a_missing_binary_prints_install_guidance() { let output = Command::new(env!("CARGO_BIN_EXE_mapbox")) .env("MAPBOX_CLI_NO_TELEMETRY", "1") .env_remove("MAPBOX_CLI_TOKEN") + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", "http://127.0.0.1:9/") + .env("MAPBOX_INTERNAL_TOKENS_URL", "http://127.0.0.1:9/") .args([TOKEN[0], TOKEN[1], "tilesets-cli", "list", "someone"]) .env_remove("MAPBOX_TILESETS_CLI") .env("PATH", "") @@ -456,6 +472,8 @@ fn a_broken_override_blames_the_env_var() { let output = Command::new(env!("CARGO_BIN_EXE_mapbox")) .env("MAPBOX_CLI_NO_TELEMETRY", "1") .env_remove("MAPBOX_CLI_TOKEN") + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", "http://127.0.0.1:9/") + .env("MAPBOX_INTERNAL_TOKENS_URL", "http://127.0.0.1:9/") .args([TOKEN[0], TOKEN[1], "tilesets-cli", "list", "someone"]) .env("MAPBOX_TILESETS_CLI", "/nonexistent/tilesets") .output() @@ -500,6 +518,8 @@ fn the_environment_variable_is_not_warned_about() { let out = Command::new(env!("CARGO_BIN_EXE_mapbox")) .env("MAPBOX_CLI_NO_TELEMETRY", "1") .env_remove("MAPBOX_CLI_TOKEN") + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", "http://127.0.0.1:9/") + .env("MAPBOX_INTERNAL_TOKENS_URL", "http://127.0.0.1:9/") .args(["tilesets-cli", "list", "someone"]) .env("MAPBOX_TILESETS_CLI", &stub) .env("MAPBOX_YES", "1") @@ -547,6 +567,8 @@ fn a_forwarded_token_is_not_printed_by_debug() { let out = Command::new(env!("CARGO_BIN_EXE_mapbox")) .env("MAPBOX_CLI_NO_TELEMETRY", "1") .env_remove("MAPBOX_CLI_TOKEN") + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", "http://127.0.0.1:9/") + .env("MAPBOX_INTERNAL_TOKENS_URL", "http://127.0.0.1:9/") .args(["tilesets-cli", "--token", FAKE_TOKEN, "list", "someone"]) .env("MAPBOX_TILESETS_CLI", &stub) .env("MAPBOX_DEBUG", "1") diff --git a/tests/update_check.rs b/tests/update_check.rs index dc48cd7..3904a02 100644 --- a/tests/update_check.rs +++ b/tests/update_check.rs @@ -84,6 +84,8 @@ fn command(home: &Path) -> Command { .env_remove("MAPBOX_CLI_NO_TELEMETRY") // Telemetry stays on, as it is for a user, but goes nowhere. .env_remove("MAPBOX_CLI_TOKEN") + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", "http://127.0.0.1:9/") + .env("MAPBOX_INTERNAL_TOKENS_URL", "http://127.0.0.1:9/") .env("MAPBOX_INTERNAL_TELEMETRY_URL", "http://127.0.0.1:9/") .env("HOME", home) .env("XDG_CONFIG_HOME", home.join(".config")) @@ -447,6 +449,8 @@ fn under_a_pty( .env_remove("MAPBOX_NO_UPDATE_CHECK") .env_remove("MAPBOX_CLI_NO_TELEMETRY") .env_remove("MAPBOX_CLI_TOKEN") + .env("MAPBOX_INTERNAL_CLI_TOKEN_URL", "http://127.0.0.1:9/") + .env("MAPBOX_INTERNAL_TOKENS_URL", "http://127.0.0.1:9/") .env("MAPBOX_INTERNAL_TELEMETRY_URL", "http://127.0.0.1:9/") .env("HOME", home) .env("MAPBOX_CONFIG_DIR", config_dir(home))