Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
7e5a6e0
Add the universal error record to runvil-types
vinniefalco Nov 3, 2026
8993c93
Add harness codes, a run fact, and default prose
vinniefalco Nov 3, 2026
c790985
Re-export the universal error from runvil and runvil-plugin
vinniefalco Nov 3, 2026
b34889f
Convert a store failure into the universal error
vinniefalco Nov 3, 2026
dded648
Replace IdError with the universal error
vinniefalco Nov 3, 2026
00c252d
Replace ParseRoleError with the universal error
vinniefalco Nov 3, 2026
074b1f5
Replace ParseErrorKindError with the universal error
vinniefalco Nov 3, 2026
c944666
Replace TemperatureError with the universal error
vinniefalco Nov 3, 2026
bc85308
Replace ToolSchemaError with the universal error
vinniefalco Nov 3, 2026
7519860
Replace SecretError with the universal error
vinniefalco Nov 3, 2026
bfd4177
Replace GatewayConfigError with the universal error
vinniefalco Nov 3, 2026
058b6ec
Replace ServiceError with the universal error
vinniefalco Nov 3, 2026
4fa7c20
Replace InstallError with the universal error
vinniefalco Nov 3, 2026
ef0c594
Add with_cause to the universal error
vinniefalco Nov 3, 2026
1a63a8d
Replace ToolError and ToolErrorKind with the universal error
vinniefalco Nov 3, 2026
b24268a
Replace CompletionError with the universal error
vinniefalco Nov 3, 2026
c75d678
Replace GatewaySearchError with the universal error
vinniefalco Nov 3, 2026
ef07d6e
Replace RecorderError and MemoryError with the universal error
vinniefalco Nov 3, 2026
7406dfd
Stop with_cause repeating text the message ends with
vinniefalco Nov 3, 2026
ee6318a
Replace PluginFailure and LoadError with the universal error
vinniefalco Nov 3, 2026
50feb65
Replace OutputError with the universal error
vinniefalco Nov 3, 2026
7307674
Replace InputFileError with the universal error
vinniefalco Nov 3, 2026
c7664f0
Replace CurrentModelError with the universal error
vinniefalco Nov 3, 2026
c20efb4
Name the prompt's file when it is parsed
vinniefalco Nov 3, 2026
e2a581c
Replace ParseError with the universal error
vinniefalco Nov 3, 2026
e0abd45
Replace PrepareError with the universal error
vinniefalco Nov 3, 2026
5bb3bb3
Replace HarnessError with the universal error
vinniefalco Nov 3, 2026
cc7ba27
Replace SubstitutionError with the universal error
vinniefalco Nov 3, 2026
4853b52
Replace the Engine's Error with the universal error
vinniefalco Nov 3, 2026
0c889cb
Replace the Lua crate's Error and Raised with the universal error
vinniefalco Nov 3, 2026
8e8db97
Name the 8xx error family for the caller
vinniefalco Nov 3, 2026
1c04001
Update async-executor past its yanked release
vinniefalco Nov 3, 2026
4ec31cc
Keep Engine and Harness failures on the universal Error
vinniefalco Nov 3, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
1 change: 1 addition & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ Seven terms have exactly one meaning each, everywhere in this repository: code c
- A Cargo feature gates a real constraint such as a toolchain requirement or heavy native build. It does not describe product shape. Feature-disabled builds must not leak optional types into core paths.
- Runtime and serve paths never compile native dependencies or invoke build tools.
- Unsafe code stays in its explicitly owned boundary.
- Engine and Harness crates report every failure as the universal `Error` in `runvil-types`, which the `runvil` facade re-exports. A new kind of failure is a new `ErrorCode` variant in its hundreds family, with a number never used before, plus a `FactKey` variant only when no existing key names the value. They add no other error enum or struct, no `thiserror` derive, and no other `impl std::error::Error`; a foreign cause joins the message through `Error::with_cause`. `VfsError` and `ParsePathReasonError` in `runvil-vfs` are the only exceptions, and test code may define a throwaway error type to exercise `with_cause`.
- Comments explain a non-obvious constraint, ordering requirement, or workaround. Every platform or external-bug workaround cites its upstream issue URL in the explanatory comment.
- JSON that reaches a recorder or a replay comparison round-trips exactly - `to_value`, `to_string`, `from_str` yield an identical value, object keys stay canonical (sorted), numbers must be finite. Exact parsing (`float_roundtrip`) carries that guarantee; a value derived and then logged is additionally rounded to its meaningful precision at the source.

Expand Down
11 changes: 2 additions & 9 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion crates/build-xtask/src/facade_shape-tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ fn every_allowed_form_passes() {
let text = r#"#![doc = include_str!("lib.md")]

pub use runvil_engine::Run;
pub use runvil_model_client::model::CompletionError as ModelFailure;
pub use runvil_model_client::model::CompletionOptions as ModelOptions;
pub use ::runvil_engine::Step;

/// The effects a run issues.
Expand Down
7 changes: 3 additions & 4 deletions crates/harness-gateway-client/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,9 @@ harness.workspace = true
# result, and error vocabulary, which `GatewaySearch` implements.
plugin-web.workspace = true
# The Engine's public API: the chat vocabulary (Message, ToolSchema,
# Completion, CompletionError) the wire code reads and builds through its
# public constructors, and the canonical metrics and model catalog
# vocabulary.
# Completion) and the universal error the wire code reads and builds
# through its public constructors, and the canonical metrics and model
# catalog vocabulary.
runvil.workspace = true
# `SearchProvider` is an `async_trait` trait.
async-trait.workspace = true
Expand All @@ -31,7 +31,6 @@ serde.workspace = true
# The request body, every streamed payload, and the reassembled response
# are JSON.
serde_json.workspace = true
thiserror.workspace = true
# `time` for the per-receive timeout. Non-test code spawns no task.
tokio = { workspace = true, features = ["time"] }
url.workspace = true
Expand Down
11 changes: 6 additions & 5 deletions crates/harness-gateway-client/src/broker.rs
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,9 @@ use std::sync::Arc;

use harness::{BoxFuture, InferenceBroker};
use runvil::effect::Round;
use runvil::error::Error;
use runvil::model::{
Completion, CompletionError, CompletionOptions, Message, ModelBinding, ModelCatalog, ToolSchema,
Completion, CompletionOptions, Message, ModelBinding, ModelCatalog, ToolSchema,
};

use crate::catalog::fetch_model_catalog;
Expand Down Expand Up @@ -90,7 +91,7 @@ impl GatewayBroker {
tools: Vec<ToolSchema>,
options: CompletionOptions,
on_piece: Arc<dyn Fn(StreamDelta) + Send + Sync>,
) -> BoxFuture<Result<Box<Completion>, CompletionError>> {
) -> BoxFuture<Result<Box<Completion>, Error>> {
self.round(messages, tools, options, move |piece| on_piece(piece))
}

Expand All @@ -101,7 +102,7 @@ impl GatewayBroker {
tools: Vec<ToolSchema>,
options: CompletionOptions,
on_piece: impl Fn(StreamDelta) + Send + Sync + 'static,
) -> BoxFuture<Result<Box<Completion>, CompletionError>> {
) -> BoxFuture<Result<Box<Completion>, Error>> {
let client = self.client.clone();
Box::pin(async move {
// An empty advertisement sends no `tools` field at all, the
Expand All @@ -116,7 +117,7 @@ impl GatewayBroker {
}

impl InferenceBroker for GatewayBroker {
fn models(&self) -> BoxFuture<Result<ModelCatalog, CompletionError>> {
fn models(&self) -> BoxFuture<Result<ModelCatalog, Error>> {
let api_root = self.endpoint.url().to_owned();
let key = self.key.clone();
Box::pin(async move { fetch_model_catalog(&api_root, key.expose()).await })
Expand All @@ -129,7 +130,7 @@ impl InferenceBroker for GatewayBroker {
tools: Vec<ToolSchema>,
options: CompletionOptions,
_round: Round,
) -> BoxFuture<Result<Box<Completion>, CompletionError>> {
) -> BoxFuture<Result<Box<Completion>, Error>> {
self.round(messages, tools, options, |_| {})
}
}
Expand Down
107 changes: 56 additions & 51 deletions crates/harness-gateway-client/src/catalog-tests.rs
Original file line number Diff line number Diff line change
@@ -1,9 +1,19 @@
//! Tests for the model catalog fetch against an axum mock gateway: the body
//! bounds, the preserved failure sources, the entry filter, and failure
//! messages that carry no Gateway-supplied text.
//! bounds, the failure causes kept in the message, the entry filter, and
//! failure prose that carries no Gateway-supplied text.

use runvil::error::ErrorCode;

use super::*;
use crate::CompletionErrorKind;

/// The text the fetch appended to `err`'s message after its prose: the
/// provider detail or the cause, which the prose never holds.
fn appended(err: &Error) -> &str {
err.message
.strip_prefix(err.prose.as_str())
.and_then(|rest| rest.strip_prefix(": "))
.unwrap_or_else(|| panic!("the message extends the prose: {err:?}"))
}

async fn spawn_models(app: axum::Router) -> std::net::SocketAddr {
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
Expand Down Expand Up @@ -31,12 +41,12 @@ async fn fetch_model_catalog_bounds_and_reports_non_success_body() {
let err = fetch_model_catalog(&format!("http://{addr}"), "tok")
.await
.expect_err("a 500 response must surface as an error");
assert_eq!(err.kind(), CompletionErrorKind::ServerError);
assert_eq!(err.code, ErrorCode::ServerError);
assert_eq!(
err.to_string(),
err.prose,
"the model backend reported a fault of its own (status 500)"
);
let detail = err.detail().expect("the bounded body is the detail");
let detail = appended(&err);
assert!(
detail.len() < MAX_CATALOG_ERROR_BODY + 128,
"the error-path body must be bounded, got {} bytes",
Expand All @@ -62,22 +72,25 @@ async fn fetch_model_catalog_bounds_an_oversized_success_body() {
let err = fetch_model_catalog(&format!("http://{addr}"), "tok")
.await
.expect_err("an oversized success body must be refused");
assert_eq!(err.kind(), CompletionErrorKind::MalformedResponse);
assert_eq!(err.code, ErrorCode::MalformedResponse);
assert!(
err.to_string().contains("exceeds"),
"the bound must report the size limit, got {err:?}"
);
assert_eq!(err.detail(), None, "our own wording is not provider text");
assert_eq!(
err.message, err.prose,
"our own wording is not provider text"
);
}

#[tokio::test]
async fn fetch_model_catalog_preserves_the_json_decode_source() {
async fn fetch_model_catalog_keeps_the_json_decode_error_in_the_message() {
use axum::Router;
use axum::routing::get;

// MODEL-009: a 200 body that is not a valid model list is classified as
// MalformedResponse, and the underlying `serde_json::Error` survives as
// the error-chain `#[source]` rather than being flattened into the text.
// MalformedResponse, and the underlying `serde_json::Error` is appended
// to the message and kept out of the prose.
async fn models() -> (axum::http::StatusCode, String) {
(axum::http::StatusCode::OK, "{ this is not json".to_owned())
}
Expand All @@ -87,13 +100,12 @@ async fn fetch_model_catalog_preserves_the_json_decode_source() {
let err = fetch_model_catalog(&format!("http://{addr}"), "tok")
.await
.expect_err("an undecodable body must surface as an error");
assert_eq!(err.kind(), CompletionErrorKind::MalformedResponse);
let source =
std::error::Error::source(&err).expect("the decode error must be a preserved source");
assert!(
source.downcast_ref::<serde_json::Error>().is_some(),
"the preserved source must be the JSON decode error, got {source}"
);
assert_eq!(err.code, ErrorCode::MalformedResponse);
let decode = serde_json::from_slice::<ModelsListResponse>(b"{ this is not json")
.expect_err("the body is not JSON")
.to_string();
assert_eq!(appended(&err), decode);
assert!(!err.prose.contains(&decode), "got {err:?}");
}

#[tokio::test]
Expand Down Expand Up @@ -194,17 +206,17 @@ async fn fetch_model_catalog_still_rejects_a_zero_context_window() {
let err = fetch_model_catalog(&format!("http://{addr}"), "tok")
.await
.expect_err("a zero context window is malformed");
assert_eq!(err.kind(), CompletionErrorKind::MalformedResponse);
assert!(err.to_string().contains("zero-token"), "got {err:?}");
assert!(!err.to_string().contains("broken"), "got {err:?}");
assert_eq!(err.code, ErrorCode::MalformedResponse);
assert!(err.prose.contains("zero-token"), "got {err:?}");
assert!(!err.prose.contains("broken"), "got {err:?}");
assert!(
err.detail().is_some_and(|detail| detail.contains("broken")),
"the model name belongs in the detail, got {err:?}"
appended(&err).contains("broken"),
"the model name belongs in the message only, got {err:?}"
);
}

/// Serves `data` as the gateway model list and returns the fetch failure.
async fn fetch_rejection(data: serde_json::Value) -> CompletionError {
async fn fetch_rejection(data: serde_json::Value) -> Error {
let list = serde_json::json!({ "object": "list", "data": data });
let app = axum::Router::new().route(
"/models",
Expand All @@ -217,34 +229,32 @@ async fn fetch_rejection(data: serde_json::Value) -> CompletionError {
}

#[tokio::test]
async fn fetch_model_catalog_moves_the_name_of_a_model_without_a_thinking_mode_to_detail() {
async fn fetch_model_catalog_keeps_the_name_of_a_model_without_a_thinking_mode_out_of_the_prose() {
let err = fetch_rejection(serde_json::json!([
{ "id": "thoughtless", "object": "model", "kind": "chat",
"description": "d", "context": 4096 }
]))
.await;
assert_eq!(err.kind(), CompletionErrorKind::MalformedResponse);
assert!(!err.to_string().contains("thoughtless"), "got {err:?}");
assert_eq!(err.code, ErrorCode::MalformedResponse);
assert!(!err.prose.contains("thoughtless"), "got {err:?}");
assert!(
err.detail()
.is_some_and(|detail| detail.contains("thoughtless")),
"the model name belongs in the detail, got {err:?}"
appended(&err).contains("thoughtless"),
"the model name belongs in the message only, got {err:?}"
);
}

#[tokio::test]
async fn fetch_model_catalog_moves_a_duplicate_id_to_detail() {
async fn fetch_model_catalog_keeps_a_duplicate_id_out_of_the_prose() {
let entry = serde_json::json!(
{ "id": "twin", "object": "model", "kind": "chat",
"description": "d", "context": 4096, "thinking": "never" }
);
let err = fetch_rejection(serde_json::json!([entry.clone(), entry])).await;
assert_eq!(err.kind(), CompletionErrorKind::MalformedResponse);
assert!(!err.to_string().contains("twin"), "got {err:?}");
assert_eq!(err.code, ErrorCode::MalformedResponse);
assert!(!err.prose.contains("twin"), "got {err:?}");
assert!(
err.detail()
.is_some_and(|detail| detail.contains("gateway/twin")),
"the duplicate identity belongs in the detail, got {err:?}"
appended(&err).contains("gateway/twin"),
"the duplicate identity belongs in the message only, got {err:?}"
);
}

Expand All @@ -262,8 +272,8 @@ async fn fetch_model_catalog_bounds_a_gateway_supplied_detail() {
];
for data in rejections {
let err = fetch_rejection(data).await;
assert_eq!(err.kind(), CompletionErrorKind::MalformedResponse);
let detail = err.detail().expect("the model name is the detail");
assert_eq!(err.code, ErrorCode::MalformedResponse);
let detail = appended(&err);
assert!(
detail.chars().count() <= MAX_CATALOG_ERROR_BODY,
"a Gateway-supplied detail must be bounded, got {} chars",
Expand All @@ -279,20 +289,20 @@ async fn fetch_model_catalog_keeps_an_invalid_id_out_of_the_message() {
"description": "d", "context": 4096, "thinking": "never" }
]))
.await;
assert_eq!(err.kind(), CompletionErrorKind::MalformedResponse);
assert_eq!(err.code, ErrorCode::MalformedResponse);
let message = err.to_string();
assert!(!message.contains("smuggled"), "got {err:?}");
assert!(!message.contains('\u{7}'), "got {err:?}");
}

#[tokio::test]
async fn fetch_model_catalog_preserves_a_body_read_failure_source() {
async fn fetch_model_catalog_keeps_a_body_read_failure_in_the_message() {
use tokio::io::{AsyncReadExt, AsyncWriteExt};

// MODEL-010: a non-success response whose body cannot be fully read
// (the server promises a large body then drops the connection) must
// surface as a typed transport failure that keeps the `reqwest::Error`
// as its `#[source]`, not display text.
// surface as a typed transport failure whose message, not its prose,
// holds the `reqwest::Error`.
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let addr = listener.local_addr().unwrap();
tokio::spawn(async move {
Expand All @@ -310,15 +320,10 @@ async fn fetch_model_catalog_preserves_a_body_read_failure_source() {
let err = fetch_model_catalog(&format!("http://{addr}"), "tok")
.await
.expect_err("a truncated error body must surface as an error");
assert_eq!(err.kind(), CompletionErrorKind::Transport);
assert_eq!(
err.to_string(),
"the connection to the model backend failed"
);
let source =
std::error::Error::source(&err).expect("the read failure must be a preserved source");
assert_eq!(err.code, ErrorCode::Transport);
assert_eq!(err.prose, "the connection to the model backend failed");
assert!(
source.downcast_ref::<reqwest::Error>().is_some(),
"the preserved source must be the reqwest read error, got {source}"
!appended(&err).is_empty(),
"the message holds the reqwest read error, got {err:?}"
);
}
Loading
Loading