feat(aura-cli): display LLM reasoning output for coordinator and workers#165
feat(aura-cli): display LLM reasoning output for coordinator and workers#165justintime4tea wants to merge 1 commit into
Conversation
6575e49 to
6c7b180
Compare
|
Promptless prepared a documentation update related to this change. Triggered by PR #165 Added a "Reasoning Output" section to the CLI documentation covering how LLM reasoning is displayed for both single-agent and orchestration modes, including Review: Add CLI guide with one-shot output, logging, OTel, and reasoning display documentation |
6c7b180 to
a3c7cf7
Compare
|
In the commit message you can reference a github issue the same way
both are legal / satisfy the commit linter. and still get all the deep linking benefits in all the places. your commit message has |
Shearerbeard
left a comment
There was a problem hiding this comment.
Mostly just questions and observations - one truncation concern we should look into pre merge. It runs absolutely amazing.
| "parent_agent_id": "coordinator", | ||
| "session_id": "s1" | ||
| }); | ||
| let events = vec![sse("aura.reasoning", &data.to_string()), sse("", "[DONE]")]; |
There was a problem hiding this comment.
Thinking ahead - I wonder if we should canonicalize some of these strings as consts along with the SSE event structs
There was a problem hiding this comment.
I think we might have some consts already for these event names if I recall... let me do some cleanup on this.
| |_, _| {}, | ||
| |_, _, _, _| {}, | ||
| |_, _| {}, | ||
| |_, _, _| {}, | ||
| |_, _| {}, | ||
| |_, _| {}, |
There was a problem hiding this comment.
Just curiosity here - would this be a candidate for a non matching _ block since it seems like were only acting on certain combos?
There was a problem hiding this comment.
There's definitely room for improvement here with regards to currently requiring multiple no-op handlers to fill the function params. I could switch to using a trait that has default no-ops and then you just supply the handler you wish to override. I'll do some cleanup here.
| content: String, | ||
| agent_id: String, | ||
| #[serde(default)] | ||
| fields: BTreeMap<String, serde_json::Value>, |
There was a problem hiding this comment.
Btreemap because your preserving order?
There was a problem hiding this comment.
Yes, preserving order. With BTreeMap the /expand fields tree renders consistently and tests aren't subject to HashMap's randomized iteration. If we ever needed to preserve the original JSON field order we could switch to IndexMap, but stable sorted order is what I was going for here.
| // on_reasoning — no-op in oneshot mode (reasoning chunks | ||
| // would interleave with the rendered response; the user | ||
| // asked for the answer, not the thought process) | ||
| |_content, _agent_id, _fields| {}, | ||
| // on_raw_event — no-op in oneshot mode |
There was a problem hiding this comment.
Similar question as above. Again just an observation from a shallow diff
There was a problem hiding this comment.
Similar to other comment - I'll switch to using a trait that has default no-ops. I'll clean this up.
| let display = if text.chars().count() > 120 { | ||
| let prefix: String = text.chars().take(117).collect(); | ||
| format!("{prefix}...") |
There was a problem hiding this comment.
Only concerning piece I'm seeing here - is this truncation utf8/emoji safe? We have some special handling with safe_truncation somewhere in aura that we use a lot but you approach this one a bit differently so I'm not sure if it applies.
There was a problem hiding this comment.
It's UTF-8 safe because I'm using .chars().take()/.collect(), which only ever cuts on character boundaries, so it can't split a multi-byte sequence and panic.
safe_truncate solves the byte-budget case (it needs floor_char_boundary because it slices by byte index) but here I'm truncating to a visible-character count for a preview, so the character-based approach is the right unit and safe_truncate doesn't quite fit.
There is something though that neither handles and that is multi-scalar emoji (ZWJ/skin-tone/flags) being cut mid-grapheme. I didn't come up with this, Claude did when I asked it to further look into my truncation to see if it could find issues with it that I hadn't and to answer your question of "is it emoji safe". This, it turns out, is purely cosmetic - no crash, but multi-scalar emoji's aren't safe (either with this or safe_truncate). I could switch (this, not safe_truncate) to unicode-segmentation::graphemes though if we want to be sure that we not cut off mid-emoji :)
5b65663 to
d35027d
Compare
Shows "Reasoning: ..." output both at the top-level of the output, for the coordinator, and in-line with a given task for worker reasoning. Ref: GH-114
d35027d to
85377ba
Compare
Shows "Reasoning: ..." output both at the top-level of the output, for the coordinator, and in-line with a given task for worker reasoning.
Here is an example of running against an orchestration enabled AURA config using Anthropic
claude-sonnet-4-6model.Here is an example of running against a single-agent AURA config using Anthropic
claude-sonnet-4-6model.Ref: GH-114