Skip to content

[codex-analytics] add item lifecycle timing#20514

Merged
rhan-oai merged 1 commit intomainfrom
rhan/core-item-timing-production
May 4, 2026
Merged

[codex-analytics] add item lifecycle timing#20514
rhan-oai merged 1 commit intomainfrom
rhan/core-item-timing-production

Conversation

@rhan-oai
Copy link
Copy Markdown
Collaborator

@rhan-oai rhan-oai commented Apr 30, 2026

Why

Tool families already disagree on what their existing duration fields mean, so lifecycle latency should live on the shared item envelope instead of being inferred from per-tool execution fields. Carrying that envelope through app-server notifications gives downstream consumers one reusable timing signal without pretending every tool has the same execution semantics.

What changed

  • Adds started_at_ms to core ItemStartedEvent values and completed_at_ms to core ItemCompletedEvent values.
  • Populates those timestamps in the shared session lifecycle emitters, so protocol-native items get timing without each producer tracking its own clock state.
  • Exposes startedAtMs on app-server item/started notifications and completedAtMs on item/completed notifications.
  • Maps the lifecycle timestamps through the app-server boundary while leaving legacy-converted notifications nullable when no lifecycle timestamp exists.
  • Regenerates the app-server JSON schema and TypeScript fixtures for the notification-envelope change and updates downstream fixtures that construct those notifications directly.
  • Extends the existing web-search and image-generation integration flows to assert the new lifecycle timestamps on the native item events.

Verification

  • cargo check -p codex-protocol -p codex-core -p codex-app-server-protocol -p codex-app-server -p codex-tui -p codex-exec -p codex-app-server-client
  • cargo test -p codex-core --test all web_search_item_is_emitted
  • cargo test -p codex-core --test all image_generation_call_event_is_emitted
  • cargo test -p codex-app-server-protocol

Stack created with Sapling. Best reviewed with ReviewStack.

@rhan-oai rhan-oai requested a review from a team as a code owner April 30, 2026 22:28
@rhan-oai rhan-oai force-pushed the rhan/core-item-timing-production branch from 5c272b8 to c6e0ea5 Compare April 30, 2026 22:57
@rhan-oai rhan-oai marked this pull request as draft April 30, 2026 22:59
@rhan-oai rhan-oai force-pushed the rhan/core-item-timing-production branch 2 times, most recently from 7a61984 to 88c1fb0 Compare April 30, 2026 23:32
rhan-oai added a commit that referenced this pull request May 1, 2026
## Why

Several analytics event families need the same per-thread attribution
state: the app-server client/runtime associated with a thread and, for
lifecycle-oriented events, the thread metadata captured during
initialization. Keeping connection ids and lifecycle metadata in
separate maps made each consumer rebuild the same thread context and
made subagent attribution harder to resolve consistently.

## What changed

- Replaces the separate thread connection and metadata maps with one
reducer-owned `threads` map.
- Routes guardian, compaction, turn-steer, and turn analytics through
shared thread-state lookups while preserving turn-origin attribution for
turn events and request-origin attribution for steer events.
- Lets newly observed spawned subagent threads inherit their parent
thread connection so later thread-scoped analytics can resolve through
the same state model.
- Adds regression coverage for standalone `SubAgentThreadStarted`
publication plus the `SubAgentSource::ThreadSpawn` parent fallback
through a thread-scoped consumer that depends on inherited connection
state.

## Verification

- `cargo test -p codex-analytics`

---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/20300).
* #18748
* #18747
* #17090
* #17089
* #20239
* #20515
* #20514
* __->__ #20300
Base automatically changed from rhan/thread-analytics-state to main May 1, 2026 01:58
@rhan-oai rhan-oai force-pushed the rhan/core-item-timing-production branch 2 times, most recently from d55b667 to 42cdca0 Compare May 1, 2026 02:53
@rhan-oai rhan-oai force-pushed the rhan/core-item-timing-production branch 6 times, most recently from 53976e4 to 96a73c3 Compare May 1, 2026 18:31
@rhan-oai rhan-oai force-pushed the rhan/core-item-timing-production branch 2 times, most recently from 92bf80c to 2d55681 Compare May 2, 2026 06:55
@rhan-oai rhan-oai force-pushed the rhan/core-item-timing-production branch from 2d55681 to db23f16 Compare May 3, 2026 03:38
@rhan-oai rhan-oai changed the title [codex-analytics] add core item timing production [codex-analytics] add item lifecycle timing May 3, 2026
@rhan-oai rhan-oai force-pushed the rhan/core-item-timing-production branch 2 times, most recently from 73ff402 to 24af98e Compare May 4, 2026 17:28
@rhan-oai rhan-oai marked this pull request as ready for review May 4, 2026 17:32
@rhan-oai rhan-oai requested a review from owenlin0 May 4, 2026 17:32
@rhan-oai rhan-oai force-pushed the rhan/core-item-timing-production branch from 24af98e to 4c982ac Compare May 4, 2026 17:38
Copy link
Copy Markdown
Collaborator

@owenlin0 owenlin0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's see if we can make these timestamps non-optional. first at the core layer, then at app-server (if possible). want to give it a shot? this would be much nicer instead of keeping around optional forever 🙏

Comment thread codex-rs/protocol/src/protocol.rs Outdated
pub item: TurnItem,
#[serde(default, skip_serializing_if = "Option::is_none")]
#[ts(optional)]
pub started_at_ms: Option<i64>,
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's make this non-optional. it's safe because we never persisted ItemStartedEvent in rollouts :)

Comment thread codex-rs/protocol/src/protocol.rs Outdated
pub item: TurnItem,
#[serde(default, skip_serializing_if = "Option::is_none")]
#[ts(optional)]
pub completed_at_ms: Option<i64>,
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's make this non-optional too. Can we do something like this?

fn unknown_completed_at_ms() -> i64 {
    0
}

pub struct ItemCompletedEvent {
    // other fields
    // ...
    #[serde(default = "unknown_completed_at_ms")]
    pub completed_at_ms: i64,
}

I think we need this since we read EventMsg::ItemCompleted(TurnItem::Plan(_)) from rollout files, so we want to keep that deserializing properly with a serde(default).

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and i think it's totally fine to return 0 since app-server only exposes item/completed notification while streaming a thread live. APIs that return thread history doesn't use this.

@rhan-oai rhan-oai force-pushed the rhan/core-item-timing-production branch 3 times, most recently from fbff229 to 9250609 Compare May 4, 2026 20:46
@rhan-oai rhan-oai requested a review from owenlin0 May 4, 2026 20:57
@rhan-oai rhan-oai force-pushed the rhan/core-item-timing-production branch 2 times, most recently from 9f8032e to 6189df0 Compare May 4, 2026 21:10
pub thread_id: String,
pub turn_id: String,
/// Unix timestamp (in milliseconds) when this item lifecycle started.
pub started_at_ms: i64,
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's make sure to add the #[ts(type = "number")] annotation for this field

pub thread_id: String,
pub turn_id: String,
/// Unix timestamp (in milliseconds) when this item lifecycle completed.
pub completed_at_ms: i64,
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here too

Comment thread codex-rs/protocol/src/protocol.rs Outdated
pub turn_id: String,
pub item: TurnItem,
// Old rollout files may not have this field. Default to 0 so they still
// deserialize after tightening the core event contract.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: can we call out ItemCompleted for PlanItem in particular? that's the only one that ever got persisted

Copy link
Copy Markdown
Collaborator

@owenlin0 owenlin0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

small comments but overall lgtm!

@rhan-oai rhan-oai force-pushed the rhan/core-item-timing-production branch 2 times, most recently from a011365 to 22edd59 Compare May 4, 2026 21:49
@rhan-oai rhan-oai force-pushed the rhan/core-item-timing-production branch from 22edd59 to 7fd7c90 Compare May 4, 2026 22:07
@rhan-oai rhan-oai enabled auto-merge (squash) May 4, 2026 22:08
@rhan-oai rhan-oai disabled auto-merge May 4, 2026 22:26
@rhan-oai rhan-oai enabled auto-merge (squash) May 4, 2026 22:26
@rhan-oai rhan-oai merged commit aee1fe2 into main May 4, 2026
26 checks passed
@rhan-oai rhan-oai deleted the rhan/core-item-timing-production branch May 4, 2026 22:33
@github-actions github-actions Bot locked and limited conversation to collaborators May 4, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants