Skip to content
@TraceLite-AI

TraceLite-AI

TraceLite AI builds traceable evaluation infrastructure for reliable AI agents.

TraceLite AI

Open research on AI agent reliability, execution forensics, environments, and post-training.

Website arXiv Hugging Face Contact


Benchmarks & Research

Project Venue / Links Focus
TraceTrial-Bench arXiv:2610.32285 · Report · Leaderboard 95 mechanism-focused tasks across 7 dimensions (C, E, G, L, O, T, V) separating model capability from agent harness reliability
Judging-by-Evidence Research Code Typed evidence acquisition and verification for Agent-as-a-Judge
MM-VeriRubric EMNLP 2026 Verifiable evidence-grounded rubrics for multimodal reward models
DPS EMNLP 2026 Disentangled privileged states for simulator-assisted multi-turn dialogue policy training
Awesome-Agentic-Environments Curated Survey Papers, benchmarks, and frameworks for building and scaling LLM agent environments

Upstream Ecosystem Contributions

Using TraceTrial-Bench as a diagnostic probe, we actively contribute bug reports, security fixes, and runtime improvements back to open-source agent frameworks:

  • OpenHands (software-agent-sdk) — 10 pull requests addressing fail-closed sub-agent confirmation & security analyzer inheritance (#7001, priority:high), secret rotation masking (#7003), cross-restart sub-agent recovery (#6986), MCP tool collision isolation & structured payload preservation (#6988, #6989), sub-agent event streaming (#6997), entity-preserving condensation (#6998), and deterministic terminal timeout/truncation contracts (#6987, #7024, #7025).
  • OpenCode (anomalyco/opencode) — Fixes and proposals for direct MCP structured results (#64838), sub-agent session permission inheritance (#64841), non-interactive --format json background event streaming (#64852), and native interactive PTY sessions (#65017).
  • Aider (Aider-AI/aider) — Non-interactive --yes-shell-commands execution and shell output reflection (#6002, #6003).

Follow Us

Xiaohongshu

Xiaohongshu QR Code
WeChat Official Account

WeChat Official Account QR Code

Pinned Loading

  1. TraceTrial-Bench TraceTrial-Bench Public

    TraceTrial-Bench: A benchmark for AI agent harness reliability and failure attribution, featuring 95 tasks across 7 components, execution tracing, and evidence-based evaluation that separates task …

    Python 28 8

  2. Awesome-Agentic-Environments Awesome-Agentic-Environments Public

    A curated collection of papers, benchmarks, and frameworks on agentic environments — how they are built, scaled, and used for LLM agent post-training and evaluation.

    HTML 7 5

Repositories

Showing 10 of 18 repositories

People

This organization has no public members. You must be a member to see who’s a part of this organization.

Top languages

Loading…

Most used topics

Loading…