Skip to content

[Task]: Add time-decay to agent reputation scores #83

Description

@grantfox-oss

packages/registry/src/reputation.ts computes an agent's reputation from accumulated feedback, but the score has no notion of recency. An agent that performed well months ago and has since degraded (or gone offline) keeps a high score forever, and the orchestrator's selector keeps preferring it. Conversely, a newer agent performing well right now is under-weighted against long-standing history. Reputation should track current behavior, not lifetime totals.

Goal

Weight each feedback event by its age so recent outcomes dominate the aggregate, using an exponential decay with a configurable half-life.

Requirements & constraints

  • Decay is exponential by age (a half-life parameter: an event one half-life old counts half as much). The half-life is configurable via an env var (e.g. REPUTATION_HALF_LIFE_MS) with a documented default.
  • Apply the weight consistently wherever feedback is aggregated (weighted average for ratings, weighted counts for success rate).
  • The output score's range and shape must be unchanged so selector.ts needs no modification.
  • Document the tradeoff in a comment: decay improves responsiveness at the cost of "forgetting" old evidence; the half-life is the knob that balances the two.

Design decisions left to the implementer

  • The exact weighting scheme (how ratings vs success-rate incorporate the weight) and how a brand-new agent with little/no feedback is handled (avoid a cold-start agent looking artificially perfect or terrible).

Testing constraint

  • Tests must be deterministic: inject or fix the "now" timestamp rather than calling Date.now() at assert time, so results don't drift with wall-clock.

Acceptance criteria

  • Aggregation applies exponential decay by feedback age
  • Half-life configurable via env with a documented default
  • Output range/shape unchanged (no selector.ts change required)
  • Cold-start behavior for sparse-feedback agents is defined and tested
  • Deterministic tests cover: recent-vs-old weighting, all-recent ≈ prior behavior, and decay monotonicity
  • npm test passes

Relevant files

  • packages/registry/src/reputation.ts, packages/registry/src/reputation.test.ts

Notes for contributors

If you'd like to work on this, comment below so we can assign it to you.
Questions welcome — see CONTRIBUTING.md for setup
and workflow.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions