Proposal: adding benchmark stats to source files

[bruno-dasilva]

Discussion

I've opened some discussions with people around the best way to get benchmark stats out of teiserver events into somewhere graphable. Seems if we have ETL setup via this repo/dbt, it's a great option.

There's a few considerations I see, but I'm looking for more eyes:

1. what is the latency/freshness of data? ie. how often does data get exported out of teiserver? While data doesn't need to be realtime, it would be nice if it was fresher than once a week (ideally once a day) so we can check in after releases.
2. is there any concern about the underlying data (see data section below) being fully publically available? I assume no.
3. are we able create a source of JUST the system:benchmark rows, WITHOUT user_id column? We'd want to avoid dumping the other data in the table.

Let me know what you think.

Context

End Goal

I'd like to be able to start tracking "Engine Health" or "Engine Performance" over time. See this discussion in engine repo:
See: beyond-all-reason/RecoilEngine#2929

Data in teiserver

Teiserver has an analytics pipeline of sorts where all messages that come in as client events gets stored into telemetry_complex_client_events (similar telemetry_complex_anon_events for anon users).

The message event we care about in particular is system:benchmark.

An example payload in the DB currently is this:

id            | 5
user_id       | 5
timestamp     | 2026-04-15 17:40:05
event_type_id | 3 <-- system:benchmark
value         | {"Sim": {"mean": 13.961731, "count": 1800, "total": 25131.1152, "spread": 1.27117431, "percentiles": {"0": 9.9630003, "1": 11.0419998, "2": 11.2279997, "5": 11.6110001, "10": 12.0139999, "20": 12.5120001, "35": 13.2309999, "50": 13.8470001, "65": 14.46, "80": 15.1219997, "90": 15.9379997, "95": 16.7099991, "98": 17.7919998, "99": 18.5750008, "100": 43.8349991}}, "cpu": "AMD Ryzen 9 7950X3D 16-Core Processor          ; 64632MB RAM, 73336MB pagefile", "gpu": "NVIDIA GeForce RTX 5070 Ti/PCIe/SSE2", "Draw": {"mean": 9.84391785, "count": 2388, "total": 23507.2754, "spread": 3.77889252, "percentiles": {"0": 1.70700002, "1": 1.74100006, "2": 1.75399995, "5": 1.85000002, "10": 2.12700009, "20": 3.73200011, "35": 8.6960001, "50": 10.9720001, "65": 12.3769999, "80": 13.1339998, "90": 14.1190004, "95": 16.1040001, "98": 18.6919994, "99": 20.1100006, "100": 158.367004}}, "Update": {"mean": 1.88539255, "count": 2388, "total": 4502.31738, "spread": 0.9580617, "percentiles": {"0": 0.214, "1": 0.227, "2": 0.23100001, "5": 0.245, "10": 0.29499999, "20": 0.588, "35": 1.079, "50": 2.43099999, "65": 2.54900002, "80": 2.67600012, "90": 2.83500004, "95": 2.96399999, "98": 3.16599989, "99": 3.61100006, "100": 8.98400021}}, "display": "3840x1600", "mapName": "Starwatcher 1.0", "gameName": "Beyond All Reason test-29918-105c4e6", "engineVersion": "2025.06.19", "benchmarkcommand": "luarules fightertest corak armpw 650 10 2040"}

We would care specifically about id, timestamp, event type, and value.

[bruno-dasilva]

what is the latency/freshness of data? ie. how often does data get exported out of teiserver? While data doesn't need to be realtime, it would be nice if it was fresher than once a week (ideally once a day) so we can check in after releases.

I see there's a workflow that runs biweekly (twice a week). That should be sufficient