Skip to content

Add automated benchmarks in CI pipeline #78

@fako1024

Description

@fako1024

In order to track side-effects of changes on performance it would be enormously helpful to have automated benchmarks / comparison via benchstat as part of the CI pipeline (maybe not on each commit, but e.g. on filing a PR). Of course this is inherently difficult (because of reproducibility or rather lack thereof), but I've seen other projects do it.

DoD

  • Check for references and / or existing solutions for Github pipelines
  • Enhance CI with solution
  • Add ruleset / notifications (e.g. if geometric mean and / or other metrics indicate a significant change)

Metadata

Metadata

Assignees

Labels

enhancementNew feature or requestperformancePerformance / optimization related topics

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions