PMM-15197 Add on-demand min-* agent templates for RC/release testing. - #394
PMM-15197 Add on-demand min-* agent templates for RC/release testing.#394talhabinrizwan wants to merge 1 commit into
Conversation
nogueiraanderson
left a comment
There was a problem hiding this comment.
Two changes, one optional:
- Give the on-demand templates their own description (line 386):
onDemand ? OSType + '-ondemand' : OSType, // String descriptionThe plugin counts instances per description and lumps spot and on-demand together, so running spot instances eat the on-demand cap of 8. During an RC window with 8+ spot instances of an OS, the on-demand template provisions nothing. Distinct descriptions separate the pools. Best merged before the consumer PR (jenkins-pipelines PR 4325), while no instances exist under the old description.
- Make the on-demand agents label-exclusive (line 385):
onDemand ? Node.Mode.EXCLUSIVE : Node.Mode.NORMAL, // Node.Mode modeOtherwise any unlabeled job can schedule onto the paid on-demand pool.
- Optional: build the spot and on-demand registrations from one shared OS list, so the next OS added cannot silently miss its on-demand variant.
Everything else checks out: null spotConfig correctly selects on-demand provisioning, the constructor arguments line up, and the labels match the consumer PR.
|
Validated both variants in isolation on the pmm master (Script Console harness that constructs the templates against the live ec2-plugin build without registering anything):
The diff as-is constructs cleanly on the live plugin version (constructor arity, the new 4th parameter, all map lookups), and the two requested one-liners flip the last two rows to pass. |
Adds on-demand variants of all 18
min-*package-testing agent templates (9 x64 + 9 arm64) to the PMM Jenkins master's EC2 cloud config. Today everymin-*label is spot-only, which causes RC/release package testing to get interrupted by spot evictions.getTemplate()gains anonDemandflag. Whentrue: noSpotConfiguration(→ on-demand pricing), label gets a-ondemandsuffix (e.g.min-ol-8-x64-ondemand), and the instance cap is set to8rather than the spot templates'15.m6a.large/m7i.largefallback pools added under PMM-15066 exist specifically to route around spot capacity/eviction issues, which don't apply once we're not on spot.On-demand instances cost more per hour than spot, and RC/release testing is infrequent and bounded (Peter's estimate: ~72 concurrent at peak for a single package-test matrix run). Capping at 8 per OS keeps the ceiling reasonable while still comfortably covering real usage; it can be raised later if RC testing ever needs more headroom.
One thing I deliberately left out of is migrating the existing UI-configured -ondemand labels (agent-amd64-ondemand, cli-ondemand, agent-arm64-ondemand) into code. Those still live purely in the Jenkins UI, not in IaC, and folding them into percona-cd-platform properly felt like a bigger, riskier lift than I wanted to bundle into this change.
Also, I'll set up docker-ondemand directly from the Jenkins UI myself, matching how the others were originally configured, no code change needed there on your end.
Test plan
jenkins iac deployapplies cleanly against the PMM master-ondemandlabels appear as available node templates in Jenkinsjenkins-pipelinescompanion change lands, trigger a package-testing job withUSE_ONDEMAND=trueand confirm it schedules onto the new on-demand pool instead of spot