Skip to content

Feat/plugin health failure hook - #68

Open
maya-ots wants to merge 3 commits into
developfrom
feat/plugin-health-failure-hook
Open

maya-ots wants to merge 3 commits into
developfrom
feat/plugin-health-failure-hook

Conversation

@maya-ots

@maya-ots maya-ots commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

What

Added the on_health_failure plugin hook so plugins can react when a container becomes unhealthy after deployment.
Closes #41

How

  • Added a health monitor that periodically checks the health status of running instance containers.
  • The hook is triggered when a container transitions to unhealthy.
  • Avoided dispatching the same event repeatedly while the container remains unhealthy. If it recovers and becomes unhealthy again, the hook can be triggered again.
  • Added the health failure payload with the instance, challenge, container, status, and timestamp.
  • Started the monitor alongside the existing reaper in the instancer.
  • Added tests covering unhealthy transitions, repeated checks, and recovery.

Verification

  • go build ./...
  • go test ./...
  • golangci-lint run ./...
  • mctf generate --check on the sample event (if derived files could change)
  • Docs updated (docs/spec/, CHANGELOG) if behavior/CLI changed
  • Schema + make schemas synced (if on-disk shape changed)

Notes

The health monitor uses Docker inspection to check container health, so containers without a Docker health check don't trigger the hook.
also, all tests related to our health-failure hook are passing :

  • internal/instancer — PASS
  • internal/plugin — PASS
  • The other packages shown — PASS

but i got failures in internal/kubernetes. what appears to be failing :

  1. Windows path separators: Tests expect pwn/pwn-me, but Windows produces pwn\pwn-me. The same issue affects web and misc.
  2. Unexpected error behavior: TestE2E_ApplyUp_ErrorStopsExecution expected an error but didn't receive one. This may be a separate issue.

The Kubernetes test failures appear unrelated to this change and were left untouched and i got them too while working on a previous issue. i'll fix them soon as a new issue

@maya-ots maya-ots linked an issue Oct 30, 2026 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

plugin: on_health_failure hook (needs a health-monitoring loop)

1 participant