Skip to content

Fix IOP tests to properly validate service endpoints#544

Open
ehelms wants to merge 1 commit into
theforeman:masterfrom
ehelms:improve-iop-tests
Open

Fix IOP tests to properly validate service endpoints#544
ehelms wants to merge 1 commit into
theforeman:masterfrom
ehelms:improve-iop-tests

Conversation

@ehelms

@ehelms ehelms commented Jun 8, 2026

Copy link
Copy Markdown
Member

Why are you introducing these changes? (Problem description, related links)

The IOP HTTP endpoint tests had several issues that prevented them from actually validating services:

  • Tests curled localhost for ports not published to the host (only the gateway publishes 24443)
  • || echo fallbacks made rc always 0, so tests passed whether services were up or down
  • if result.succeeded guards silently skipped assertions when podman run failed due to missing --network
  • Endpoint paths didn't match the actual health/liveness endpoints exposed by the services

What are the changes introduced in this pull request?

  • Add --network=iop-core-network to all podman run curl commands so container DNS resolves
  • Remove all || echo fallbacks and if result.succeeded guards so failures surface
  • Use correct service endpoints from puppet-iop acceptance specs: /api/insights/v1/status/live/ for advisor, /health for remediations

How to test this pull request

Steps to reproduce:

  • Deploy IOP with ./foremanctl deploy
  • Run IOP tests: ./forge test --pytest_args="tests/iop/"
  • Verify all HTTP endpoint tests actually reach services and assert real status codes

Checklist

  • Tests added/updated (if applicable)
  • Documentation updated (if applicable)

Comment thread tests/iop/conftest.py Outdated
Comment on lines +31 to +32


Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wat

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

huh

Comment thread tests/iop/test_advisor.py Outdated
Comment thread tests/iop/test_integration.py Outdated
Comment thread tests/iop/test_inventory.py Outdated
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@ehelms ehelms force-pushed the improve-iop-tests branch from 34c17b2 to e065473 Compare June 9, 2026 15:47
@ehelms

ehelms commented Jun 10, 2026

Copy link
Copy Markdown
Member Author

Comments addressed

@evgeni evgeni left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One last missing piece, otherwise ACK

result = server.run("curl -f http://localhost:24443/api/ingress 2>/dev/null || echo 'Gateway API ingress not yet responding'")
assert result.rc == 0
result = server.run("curl -s -o /dev/null http://localhost:24443/api/ingress")
assert result.succeeded

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These two don't use --fail, which I assume is intentional, but there is no explanation why and it will be confusing the next time we touch this, can you add a comment for that please?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants