Add molecule tests to certificates role by ShimShtein · Pull Request #548 · theforeman/foremanctl

ShimShtein · 2026-06-09T10:20:37Z

This is the PR that implements https://community.theforeman.org/t/molecule-testing-in-foremanctl/46760.

It is sill draft and more of a POC than actual viable PR at this point.

ehelms · 2026-06-09T11:11:57Z

It would be good to see this PoC hooked into the Github Actions.

ehelms · 2026-06-09T11:12:43Z

@@ -0,0 +1,111 @@
+---


Can tests with molecule be written in pytest or do they have to be written in Ansible?

Currently molecule supports ansible and testinfra verifiers.
If that's not enough, we can use ansible to run any arbitrary script on the machine and test its result.

testinfra verifiers are pytest based right?

yes I think so.

evgeni · 2026-06-09T12:28:49Z

+        )
+
+
+def test_molecule_role(role: str, scenario: str, role_dir: Path) -> None:


how does this compare to pytest-ansible that already has molecule discovery and testing?

This is the integration with pytest-ansible. As you can see, this is a thin wrapper, and pytest-ansible does the heavy lifting of running the full molecule "lifecycle" as part of the test.

No it is not. It uses subprocess.run to call python -m molecule.

I see. I had some weird double requirement error and I let AI to fix it, somehow I have missed the fact that it just calls it in a subprocess. At least theoretically the plugin should do the discovery automagically. I suppose it would be harder to control it, if we want to keep standalone pytests side by side with molecule.

Although in a broader picture, we can do the opposite: let molecule to be the main driver instead of forge. This is how molecule is actually intended to be used. It has the options to spin up a dev machine/container, then you can work on it with molecule converge and in the end it will destroy the machine on its own.

I mean instead of running ./forge test .... we can call molecule test that will do the same thing.

Using pytest-ansible would allow the test driver to remain pytest right? All ./forge test is doing is preventing manual work to generate the ssh-config from Vagrant to then run pytest with some basic initial variables. You can still run pytest directly if needed. In fact, if switching to pytest or make would be cleaner I have no problem with that. We'd still need something to do that generating of the ssh-config.

Molecule has all the framework laid out for setup. It has the specific subcommand to create the environment. Under the hood it knows how to work with vagrant and containers (there are also other drivers, but I didn't explore too much in that direction).
The TL;DR version of molecule goes like this:
under the hood molecule test executes the following stages:

dependency → cleanup → destroy → syntax → create → prepare → converge → idempotence → side_effect → verify → cleanup → destroy

each one of he stages has an out of the box playbook, or you can override it with a custom one. For example for create they already have support for containers and vagrant (there are more, but I didn't explore in this direction).
I would say that we can replace the whole forge concept since it supports also a role dev cycle as create, converge, validate, iterate, destroy.
Anything that forge gives us beyond that?

evgeni · 2026-06-09T12:30:05Z

+  - name: quadlet
+    image: registry.access.redhat.com/ubi9/ubi
+    volumes:
+      - "${MOLECULE_PROJECT_DIRECTORY}/../../../:/vagrant:Z"


why is this mounting things at /vagrant? this should run in a container, right?

Because I was lazy when switched between drivers. I need the sources mounted into the container to do things like copying the custom certificates: https://github.com/theforeman/foremanctl/pull/548/changes#diff-8e0370b95ab38b1aa4d27ac77915e805f1832f8adba342ef74dda35f550660a1R6.
Since I was lazy to change the folder names, and if I use vagrant as the driver, I have synced the folder structure between the drivers.

ShimShtein · 2026-06-09T12:50:11Z

It would be good to see this PoC hooked into the Github Actions.

@ehelms right now it is a matter of adding the --molecule flag to the pytest command:

foremanctl/.github/workflows/test.yml

Line 168 in 377474a

    
                     ./forge test --pytest-args="--certificate-source=${{ matrix.certificate_source }} --database-mode=${{ matrix.database }}"

I wanted to showcase the method before we hook it up into the CI

ehelms · 2026-06-10T10:57:51Z

INFO     Found config file /home/ehelms/workspace/upstream/installer/foremanctl-2/.config/molecule/config.yml
ERROR    Failed to find driver podman. Please ensure that the driver is correctly installed.

After I installed the molecule-podman package I get:

TASK [Set async_dir for HOME env] **********************************************
[ERROR]: Task failed: Conditional result (True) was derived from value of type 'str' at "<environment variable 'HOME'>". Conditionals must have a boolean result.

Task failed.
Origin: /home/ehelms/workspace/upstream/installer/foremanctl-2/.venv/lib64/python3.14/site-packages/molecule_podman/playbooks/destroy.yml:11:7

 9     podman_exec: "{{ lookup('env','MOLECULE_PODMAN_EXECUTABLE')|default('podman',true) }}"
10   tasks:
11     - name: Set async_dir for HOME env
         ^ column 7

<<< caused by >>>

Conditional result (True) was derived from value of type 'str' at "<environment variable 'HOME'>". Conditionals must have a boolean result.
Origin: /home/ehelms/workspace/upstream/installer/foremanctl-2/.venv/lib64/python3.14/site-packages/molecule_podman/playbooks/destroy.yml:14:13

Add molecule tests to certificates role

ba77afc

pr-processor Bot added the Not yet reviewed label Jun 9, 2026

ehelms reviewed Jun 9, 2026

View reviewed changes

pr-processor Bot removed the Not yet reviewed label Jun 9, 2026

evgeni reviewed Jun 9, 2026

View reviewed changes

		)


		def test_molecule_role(role: str, scenario: str, role_dir: Path) -> None:

Conversation

ShimShtein commented Jun 9, 2026

Uh oh!

ehelms commented Jun 9, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ShimShtein Jun 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ShimShtein Jun 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ShimShtein commented Jun 9, 2026

Uh oh!

ehelms commented Jun 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ShimShtein Jun 9, 2026 •

edited

Loading

ShimShtein Jun 10, 2026 •

edited

Loading