Centralize database configuration into a single reusable list

[ehelms]

Summary

Database configuration was duplicated across multiple consumers — each playbook and role that needed the list of databases would manually enumerate them with conditional logic for features like IOP. Adding a new database service meant editing every consumer.

This PR:

• Defines a canonical databases list in database.yml where each entry is tagged with a feature (e.g. foreman, katello, iop)
• Adds a computed all_databases variable that filters databases by enabled_features
• Adds to_postgresql_databases and to_postgresql_users custom Jinja2 filters that derive the postgresql role's input lists from all_databases
• Merges database_iop.yml into database.yml — one file for all database configuration
• Removes the pre_tasks merge block from deploy.yaml (no longer needed)
• Refactors check_database_connection to loop over checks_databases (a namespaced variable passed from the checks role) instead of a hardcoded inline list

Motivation

Every new command that needs database awareness (backup, restore, upgrade, health-check) was forced to repeat the same conditional enumeration pattern: foreman always, candlepin+pulp if katello, 5 IOP databases if iop. This was already duplicated in deploy.yaml, checks.yaml, and check_database_connection, and would only grow worse.

Design

database.yml
  ├── per-service variables (foreman_database_name, etc.)
  ├── databases[]          — canonical list, each entry tagged with feature
  ├── all_databases        — databases filtered by enabled_features
  ├── postgresql_databases — derived via to_postgresql_databases filter
  └── postgresql_users     — derived via to_postgresql_users filter

The databases list captures the full connection info for each database (name, host, port, user, password, ssl_mode, ssl_ca). The feature field maps to enabled_features so the list self-filters. Custom filters project this into the {name, owner} and {name, password} shapes that the postgresql role expects.

Changes

File	Change
src/vars/database.yml	Merged IOP vars, added databases/all_databases, replaced static postgresql_databases/postgresql_users with filter-derived computed vars
src/vars/database_iop.yml	Deleted (merged into database.yml)
src/filter_plugins/foremanctl.py	Added to_postgresql_databases and to_postgresql_users filters
src/playbooks/deploy/deploy.yaml	Removed IOP merge pre_tasks, pass checks_databases to checks role
src/playbooks/checks/checks.yaml	Added flavors vars_file, pass checks_databases to checks role
src/roles/check_database_connection/tasks/main.yaml	Replaced hardcoded 3-item loop with checks_databases
src/roles/check_database_connection/tasks/check.yaml	Aligned field names (dbname→database, sslmode→ssl_mode, ca_cert→ssl_ca)
src/roles/checks/defaults/main.yml	New — declares checks_databases: [] default
development/playbooks/deploy-dev/deploy-dev.yaml	Removed include_vars for deleted database_iop.yml

Test plan

• cd src; ansible-lint passes
• cd development; ansible-lint passes
• Deploy with --tuning development and verify postgresql databases and users are created correctly
• Deploy with IOP enabled and verify all 8 databases are created
• Run foremanctl checks with database_mode: external and verify all databases are checked
• Verify dev deploy (forge deploy-dev) still works with its test DB and SUPERUSER overrides

[ehelms]

@sjha4 Based on our discussions from the backup PR, I extracted the idea into a dedicated change that could serve as a basis for the backup.

@evgeni I'd appreciate your thoughts on this implementation change.

[arvind4501]

Not a blocker but more of a thought: We are gating on features, which means if the katello feature is there then only we will have pulp and candlepin databases, if its missing we don't have those, which is fine by today as we only have katello flavor, but in case of foreman-proxy-content flavor, we only will need pulp db not candlepin and then this becomes a issue,

[evgeni]

That's being filtered later in:

all_databases: >-
  {{ databases | selectattr('feature', 'in', enabled_features) | list }}

[arvind4501]

what i meant was enabled_features never know what pulp and candlepin is, only katello. thus no gating on pulp and candlepin

[evgeni]

Right, we have no pulp or candlepin feature (yet), which I think is also something we need/want to tackle for the tests too. Once we introduce those features, we can adjust here.

[sjha4]

Tested foremanctl check to make sure all DBs are made available to it and it works as advertised..Deploy has worked well with the change deploying all IoP DBs correctly.

Do we want a developer doc change to go with this for devs/agents to follow?

[sjha4]

Do we need to refactor this similarly?

[ehelms]

Do we want a developer doc change to go with this for devs/agents to follow?

I do not think so, I believe this is discoverable as a internal mechanism.

[ehelms]

Ignoring EL10 test results as there is an issue with the VM image affecting all tests right now.