Skip to content

chore: delete all fuctional of Cyndi#2322

Open
RostyslavKachan wants to merge 1 commit into
RedHatInsights:masterfrom
RostyslavKachan:cyndi_removal
Open

chore: delete all fuctional of Cyndi#2322
RostyslavKachan wants to merge 1 commit into
RedHatInsights:masterfrom
RostyslavKachan:cyndi_removal

Conversation

@RostyslavKachan
Copy link
Copy Markdown
Collaborator

@RostyslavKachan RostyslavKachan commented May 7, 2026

RHINENG-23296

Secure Coding Practices Checklist GitHub Link

Secure Coding Checklist

  • Input Validation
  • Output Encoding
  • Authentication and Password Management
  • Session Management
  • Access Control
  • Cryptographic Practices
  • Error Handling and Logging
  • Data Protection
  • Communication Security
  • System Configuration
  • Database Security
  • File Management
  • Memory Management
  • General Coding Practices

Summary by Sourcery

Remove legacy Cyndi inventory integration and rely solely on system_platform and related tables for system metadata, grouping, and filtering.

Enhancements:

  • Replace cyndi_join with a SystemGroupSet-focused join helper and update queries and filters to use SystemGroupSet and SystemPlatform instead of inventory.hosts and system_profile JSON.
  • Standardize operating system fields and sorting on the OperatingSystem table, removing custom JSON-based OS formatting and related feature flag handling.
  • Update system handlers, dashboards, reports, and status endpoints to use the new join and OS mechanisms and to read owner and tag data directly from SystemPlatform-related tables.
  • Simplify RHEL version, Ansible, MSSQL, SAP, and inventory group filters to no longer depend on inventory.hosts or feature flags.
  • Rename and clarify system counting helpers to reflect that they now only optionally include group joins rather than Cyndi-specific behavior.

Chores:

  • Remove Cyndi-specific schemas, initialization scripts, database objects, feature flags, metrics, admin endpoints, validation utilities, and test data, along with their associated tests and configuration references.

@sourcery-ai
Copy link
Copy Markdown

sourcery-ai Bot commented May 7, 2026

Reviewer's Guide

Removes all legacy Cyndi/inventory.hosts integration and feature-flagged paths in favor of the new SystemPlatform + SystemGroupSet/OperatingSystem model, simplifying joins, filters, metrics, admin endpoints, tests, and local DB setup while standardizing how OS, groups, tags, and ownership are handled.

Updated class diagram for join helpers and data models (class)

classDiagram
    class SystemPlatform {
        UUID inventory_id
        int rh_account_id
        int operating_system_id
        UUID tag_set_id
        datetime last_evaluation
        datetime last_upload
        datetime updated_timestamp
        datetime when_deleted
        bool opt_out
        bool advisor_evaluated
        bool stale
        jsonb sap_ids
        bool ansible
        bool mssql
        int cve_count_cache
        string host_type
        string owner_id
    }

    class SystemGroupSet {
        int id
        jsonb groups
    }

    class SystemTagSet {
        int id
        jsonb tags
    }

    class OperatingSystem {
        int id
        string name
        int major
        int minor
        OperatingSystem display_os_format()
        list~SQL~ sort_columns()
    }

    class ManagerBase {
        Query SystemGroupSet_join(Query query)
        Query filter_allowed_groups(Query query, jsonb groups_column)
        Query filter_kessel_workspace_opt_out(Query query, jsonb groups_column)
    }

    class FiltersModule {
        Query _filter_system_by_tags(Query query, dict args, dict kwargs)
        Query _filter_system_by_sap_sids(Query query, dict args, dict kwargs)
        Query _filter_system_by_rhel_version(Query query, dict args, dict kwargs)
        Query _filter_system_by_ansible(Query query, dict args, dict kwargs)
        Query _filter_system_by_mssql(Query query, dict args, dict kwargs)
        Query _filter_inventory_group_names(Query query, dict args, dict kwargs)
        Query _filter_inventory_group_ids(Query query, dict args, dict kwargs)
    }

    class Handlers {
        Query cve_handler_full_query()
        Query cve_handler_unpatched_query()
        Query system_handler_full_query()
        Query system_handler_unpatched_query()
        Query system_view_query()
        Query report_handler_query()
        Query dashboard_handler_query()
        Query dashbar_handler_query()
        Query vulnerabilities_handler_query()
        Query status_handler_query()
    }

    ManagerBase --> SystemGroupSet : join
    ManagerBase --> SystemPlatform : join

    SystemPlatform --> SystemGroupSet : has tag_set_id
    SystemPlatform --> SystemTagSet : has tag_set_id
    SystemPlatform --> OperatingSystem : has operating_system_id

    FiltersModule --> SystemGroupSet : uses groups
    FiltersModule --> SystemPlatform : uses sap_ids
    FiltersModule --> OperatingSystem : uses name, major, minor

    Handlers --> ManagerBase : calls SystemGroupSet_join
    Handlers --> FiltersModule : apply filters
    Handlers --> OperatingSystem : uses display_os_format, sort_columns
Loading

File-Level Changes

Change Details Files
Replace cyndi_join + InventoryHosts-based paths with SystemGroupSet_join and SystemPlatform/OperatingSystem everywhere.
  • Introduce SystemGroupSet_join helper that joins SystemPlatform to SystemGroupSet and applies group permission and workspace filters.
  • Replace all uses of cyndi_join in handlers (CVE, system, report, dashboard, dashbar, vulnerabilities, status, base utilities) with SystemGroupSet_join.
  • Standardize group-based filters to always use SystemGroupSet.groups instead of InventoryHosts.groups.
  • Standardize tag queries and filtering to use SystemTagSet.tags instead of InventoryHosts.tags.
manager/cve_handler.py
manager/system_handler.py
manager/report_handler.py
manager/dashboard_handler.py
manager/dashbar_handler.py
manager/status_handler.py
manager/vulnerabilities_handler.py
manager/base.py
manager/filters.py
Remove InventoryHosts model, all Cyndi/inventory.hosts schema dependencies, and related utilities/endpoints/metrics/tests.
  • Delete InventoryHosts peewee model and any direct references to inventory.hosts tables or schemas.
  • Remove wait_on_cyndi, its wiring in create_app, and any startup logic or env vars related to Cyndi.
  • Drop admin endpoints and tests for /systems/missing_in_inventory and associated DB metrics for Cyndi system counts and missing-in-Cyndi counts.
  • Update DB schema, local init script, truncate script, and test DB bootstrap to no longer create or depend on the inventory schema or Cyndi dev data.
  • Remove Cyndi-related feature flag CYNDI_REPLICATION_REPLACEMENT and environment-variable checks/scripts (e.g., CYNDI_MOCK).
common/peewee_model.py
manager/main.py
manager/admin_handler.py
manager.admin.spec.yaml
taskomatic/jobs/db_metrics.py
tests/manager_tests/test_admin_handler.py
database/schema/local_init_db.sh
database/schema/ve_db_postgresql.sql
tests/zz_database_tests/test_upgrade.py
tests/conftest.py
tests/data/truncate_dev_data.sql
tests/utils.py
scripts/check_vars.sh
common/feature_flags.py
database/schema/ve_db_cyndi_user.sql
database/schema/ve_db_dev_cyndi.sql
Switch OS, ownership, SAP/Ansible/MSSQL and RHEL version filtering from system_profile JSON to normalized columns and helpers.
  • Remove system_profile from selected columns where no longer needed and rely on OperatingSystem.display_os_format/sort_columns for os fields and sorting.
  • Change owner_id source from InventoryHosts.system_profile JSON to SystemPlatform.owner_id in system detail handlers and tests.
  • Update filters to derive SAP SIDs, Ansible, MSSQL, and RHEL version conditions from SystemPlatform.sap_ids, SystemPlatform.ansible, SystemPlatform.mssql, and OperatingSystem.major/minor instead of InventoryHosts.system_profile JSON.
  • Remove OS_INFO_QUERY, OS_INFO_SORT, RHSM_LOCK_QUERY helper SQL snippets that depended on system_profile JSON and use SystemPlatform/OperatingSystem columns instead.
manager/cve_handler.py
manager/system_handler.py
manager/filters.py
manager/base.py
manager/system_handler.py
tests/manager_tests/test_system_handler.py
Clean up Cyndi-related validation and mock behavior in utilities and platform mock.
  • Remove validate_system_inventory and a_validate_system_inventory helpers and their internal _is_system_msg_recent check that queried inventory.hosts.
  • Stop writing into inventory.hosts_v1_1 in platform_mock when synthesizing uploads; only send messages to the queue now.
  • Adjust tests and fixtures that previously relied on Cyndi data or the Cyndi wait flag to use the simplified create_app/get_pg_class signatures.
common/utils.py
platform_mock/platform_mock.py
tests/manager_tests/vuln_testcase.py
tests/manager_tests/test_admin_handler.py
tests/utils.py
tests/conftest.py

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 7, 2026

SC Environment Impact Assessment

Overall Impact:NONE

No SC Environment-specific impacts detected in this PR.

What was checked

This PR was automatically scanned for:

  • Database migrations
  • ClowdApp configuration changes
  • Kessel integration changes
  • AWS service integrations (S3, RDS, ElastiCache)
  • Kafka topic changes
  • Secrets management changes
  • External dependencies

Copy link
Copy Markdown

@sourcery-ai sourcery-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 1 issue, and left some high level feedback:

  • In _filter_system_by_rhel_version, the new loop builds expr incorrectly (expr = expr if expr is None else expr | expr), which effectively discards prior conditions and never incorporates the current one; this should OR the existing expression with the current condition (e.g., expr = cond if expr is None else expr | cond).
  • The new helper SystemGroupSet_join is a function but is named in PascalCase, which is inconsistent with the rest of the module’s snake_case functions; consider renaming it (e.g., system_group_set_join) to match the project’s naming conventions and avoid confusion with classes.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- In `_filter_system_by_rhel_version`, the new loop builds `expr` incorrectly (`expr = expr if expr is None else expr | expr`), which effectively discards prior conditions and never incorporates the current one; this should OR the existing expression with the current condition (e.g., `expr = cond if expr is None else expr | cond`).
- The new helper `SystemGroupSet_join` is a function but is named in PascalCase, which is inconsistent with the rest of the module’s snake_case functions; consider renaming it (e.g., `system_group_set_join`) to match the project’s naming conventions and avoid confusion with classes.

## Individual Comments

### Comment 1
<location path="manager/filters.py" line_range="463" />
<code_context>
+                    expr = OperatingSystem.major == int(version)
+            except (ValueError, TypeError):
+                continue
+            expr = expr if expr is None else expr | expr
+        if expr is not None:
+            query = query.where((OperatingSystem.name == "RHEL") & (expr))
</code_context>
<issue_to_address>
**issue (bug_risk):** The RHEL version filter is now logically incorrect; the OR expression uses `expr | expr` instead of accumulating conditions.

In `_filter_system_by_rhel_version`, the refactor broke the accumulation of per-version conditions. `expr = expr if expr is None else expr | expr` never incorporates the new condition and just ORs `expr` with itself, so `expr` either stays `None` (no filter) or is a no-op across iterations.

You likely want to accumulate `cond` instead, e.g.:

```python
for version in args["rhel_version"]:
    try:
        if "." in version:
            major, minor = version.split(".", 1)
            major, minor = int(major), int(minor)
            cond = (OperatingSystem.major == major) & (OperatingSystem.minor == minor)
        else:
            cond = OperatingSystem.major == int(version)
    except (ValueError, TypeError):
        continue
    expr = cond if expr is None else expr | cond

if expr is not None:
    query = query.where((OperatingSystem.name == "RHEL") & expr)
```

As written, multi-version filters (and possibly single-version filters, depending on how Peewee treats `None`) will not behave as intended.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment thread manager/filters.py
expr = OperatingSystem.major == int(version)
except (ValueError, TypeError):
continue
expr = expr if expr is None else expr | expr
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (bug_risk): The RHEL version filter is now logically incorrect; the OR expression uses expr | expr instead of accumulating conditions.

In _filter_system_by_rhel_version, the refactor broke the accumulation of per-version conditions. expr = expr if expr is None else expr | expr never incorporates the new condition and just ORs expr with itself, so expr either stays None (no filter) or is a no-op across iterations.

You likely want to accumulate cond instead, e.g.:

for version in args["rhel_version"]:
    try:
        if "." in version:
            major, minor = version.split(".", 1)
            major, minor = int(major), int(minor)
            cond = (OperatingSystem.major == major) & (OperatingSystem.minor == minor)
        else:
            cond = OperatingSystem.major == int(version)
    except (ValueError, TypeError):
        continue
    expr = cond if expr is None else expr | cond

if expr is not None:
    query = query.where((OperatingSystem.name == "RHEL") & expr)

As written, multi-version filters (and possibly single-version filters, depending on how Peewee treats None) will not behave as intended.

@RostyslavKachan
Copy link
Copy Markdown
Collaborator Author

good luck with review)

@RostyslavKachan
Copy link
Copy Markdown
Collaborator Author

/retest

1 similar comment
@RostyslavKachan
Copy link
Copy Markdown
Collaborator Author

/retest

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant