Skip to content

Remove interval content query experiment - #1715

Open
larrymou9 wants to merge 2 commits into
pulp:mainfrom
larrymou9:lrios/remove-interval-content-experiment
Open

larrymou9 wants to merge 2 commits into
pulp:mainfrom
larrymou9:lrios/remove-interval-content-experiment

Conversation

@larrymou9

@larrymou9 larrymou9 commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Remove the temporary PULP-1996/PULP-2346 interval content-query experiment from the pulp-service image.
  • Add migration 0027 to remove the relative_path index owned by pulp-service migration 0024.
  • Remove the lazy experiment dispatch helper and its test/documentation.
  • Remove the old change notes for the temporary experiment/index and add a new cleanup note instead.

Migration 0024 remains in the repository because it is part of Django migration history; migration 0027 removes the deployed service-owned index.

Validation

  • Repository pre-commit checks passed.
  • Targeted Python compilation passed.
  • git diff --check passed.
  • Production GABI confirmed the current index annotation is:
    pulp-service:0024:core_ca_relpath_pattern_idx

Follow-up pulpcore implementation: pulp/pulpcore#8300

Assisted-by: pi coding agent

Summary by Sourcery

Remove the temporary interval content-query experiment and clean up its deployed database index.

Enhancements:

  • Remove the temporary interval content-query experiment and its supporting dispatch, patch, and tests from the pulp-service image.
  • Add a migration to safely remove the service-owned experiment index while preserving prior migration history.

Build:

  • Stop applying the temporary read-only content-query optimization patch to the pulp-service image.

Chores:

  • Replace obsolete experiment change notes with a cleanup note.

Drop the temporary PULP-1996/PULP-2346 experiment and its pulp-service-owned index. The released pulpcore implementation will be adopted separately after upstream release.

Assisted-by: pi coding agent

Co-authored-by: Elijah DeLee <kdelee@redhat.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@sourcery-ai

sourcery-ai Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Reviewer's Guide

Schema and migrations

Add migration 0027 to concurrently remove the relative-path index only when its annotation confirms it was created by pulp-service migration 0024.

  • pulp_service/pulp_service/app/migrations/0027_remove_experiment_relative_path_index.py L1-36

Implementation

Remove the interval content-query experiment helper and the imports and constants used only by that helper.

  • pulp_service/pulp_service/app/experiments.py L7-132 (4 small edits)

Configuration

Stop applying the temporary pulpcore query-optimization patch when building the pulp-service image, and remove the patch itself.

  • Dockerfile L189 (removed lines)
  • images/assets/patches/0080-Optimize-read-only-content-queries.patch (deleted)

Documentation

Remove the obsolete change notes for the temporary experiment and index.

  • CHANGES/1641.misc (deleted)
  • CHANGES/2346.misc (deleted)

Tests

Delete tests dedicated to the retired content-query experiment.

  • pulp_service/pulp_service/tests/unit/test_content_query_experiment.py (deleted)

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 1 security issue, and 3 other issues

Security issues:

  • Avoiding SQL string concatenation: untrusted input concatenated with raw SQL query can result in SQL Injection. In order to execute raw query safely, prepared statement should be used. SQLAlchemy provides TextualSQL to easily used prepared statement with named parameters. For complex SQL composition, use SQL Expression Language or Schema Definition Language. In most cases, SQLAlchemy ORM will be a better option. (link)
Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py" line_range="31" />
<code_context>
+    atomic = False  # DROP INDEX CONCURRENTLY cannot run in a transaction.
+
+    dependencies = [
+        ("service", "0024_contentartifact_relative_path_pattern_index"),
+    ]
+
</code_context>
<issue_to_address>
**Migrations cannot run**

When django loads this migration alongside the existing service migrations, django treats this migration and the branch through `0025_delete_domainorg` and `0026_andgatecontentguard` as separate leaves, so `migrate` fails with a conflicting-migrations error and `makemigrations` cannot generate later migrations.

Make this migration depend on the existing branch, or otherwise merge the migration graph into a single leaf.

Also at `pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py:32`.
</issue_to_address>

### Comment 2
<location path="pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py" line_range="22" />
<code_context>
+        )
+        row = cursor.fetchone()
+        if row and row[0] == _INDEX_MARKER:
+            cursor.execute(_DROP_INDEX_SQL)
+
+
</code_context>
<issue_to_address>
**Directory listings become slow**

When a directory listing runs after this migration, especially on a large repository, `remove_experiment_index` drops the `text_pattern_ops` index while directory-listing queries still filter artifacts with `relative_path__startswith`; PostgreSQL scans the artifact tables and directory listings become slow.

Keep or add an index that supports the prefix queries for as long as directory listings use them.
</issue_to_address>

### Comment 3
<location path="pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py" line_range="8" />
<code_context>
+_DROP_INDEX_SQL = f'DROP INDEX CONCURRENTLY IF EXISTS "{_INDEX_NAME}"'
+
+
+def remove_experiment_index(apps, schema_editor):  # noqa: ARG001
+    """Remove only the index owned by the temporary pulp-service migration."""
+    with schema_editor.connection.cursor() as cursor:
</code_context>
<issue_to_address>
**Release notes omit the cleanup**

When towncrier builds release notes for this change, `CHANGES/` has no fragment for this index-removal migration, so Towncrier omits the cleanup from the release notes.

Add a Towncrier fragment documenting the index cleanup.

Also at `pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py:35`.
</issue_to_address>

### Comment 4
<location path="pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py" line_range="22" />
<code_context>
            cursor.execute(_DROP_INDEX_SQL)
</code_context>
<issue_to_address>
**security (python.sqlalchemy.security.sqlalchemy-execute-raw-query):** Avoiding SQL string concatenation: untrusted input concatenated with raw SQL query can result in SQL Injection. In order to execute raw query safely, prepared statement should be used. SQLAlchemy provides TextualSQL to easily used prepared statement with named parameters. For complex SQL composition, use SQL Expression Language or Schema Definition Language. In most cases, SQLAlchemy ORM will be a better option.

*Source: opengrep*
</issue_to_address>

Sourcery assessment

Needs a human reviewer. 4 findings to address first, and the migration concurrently drops the marked database index, and reverting the code will not recreate it. If that index is still needed, query performance may degrade until it is explicitly rebuilt, but the index can be restored.

Blocking findings: pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py:31, pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py:22, pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py:8, pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py:22


Sourcery is free for open source - if you like our reviews please consider sharing them ✨

)
row = cursor.fetchone()
if row and row[0] == _INDEX_MARKER:
cursor.execute(_DROP_INDEX_SQL)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium · Directory listings become slow

When a directory listing runs after this migration, especially on a large repository, remove_experiment_index drops the text_pattern_ops index while directory-listing queries still filter artifacts with relative_path__startswith; PostgreSQL scans the artifact tables and directory listings become slow.

Keep or add an index that supports the prefix queries for as long as directory listings use them.

Prompt for AI agents
In `pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py` at line 22:

**Directory listings become slow**

When a directory listing runs after this migration, especially on a large repository, `remove_experiment_index` drops the `text_pattern_ops` index while directory-listing queries still filter artifacts with `relative_path__startswith`; PostgreSQL scans the artifact tables and directory listings become slow.

Keep or add an index that supports the prefix queries for as long as directory listings use them.

_DROP_INDEX_SQL = f'DROP INDEX CONCURRENTLY IF EXISTS "{_INDEX_NAME}"'


def remove_experiment_index(apps, schema_editor): # noqa: ARG001

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium · Release notes omit the cleanup

When towncrier builds release notes for this change, CHANGES/ has no fragment for this index-removal migration, so Towncrier omits the cleanup from the release notes.

Add a Towncrier fragment documenting the index cleanup.

Also at pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py:35.

Prompt for AI agents
In `pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py` at line 8:

**Release notes omit the cleanup**

When towncrier builds release notes for this change, `CHANGES/` has no fragment for this index-removal migration, so Towncrier omits the cleanup from the release notes.

Add a Towncrier fragment documenting the index cleanup.

Also at `pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py:35`.

From CLAUDE.md L76:

Uses **towncrier**. For any non-trivial change, create a file in `CHANGES/` named `{issue_number}.{category}` where category is one of: `feature`, `bugfix`, `doc`, `removal`, `deprecation`, `misc`.

The service app already has 0025_delete_domainorg and 0026_andgatecontentguard,
so a cleanup migration numbered 0025 that depends on 0024 forks the migration
graph and makes migrate fail with conflicting migrations. Renumber it to 0027
and make it depend on the latest migration, 0026_andgatecontentguard.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New security issues found

)
row = cursor.fetchone()
if row and row[0] == _INDEX_MARKER:
cursor.execute(_DROP_INDEX_SQL)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

security (python.sqlalchemy.security.sqlalchemy-execute-raw-query): Avoiding SQL string concatenation: untrusted input concatenated with raw SQL query can result in SQL Injection. In order to execute raw query safely, prepared statement should be used. SQLAlchemy provides TextualSQL to easily used prepared statement with named parameters. For complex SQL composition, use SQL Expression Language or Schema Definition Language. In most cases, SQLAlchemy ORM will be a better option.

Source: opengrep

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant