Skip to content

Remove interval content query experiment - #1701

Closed
kdelee wants to merge 1 commit into
pulp:mainfrom
kdelee:kdelee/remove-interval-content-experiment
Closed

kdelee wants to merge 1 commit into
pulp:mainfrom
kdelee:kdelee/remove-interval-content-experiment

Conversation

@kdelee

@kdelee kdelee commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Remove the temporary PULP-1996/PULP-2346 interval content-query experiment from the pulp-service image.
  • Add migration 0025 to remove the relative_path index owned by pulp-service migration 0024.
  • Remove the lazy experiment dispatch helper and its test/documentation.
  • Remove the old change notes for the temporary experiment/index and add a new cleanup note instead.

Migration 0024 remains in the repository because it is part of Django migration history; migration 0025 removes the deployed service-owned index. The optimized content-serving patch remains in the image until the released pulpcore implementation is adopted.

Validation

  • Repository pre-commit checks passed.
  • Targeted Python compilation passed.
  • git diff --check passed.
  • Production GABI confirmed the current index annotation is:
    pulp-service:0024:core_ca_relpath_pattern_idx

Follow-up pulpcore implementation: pulp/pulpcore#8300

Assisted-by: pi coding agent

Summary by Sourcery

Retire the temporary interval content-query experiment and clean up its service-owned database index and supporting artifacts.

Enhancements:

  • Remove the temporary interval content-query experiment and its associated lazy-dispatch helper from the service image and application code.

Build:

  • Stop applying the temporary optimized read-only content-query patch to the pulp-service image.

Documentation:

  • Replace the temporary experiment and index change notes with a cleanup note.

Tests:

  • Remove tests for the retired content-query experiment.

Chores:

  • Add migration 0025 to safely remove the temporary relative-path index created by migration 0024 while preserving migration history.

@sourcery-ai

sourcery-ai Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Reviewer's Guide

Schema and migrations

Add migration 0025 to drop the relative-path index only when its database annotation identifies it as owned by migration 0024.

  • pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py L1-36

Implementation

Remove the PULP-2346 experiment imports, constants, query-dispatch helper, and single-row helper that supported the temporary experiment.

  • pulp_service/pulp_service/app/experiments.py L7-132 (4 small edits)

Configuration

Stop copying and applying the temporary read-only content-query patch in the image, and remove the patch file.

  • Dockerfile L188 (removed lines)
  • images/assets/patches/0080-Optimize-read-only-content-queries.patch (deleted)

Documentation

Remove the obsolete change notes for the temporary experiment and index.

  • CHANGES/1641.misc (deleted)
  • CHANGES/2346.misc (deleted)

Tests

Remove the tests for the deleted content-query experiment helper.

  • pulp_service/pulp_service/tests/unit/test_content_query_experiment.py (deleted)

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@kdelee
kdelee force-pushed the kdelee/remove-interval-content-experiment branch from 7fc0d39 to 167c9d9 Compare November 12, 2026 20:22

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New security issues found

)
row = cursor.fetchone()
if row and row[0] == _INDEX_MARKER:
cursor.execute(_DROP_INDEX_SQL)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

security (python.sqlalchemy.security.sqlalchemy-execute-raw-query): Avoiding SQL string concatenation: untrusted input concatenated with raw SQL query can result in SQL Injection. In order to execute raw query safely, prepared statement should be used. SQLAlchemy provides TextualSQL to easily used prepared statement with named parameters. For complex SQL composition, use SQL Expression Language or Schema Definition Language. In most cases, SQLAlchemy ORM will be a better option.

Source: opengrep

@kdelee
kdelee force-pushed the kdelee/remove-interval-content-experiment branch from 167c9d9 to de7ff8a Compare November 12, 2026 20:23

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New security issues found

)
row = cursor.fetchone()
if row and row[0] == _INDEX_MARKER:
cursor.execute(_DROP_INDEX_SQL)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

security (python.sqlalchemy.security.sqlalchemy-execute-raw-query): Avoiding SQL string concatenation: untrusted input concatenated with raw SQL query can result in SQL Injection. In order to execute raw query safely, prepared statement should be used. SQLAlchemy provides TextualSQL to easily used prepared statement with named parameters. For complex SQL composition, use SQL Expression Language or Schema Definition Language. In most cases, SQLAlchemy ORM will be a better option.

Source: opengrep

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 1 security issue, and 4 other issues

Security issues:

  • Avoiding SQL string concatenation: untrusted input concatenated with raw SQL query can result in SQL Injection. In order to execute raw query safely, prepared statement should be used. SQLAlchemy provides TextualSQL to easily used prepared statement with named parameters. For complex SQL composition, use SQL Expression Language or Schema Definition Language. In most cases, SQLAlchemy ORM will be a better option. (link)
Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py" line_range="11-23" />
<code_context>
+def remove_experiment_index(apps, schema_editor):  # noqa: ARG001
+    """Remove only the index owned by the temporary pulp-service migration."""
+    with schema_editor.connection.cursor() as cursor:
+        cursor.execute(
+            """
+            SELECT obj_description(indexrel.oid, 'pg_class')
+            FROM pg_class AS indexrel
+            JOIN pg_namespace AS namespace ON namespace.oid = indexrel.relnamespace
+            WHERE indexrel.relname = %s AND namespace.nspname = current_schema()
+            """,
+            [_INDEX_NAME],
+        )
+        row = cursor.fetchone()
+        if row and row[0] == _INDEX_MARKER:
+            cursor.execute(_DROP_INDEX_SQL)
+
+
+class Migration(migrations.Migration):
</code_context>
<issue_to_address>
**Replacement index gets dropped**

When the pulpcore replacement migration operates on the same index concurrently with this migration, `remove_experiment_index` checks the ownership comment and later drops by name, with no lock spanning those operations. A concurrent pulpcore migration that replaces or adopts the same-named index between them can have its replacement dropped, leaving the replacement implementation without its index.

Serialize the migrations or make the drop conditional on ownership at the time the index is dropped.
</issue_to_address>

### Comment 2
<location path="pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py" line_range="22" />
<code_context>
+        )
+        row = cursor.fetchone()
+        if row and row[0] == _INDEX_MARKER:
+            cursor.execute(_DROP_INDEX_SQL)
+
+
</code_context>
<issue_to_address>
**Content lookups lose index performance**

When the service migration runs before pulpcore’s replacement query and index are active, `remove_experiment_index` drops the index while the retained content-query patch still filters `relative_path` by prefix, so PostgreSQL scans `core_contentartifact` and content requests become slower on large repositories.

Keep the index until pulpcore’s replacement query and index are active.

Also at `pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py:23`.
</issue_to_address>

### Comment 3
<location path="pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py" line_range="21" />
<code_context>
+            [_INDEX_NAME],
+        )
+        row = cursor.fetchone()
+        if row and row[0] == _INDEX_MARKER:
+            cursor.execute(_DROP_INDEX_SQL)
+
</code_context>
<issue_to_address>
**Unmarked index survives cleanup**

When migration 0024 encounters or leaves a valid index without its ownership comment, `remove_experiment_index` drops the index only when its comment matches `_INDEX_MARKER`, while migration 0024 accepts a valid existing index without adding the comment, so migration 0025 leaves the service-owned index in the database.

Ensure retries mark an existing service-owned index before migration 0025 checks its comment.

Also at `pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py:22-23`.
</issue_to_address>

### Comment 4
<location path="pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py" line_range="35" />
<code_context>
+    ]
+
+    operations = [
+        migrations.RunPython(remove_experiment_index, migrations.RunPython.noop),
+    ]
</code_context>
<issue_to_address>
**Rollback leaves index missing**

When an operator reverses migration 0025 without also reversing migration 0024, `RunPython.noop` does not restore the index dropped by `remove_experiment_index`, so the prior service version resumes its optimized content queries without the index they expect.

Provide a reverse operation that restores the index, or otherwise ensure it is present when migration 0025 is reversed.
</issue_to_address>

### Comment 5
<location path="pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py" line_range="22" />
<code_context>
            cursor.execute(_DROP_INDEX_SQL)
</code_context>
<issue_to_address>
**security (python.sqlalchemy.security.sqlalchemy-execute-raw-query):** Avoiding SQL string concatenation: untrusted input concatenated with raw SQL query can result in SQL Injection. In order to execute raw query safely, prepared statement should be used. SQLAlchemy provides TextualSQL to easily used prepared statement with named parameters. For complex SQL composition, use SQL Expression Language or Schema Definition Language. In most cases, SQLAlchemy ORM will be a better option.

*Source: opengrep*
</issue_to_address>

Sourcery assessment

Needs a human reviewer. 5 findings to address first, and the migration drops the pulp-service-owned index, which may slow the content lookups if the pulpcore replacement is not present. Reverting the code does not recreate the index because the migration’s reverse operation is a no-op, but the index can be rebuilt.

Blocking findings: pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py:23, pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py:22, pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py:21, pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py:35, pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py:22


Sourcery is free for open source - if you like our reviews please consider sharing them ✨

Comment on lines +11 to +23
cursor.execute(
"""
SELECT obj_description(indexrel.oid, 'pg_class')
FROM pg_class AS indexrel
JOIN pg_namespace AS namespace ON namespace.oid = indexrel.relnamespace
WHERE indexrel.relname = %s AND namespace.nspname = current_schema()
""",
[_INDEX_NAME],
)
row = cursor.fetchone()
if row and row[0] == _INDEX_MARKER:
cursor.execute(_DROP_INDEX_SQL)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium · Replacement index gets dropped

When the pulpcore replacement migration operates on the same index concurrently with this migration, remove_experiment_index checks the ownership comment and later drops by name, with no lock spanning those operations. A concurrent pulpcore migration that replaces or adopts the same-named index between them can have its replacement dropped, leaving the replacement implementation without its index.

Serialize the migrations or make the drop conditional on ownership at the time the index is dropped.

Prompt for AI agents
In `pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py` at lines 11-23:

**Replacement index gets dropped**

When the pulpcore replacement migration operates on the same index concurrently with this migration, `remove_experiment_index` checks the ownership comment and later drops by name, with no lock spanning those operations. A concurrent pulpcore migration that replaces or adopts the same-named index between them can have its replacement dropped, leaving the replacement implementation without its index.

Serialize the migrations or make the drop conditional on ownership at the time the index is dropped.

)
row = cursor.fetchone()
if row and row[0] == _INDEX_MARKER:
cursor.execute(_DROP_INDEX_SQL)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium · Content lookups lose index performance

When the service migration runs before pulpcore’s replacement query and index are active, remove_experiment_index drops the index while the retained content-query patch still filters relative_path by prefix, so PostgreSQL scans core_contentartifact and content requests become slower on large repositories.

Keep the index until pulpcore’s replacement query and index are active.

Also at pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py:23.

Prompt for AI agents
In `pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py` at line 22:

**Content lookups lose index performance**

When the service migration runs before pulpcore’s replacement query and index are active, `remove_experiment_index` drops the index while the retained content-query patch still filters `relative_path` by prefix, so PostgreSQL scans `core_contentartifact` and content requests become slower on large repositories.

Keep the index until pulpcore’s replacement query and index are active.

Also at `pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py:23`.

[_INDEX_NAME],
)
row = cursor.fetchone()
if row and row[0] == _INDEX_MARKER:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium · Unmarked index survives cleanup

When migration 0024 encounters or leaves a valid index without its ownership comment, remove_experiment_index drops the index only when its comment matches _INDEX_MARKER, while migration 0024 accepts a valid existing index without adding the comment, so migration 0025 leaves the service-owned index in the database.

Ensure retries mark an existing service-owned index before migration 0025 checks its comment.

Also at pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py:22-23.

Prompt for AI agents
In `pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py` at line 21:

**Unmarked index survives cleanup**

When migration 0024 encounters or leaves a valid index without its ownership comment, `remove_experiment_index` drops the index only when its comment matches `_INDEX_MARKER`, while migration 0024 accepts a valid existing index without adding the comment, so migration 0025 leaves the service-owned index in the database.

Ensure retries mark an existing service-owned index before migration 0025 checks its comment.

Also at `pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py:22-23`.

]

operations = [
migrations.RunPython(remove_experiment_index, migrations.RunPython.noop),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium · Rollback leaves index missing

When an operator reverses migration 0025 without also reversing migration 0024, RunPython.noop does not restore the index dropped by remove_experiment_index, so the prior service version resumes its optimized content queries without the index they expect.

Provide a reverse operation that restores the index, or otherwise ensure it is present when migration 0025 is reversed.

Prompt for AI agents
In `pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py` at line 35:

**Rollback leaves index missing**

When an operator reverses migration 0025 without also reversing migration 0024, `RunPython.noop` does not restore the index dropped by `remove_experiment_index`, so the prior service version resumes its optimized content queries without the index they expect.

Provide a reverse operation that restores the index, or otherwise ensure it is present when migration 0025 is reversed.

)
row = cursor.fetchone()
if row and row[0] == _INDEX_MARKER:
cursor.execute(_DROP_INDEX_SQL)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

security (python.sqlalchemy.security.sqlalchemy-execute-raw-query): Avoiding SQL string concatenation: untrusted input concatenated with raw SQL query can result in SQL Injection. In order to execute raw query safely, prepared statement should be used. SQLAlchemy provides TextualSQL to easily used prepared statement with named parameters. For complex SQL composition, use SQL Expression Language or Schema Definition Language. In most cases, SQLAlchemy ORM will be a better option.

Source: opengrep

@kdelee
kdelee force-pushed the kdelee/remove-interval-content-experiment branch 3 times, most recently from f6a66c7 to 67ed69b Compare November 12, 2026 22:12
Drop the temporary PULP-1996/PULP-2346 experiment and its pulp-service-owned index. The released pulpcore implementation will be adopted separately after upstream release.

Assisted-by: pi coding agent
@kdelee
kdelee force-pushed the kdelee/remove-interval-content-experiment branch from 67ed69b to 924e010 Compare November 12, 2026 22:14
@kdelee

kdelee commented Nov 12, 2026

Copy link
Copy Markdown
Contributor Author

@sourcery-ai review

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 1 security issue, and 5 other issues

Security issues:

  • Avoiding SQL string concatenation: untrusted input concatenated with raw SQL query can result in SQL Injection. In order to execute raw query safely, prepared statement should be used. SQLAlchemy provides TextualSQL to easily used prepared statement with named parameters. For complex SQL composition, use SQL Expression Language or Schema Definition Language. In most cases, SQLAlchemy ORM will be a better option. (link)
Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py" line_range="31" />
<code_context>
+    atomic = False  # DROP INDEX CONCURRENTLY cannot run in a transaction.
+
+    dependencies = [
+        ("service", "0024_contentartifact_relative_path_pattern_index"),
+    ]
+
</code_context>
<issue_to_address>
**Migration commands report conflicting branches**

When django loads the service migration graph for migration generation or deployment, this migration and `0025_delete_domainorg` remain separate leaves because `0026_andgatecontentguard` follows only the latter, so Django reports conflicting migrations and blocks migration generation or deployment.

Join the migration branches by making the subsequent migration depend on this migration as well as `0025_delete_domainorg`.
</issue_to_address>

### Comment 2
<location path="pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py" line_range="21" />
<code_context>
+            [_INDEX_NAME],
+        )
+        row = cursor.fetchone()
+        if row and row[0] == _INDEX_MARKER:
+            cursor.execute(_DROP_INDEX_SQL)
+
</code_context>
<issue_to_address>
**Cleanup leaves the index behind**

When migration 0024 creates the index but does not successfully add its ownership comment, `remove_experiment_index` skips the index when its ownership marker is absent, so migration 0025 completes while the service-owned index remains in the database.

Handle the existing index by name even when its ownership comment is missing, so the cleanup removes it.
</issue_to_address>

### Comment 3
<location path="Dockerfile" line_range="188-189" />
<code_context>
 COPY images/assets/patches/0079-Log-Maven-path-index-fallback-reasons.patch /tmp/
 RUN patch -p1 -d /usr/local/lib/pulp/lib/python${PYTHON_VERSION}/site-packages < /tmp/0079-Log-Maven-path-index-fallback-reasons.patch

-COPY images/assets/patches/0080-Optimize-read-only-content-queries.patch /tmp/
-RUN patch -p1 -d /usr/local/lib/pulp/lib/python${PYTHON_VERSION}/site-packages < /tmp/0080-Optimize-read-only-content-queries.patch
-
 COPY images/assets/patches/0081-Fix-task-cancellation-during-graceful-shutdown.patch /tmp/
</code_context>
<issue_to_address>
**Content requests lose query optimizations**

When the image is built and migration 0025 is applied before the replacement is available, the image stops applying patch 0080 and migration 0025 drops the index used by optimized content queries; the installed directory-serving patch also loses its `relative_path` index, so content and directory requests use slower query paths.

Keep applying patch 0080 and retain the index until the replacement serving implementation is available.

Also at `pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py:21-22`.
</issue_to_address>

### Comment 4
<location path="pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py" line_range="35" />
<code_context>
+    ]
+
+    operations = [
+        migrations.RunPython(remove_experiment_index, migrations.RunPython.noop),
+    ]
</code_context>
<issue_to_address>
**Rollback leaves the index absent**

When an operator rolls back across migration 0025 to a release that uses the index, `migrations.RunPython.noop` does not restore the index, so migration 0024 remains applied while the restored application runs its optimized content queries without the index.

Provide a reverse operation that recreates the index when migration 0025 is reversed.
</issue_to_address>

### Comment 5
<location path="pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py" line_range="1" />
<code_context>
+from django.db import migrations
+
+_INDEX_NAME = "core_ca_relpath_pattern_idx"
</code_context>
<issue_to_address>
**Cleanup release note is missing**

When this non-trivial change is merged, no new `CHANGES/` fragment records the experiment and index cleanup, so towncrier release notes omit this non-trivial change.

Add a towncrier fragment in `CHANGES/` describing the cleanup.
</issue_to_address>

### Comment 6
<location path="pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py" line_range="22" />
<code_context>
            cursor.execute(_DROP_INDEX_SQL)
</code_context>
<issue_to_address>
**security (python.sqlalchemy.security.sqlalchemy-execute-raw-query):** Avoiding SQL string concatenation: untrusted input concatenated with raw SQL query can result in SQL Injection. In order to execute raw query safely, prepared statement should be used. SQLAlchemy provides TextualSQL to easily used prepared statement with named parameters. For complex SQL composition, use SQL Expression Language or Schema Definition Language. In most cases, SQLAlchemy ORM will be a better option.

*Source: opengrep*
</issue_to_address>

Sourcery assessment

Needs a human reviewer. 6 findings to address first, and the migration drops the experiment-owned database index, so reverting the code does not bring that index back and the restored queries may run more slowly. The index can be recreated, making the lingering effect bounded and repairable.

Blocking findings: pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py:31, pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py:21, Dockerfile:189, pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py:35, pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py:1, and 1 more


Sourcery is free for open source - if you like our reviews please consider sharing them ✨

atomic = False # DROP INDEX CONCURRENTLY cannot run in a transaction.

dependencies = [
("service", "0024_contentartifact_relative_path_pattern_index"),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Critical · Migration commands report conflicting branches

When django loads the service migration graph for migration generation or deployment, this migration and 0025_delete_domainorg remain separate leaves because 0026_andgatecontentguard follows only the latter, so Django reports conflicting migrations and blocks migration generation or deployment.

Join the migration branches by making the subsequent migration depend on this migration as well as 0025_delete_domainorg.

Prompt for AI agents
In `pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py` at line 31:

**Migration commands report conflicting branches**

When django loads the service migration graph for migration generation or deployment, this migration and `0025_delete_domainorg` remain separate leaves because `0026_andgatecontentguard` follows only the latter, so Django reports conflicting migrations and blocks migration generation or deployment.

Join the migration branches by making the subsequent migration depend on this migration as well as `0025_delete_domainorg`.

[_INDEX_NAME],
)
row = cursor.fetchone()
if row and row[0] == _INDEX_MARKER:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium · Cleanup leaves the index behind

When migration 0024 creates the index but does not successfully add its ownership comment, remove_experiment_index skips the index when its ownership marker is absent, so migration 0025 completes while the service-owned index remains in the database.

Handle the existing index by name even when its ownership comment is missing, so the cleanup removes it.

Prompt for AI agents
In `pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py` at line 21:

**Cleanup leaves the index behind**

When migration 0024 creates the index but does not successfully add its ownership comment, `remove_experiment_index` skips the index when its ownership marker is absent, so migration 0025 completes while the service-owned index remains in the database.

Handle the existing index by name even when its ownership comment is missing, so the cleanup removes it.

Comment thread Dockerfile
Comment on lines -188 to -189
COPY images/assets/patches/0080-Optimize-read-only-content-queries.patch /tmp/
RUN patch -p1 -d /usr/local/lib/pulp/lib/python${PYTHON_VERSION}/site-packages < /tmp/0080-Optimize-read-only-content-queries.patch

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium · Content requests lose query optimizations

When the image is built and migration 0025 is applied before the replacement is available, the image stops applying patch 0080 and migration 0025 drops the index used by optimized content queries; the installed directory-serving patch also loses its relative_path index, so content and directory requests use slower query paths.

Keep applying patch 0080 and retain the index until the replacement serving implementation is available.

Also at pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py:21-22.

Prompt for AI agents
In `Dockerfile` at lines 188-189:

**Content requests lose query optimizations**

When the image is built and migration 0025 is applied before the replacement is available, the image stops applying patch 0080 and migration 0025 drops the index used by optimized content queries; the installed directory-serving patch also loses its `relative_path` index, so content and directory requests use slower query paths.

Keep applying patch 0080 and retain the index until the replacement serving implementation is available.

Also at `pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py:21-22`.

]

operations = [
migrations.RunPython(remove_experiment_index, migrations.RunPython.noop),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium · Rollback leaves the index absent

When an operator rolls back across migration 0025 to a release that uses the index, migrations.RunPython.noop does not restore the index, so migration 0024 remains applied while the restored application runs its optimized content queries without the index.

Provide a reverse operation that recreates the index when migration 0025 is reversed.

Prompt for AI agents
In `pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py` at line 35:

**Rollback leaves the index absent**

When an operator rolls back across migration 0025 to a release that uses the index, `migrations.RunPython.noop` does not restore the index, so migration 0024 remains applied while the restored application runs its optimized content queries without the index.

Provide a reverse operation that recreates the index when migration 0025 is reversed.

@@ -0,0 +1,36 @@
from django.db import migrations

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium · Cleanup release note is missing

When this non-trivial change is merged, no new CHANGES/ fragment records the experiment and index cleanup, so towncrier release notes omit this non-trivial change.

Add a towncrier fragment in CHANGES/ describing the cleanup.

Prompt for AI agents
In `pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py` at line 1:

**Cleanup release note is missing**

When this non-trivial change is merged, no new `CHANGES/` fragment records the experiment and index cleanup, so towncrier release notes omit this non-trivial change.

Add a towncrier fragment in `CHANGES/` describing the cleanup.

From CLAUDE.md L76:

Uses **towncrier**. For any non-trivial change, create a file in `CHANGES/` named `{issue_number}.{category}` where category is one of: `feature`, `bugfix`, `doc`, `removal`, `deprecation`, `misc`.

)
row = cursor.fetchone()
if row and row[0] == _INDEX_MARKER:
cursor.execute(_DROP_INDEX_SQL)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

security (python.sqlalchemy.security.sqlalchemy-execute-raw-query): Avoiding SQL string concatenation: untrusted input concatenated with raw SQL query can result in SQL Injection. In order to execute raw query safely, prepared statement should be used. SQLAlchemy provides TextualSQL to easily used prepared statement with named parameters. For complex SQL composition, use SQL Expression Language or Schema Definition Language. In most cases, SQLAlchemy ORM will be a better option.

Source: opengrep

@kdelee

kdelee commented Nov 18, 2026

Copy link
Copy Markdown
Contributor Author

@larrymou9 is going to replace this PR and fight with sourcery

@kdelee

kdelee commented Nov 18, 2026

Copy link
Copy Markdown
Contributor Author

closing in favor of #1715

@kdelee kdelee closed this Nov 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant