Repository navigation
Conversation
Drop the temporary PULP-1996/PULP-2346 experiment and its pulp-service-owned index. The released pulpcore implementation will be adopted separately after upstream release. Assisted-by: pi coding agent Co-authored-by: Elijah DeLee <kdelee@redhat.com> Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Reviewer's GuideSchema and migrationsAdd migration 0027 to concurrently remove the relative-path index only when its annotation confirms it was created by pulp-service migration 0024.
ImplementationRemove the interval content-query experiment helper and the imports and constants used only by that helper.
ConfigurationStop applying the temporary pulpcore query-optimization patch when building the pulp-service image, and remove the patch itself.
DocumentationRemove the obsolete change notes for the temporary experiment and index.
TestsDelete tests dedicated to the retired content-query experiment.
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
There was a problem hiding this comment.
Hey - I've found 1 security issue, and 3 other issues
Security issues:
- Avoiding SQL string concatenation: untrusted input concatenated with raw SQL query can result in SQL Injection. In order to execute raw query safely, prepared statement should be used. SQLAlchemy provides TextualSQL to easily used prepared statement with named parameters. For complex SQL composition, use SQL Expression Language or Schema Definition Language. In most cases, SQLAlchemy ORM will be a better option. (link)
Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments
### Comment 1
<location path="pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py" line_range="31" />
<code_context>
+ atomic = False # DROP INDEX CONCURRENTLY cannot run in a transaction.
+
+ dependencies = [
+ ("service", "0024_contentartifact_relative_path_pattern_index"),
+ ]
+
</code_context>
<issue_to_address>
**Migrations cannot run**
When django loads this migration alongside the existing service migrations, django treats this migration and the branch through `0025_delete_domainorg` and `0026_andgatecontentguard` as separate leaves, so `migrate` fails with a conflicting-migrations error and `makemigrations` cannot generate later migrations.
Make this migration depend on the existing branch, or otherwise merge the migration graph into a single leaf.
Also at `pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py:32`.
</issue_to_address>
### Comment 2
<location path="pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py" line_range="22" />
<code_context>
+ )
+ row = cursor.fetchone()
+ if row and row[0] == _INDEX_MARKER:
+ cursor.execute(_DROP_INDEX_SQL)
+
+
</code_context>
<issue_to_address>
**Directory listings become slow**
When a directory listing runs after this migration, especially on a large repository, `remove_experiment_index` drops the `text_pattern_ops` index while directory-listing queries still filter artifacts with `relative_path__startswith`; PostgreSQL scans the artifact tables and directory listings become slow.
Keep or add an index that supports the prefix queries for as long as directory listings use them.
</issue_to_address>
### Comment 3
<location path="pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py" line_range="8" />
<code_context>
+_DROP_INDEX_SQL = f'DROP INDEX CONCURRENTLY IF EXISTS "{_INDEX_NAME}"'
+
+
+def remove_experiment_index(apps, schema_editor): # noqa: ARG001
+ """Remove only the index owned by the temporary pulp-service migration."""
+ with schema_editor.connection.cursor() as cursor:
</code_context>
<issue_to_address>
**Release notes omit the cleanup**
When towncrier builds release notes for this change, `CHANGES/` has no fragment for this index-removal migration, so Towncrier omits the cleanup from the release notes.
Add a Towncrier fragment documenting the index cleanup.
Also at `pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py:35`.
</issue_to_address>
### Comment 4
<location path="pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py" line_range="22" />
<code_context>
cursor.execute(_DROP_INDEX_SQL)
</code_context>
<issue_to_address>
**security (python.sqlalchemy.security.sqlalchemy-execute-raw-query):** Avoiding SQL string concatenation: untrusted input concatenated with raw SQL query can result in SQL Injection. In order to execute raw query safely, prepared statement should be used. SQLAlchemy provides TextualSQL to easily used prepared statement with named parameters. For complex SQL composition, use SQL Expression Language or Schema Definition Language. In most cases, SQLAlchemy ORM will be a better option.
*Source: opengrep*
</issue_to_address>Sourcery assessment
Needs a human reviewer. 4 findings to address first, and the migration concurrently drops the marked database index, and reverting the code will not recreate it. If that index is still needed, query performance may degrade until it is explicitly rebuilt, but the index can be restored.
Blocking findings: pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py:31, pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py:22, pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py:8, pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py:22
| ) | ||
| row = cursor.fetchone() | ||
| if row and row[0] == _INDEX_MARKER: | ||
| cursor.execute(_DROP_INDEX_SQL) |
There was a problem hiding this comment.
🟡 Medium · Directory listings become slow
When a directory listing runs after this migration, especially on a large repository, remove_experiment_index drops the text_pattern_ops index while directory-listing queries still filter artifacts with relative_path__startswith; PostgreSQL scans the artifact tables and directory listings become slow.
Keep or add an index that supports the prefix queries for as long as directory listings use them.
Prompt for AI agents
In `pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py` at line 22:
**Directory listings become slow**
When a directory listing runs after this migration, especially on a large repository, `remove_experiment_index` drops the `text_pattern_ops` index while directory-listing queries still filter artifacts with `relative_path__startswith`; PostgreSQL scans the artifact tables and directory listings become slow.
Keep or add an index that supports the prefix queries for as long as directory listings use them.| _DROP_INDEX_SQL = f'DROP INDEX CONCURRENTLY IF EXISTS "{_INDEX_NAME}"' | ||
|
|
||
|
|
||
| def remove_experiment_index(apps, schema_editor): # noqa: ARG001 |
There was a problem hiding this comment.
🟡 Medium · Release notes omit the cleanup
When towncrier builds release notes for this change, CHANGES/ has no fragment for this index-removal migration, so Towncrier omits the cleanup from the release notes.
Add a Towncrier fragment documenting the index cleanup.
Also at pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py:35.
Prompt for AI agents
In `pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py` at line 8:
**Release notes omit the cleanup**
When towncrier builds release notes for this change, `CHANGES/` has no fragment for this index-removal migration, so Towncrier omits the cleanup from the release notes.
Add a Towncrier fragment documenting the index cleanup.
Also at `pulp_service/pulp_service/app/migrations/0025_remove_experiment_relative_path_index.py:35`.From CLAUDE.md L76:
Uses **towncrier**. For any non-trivial change, create a file in `CHANGES/` named `{issue_number}.{category}` where category is one of: `feature`, `bugfix`, `doc`, `removal`, `deprecation`, `misc`.The service app already has 0025_delete_domainorg and 0026_andgatecontentguard, so a cleanup migration numbered 0025 that depends on 0024 forks the migration graph and makes migrate fail with conflicting migrations. Renumber it to 0027 and make it depend on the latest migration, 0026_andgatecontentguard. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
| ) | ||
| row = cursor.fetchone() | ||
| if row and row[0] == _INDEX_MARKER: | ||
| cursor.execute(_DROP_INDEX_SQL) |
There was a problem hiding this comment.
security (python.sqlalchemy.security.sqlalchemy-execute-raw-query): Avoiding SQL string concatenation: untrusted input concatenated with raw SQL query can result in SQL Injection. In order to execute raw query safely, prepared statement should be used. SQLAlchemy provides TextualSQL to easily used prepared statement with named parameters. For complex SQL composition, use SQL Expression Language or Schema Definition Language. In most cases, SQLAlchemy ORM will be a better option.
Source: opengrep
Summary
0027to remove therelative_pathindex owned by pulp-service migration0024.Migration
0024remains in the repository because it is part of Django migration history; migration0027removes the deployed service-owned index.Validation
git diff --checkpassed.pulp-service:0024:core_ca_relpath_pattern_idxFollow-up pulpcore implementation: pulp/pulpcore#8300
Assisted-by: pi coding agent
Summary by Sourcery
Remove the temporary interval content-query experiment and clean up its deployed database index.
Enhancements:
Build:
Chores: