Skip to content

Fix STS session tags on credentials vended by purge cleanup tasks - #5885

Open
FillipdotS wants to merge 1 commit into
apache:mainfrom
FillipdotS:fix/5875-purge-task-catalog-tag
Open

FillipdotS wants to merge 1 commit into
apache:mainfrom
FillipdotS:fix/5875-purge-task-catalog-tag

Conversation

@FillipdotS

Copy link
Copy Markdown

Fixes #5875

After a drop with purge, the cleanup tasks vend storage credentials with the wrong credential vending context: the
polaris:catalog and polaris:table session tags and the role session name contain the task's
name (e.g. entityCleanup_6) instead, and the namespace tag is unknown.

This breaks CloudTrail correlation. Additionally deployments whose IAM or bucket policies check aws:PrincipalTag/polaris:catalog deny the cleanup tasks, so purge even fails in those cases.

Root cause: TaskFileIOSupplier passes StorageAccessConfigProvider a resolved path containing
only the task entity. The provider takes the catalog from the first path entity and the table from
the last, so both become the task's name, and the namespace is unset.

Changes:

  • LocalIcebergCatalog.dropTableLike records the catalog name in the purge task's internal
    properties (PolarisTaskConstants.CATALOG_NAME). This covers table and view purge, and child
    tasks inherit it
  • TaskFileIOSupplier passes that catalog name and the dropped entity's TableIdentifier to a new
    StorageAccessConfigProvider.getStorageAccessConfigForTask, which builds the context from them.
    The task path is still used to look up the storage integration.
  • Request path: unchanged, except the first path entity is only used as the catalog if it is a
    CATALOG, and the now-unused TASK leaf branch is removed.

Behaviour notes:

  • Task credentials now carry the real table name and namespace instead of the task name.
  • Cleanup tasks of one table now share a credential cache key, so they share STS calls.

Tests:

  • PurgeTaskSessionTagsTest: real task executor and cleanup handlers, mocked STS client. Asserts every AssumeRole call of a drop with purge carries the right catalog, namespace and table tags and role session name.
    It needs its own test profile to enable purge and session tags

Checklist

Copilot AI balanced review requested due to automatic review settings October 27, 2026 14:49
@github-project-automation github-project-automation Bot moved this to PRs In Progress in Basic Kanban Board Oct 27, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Copilot review overview

2 open issues
What changed in this PR

Fixes incorrect STS session tag / role session name context for drop-with-purge cleanup tasks by ensuring the credential vending context is derived from the dropped table/view (catalog/namespace/table), not from the task name.

Changes:

  • Persist the originating catalog name into purge/cleanup task internal properties and assert it in existing catalog/task tests.
  • Add a dedicated task-oriented credential vending API (getStorageAccessConfigForTask) and route TaskFileIOSupplier through it.
  • Add tests validating STS AssumeRole tags/session name for purge cleanup tasks and validating context derivation logic in StorageAccessConfigProvider.
File Description
runtime/​service/​src/​test/​java/​org/​apache/​polaris/​service/​task/​TableCleanupTaskHandlerTest.java Asserts purge/cleanup tasks carry catalog name in internal properties.
runtime/​service/​src/​test/​java/​org/​apache/​polaris/​service/​task/​PurgeTaskSessionTagsTest.java End-to-end Quarkus test validating task-vended STS tags/session name during drop-with-purge.
runtime/​service/​src/​test/​java/​org/​apache/​polaris/​service/​catalog/​io/​StorageAccessConfigProviderTest.java Unit tests for context derivation (resolved path vs. task catalog+identifier).
runtime/​service/​src/​test/​java/​org/​apache/​polaris/​service/​catalog/​iceberg/​AbstractLocalIcebergCatalogTest.java Verifies drop-with-purge tasks include the catalog name internal property for table/view.
runtime/​service/​src/​main/​java/​org/​apache/​polaris/​service/​task/​TaskFileIOSupplier.java Uses the new task-specific storage access config API, passing catalog name from task properties.
runtime/​service/​src/​main/​java/​org/​apache/​polaris/​service/​catalog/​io/​StorageAccessConfigProvider.java Adds getStorageAccessConfigForTask and refactors context building to avoid task-name derived tags.
runtime/​service/​src/​main/​java/​org/​apache/​polaris/​service/​catalog/​iceberg/​LocalIcebergCatalog.java Copies storage properties on purge and injects CATALOG_NAME for cleanup task credential context.
polaris-core/​src/​main/​java/​org/​apache/​polaris/​core/​entity/​PolarisTaskConstants.java Adds CATALOG_NAME constant for task internal properties.
CHANGELOG.md Documents behavior change for STS tags/session name on purge cleanup task credentials.

Review effort: Lite

@orca-review-bot orca-review-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice fix. The root cause is right: the task-only resolved path made the provider read the task name as both catalog and table. Child tasks pick up catalogName because TableCleanupTaskHandler already copies the parent's internal properties, so all three task types are covered. I checked the request path and it's unchanged, apart from not using a root that isn't a catalog as the catalog tag. One small test ask: in StorageAccessConfigProviderTest, can we verify that getStorageIntegration still gets the task path on the task path? That lookup is what keeps the task's storage config working.

🤖 AI-generated review comment — agents make mistakes, so please verify before acting on it.

Comment thread CHANGELOG.md Resolved
@FillipdotS
FillipdotS force-pushed the fix/5875-purge-task-catalog-tag branch 2 times, most recently from b896ad5 to 4b21534 Compare October 28, 2026 11:35

@vigneshio vigneshio left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.. Clean fix & well tested.. Thanks for the contribution! @FillipdotS

@FillipdotS
FillipdotS force-pushed the fix/5875-purge-task-catalog-tag branch from 4b21534 to 4add856 Compare November 3, 2026 11:40
@adutra

adutra commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

Heads-up: #5967 has been merged, so CHANGELOG.md is no longer edited directly. Changelog entries now go in a fragment file under changelog.d/, which avoids merge conflicts on CHANGELOG.md. See changelog.d/README.md for the format.

Could you rebase onto main, revert your changes to CHANGELOG.md, and add a fragment file instead if your change needs a changelog entry? A fragment is only needed for things worth calling out in release notes, such as breaking changes, metastore schema changes, or changes that require manual steps during an upgrade. Thanks!

@FillipdotS
FillipdotS force-pushed the fix/5875-purge-task-catalog-tag branch from 4add856 to 9c6fd27 Compare November 17, 2026 11:00
@FillipdotS

Copy link
Copy Markdown
Author

Thanks @adutra! I've updated the PR to use the new changelog

@NonNull PolarisResolvedPathWrapper resolvedTaskPath) {
Optional<String> namespace =
tableIdentifier.hasNamespace()
? Optional.of(String.join(".", tableIdentifier.namespace().levels()))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't look safe, what if the namespace contains a dot character? I think it would be better to avoid concatenating namespaces altogether; if that's not possible, try using org.apache.polaris.core.rest.NamespaceUtils#joinNamespace as a safer alternative.

@FillipdotS
FillipdotS force-pushed the fix/5875-purge-task-catalog-tag branch from 9c6fd27 to 5cb77bb Compare November 19, 2026 11:58

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Purge tasks mistakenly set the polaris:catalog session tag to the task name instead of the catalog name

5 participants