Repository navigation
Fix STS session tags on credentials vended by purge cleanup tasks - #5885
FillipdotS wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Copilot review overview
2 open issues
What changed in this PR
Fixes incorrect STS session tag / role session name context for drop-with-purge cleanup tasks by ensuring the credential vending context is derived from the dropped table/view (catalog/namespace/table), not from the task name.
Changes:
- Persist the originating catalog name into purge/cleanup task internal properties and assert it in existing catalog/task tests.
- Add a dedicated task-oriented credential vending API (
getStorageAccessConfigForTask) and routeTaskFileIOSupplierthrough it. - Add tests validating STS AssumeRole tags/session name for purge cleanup tasks and validating context derivation logic in
StorageAccessConfigProvider.
| File | Description |
|---|---|
| runtime/service/src/test/java/org/apache/polaris/service/task/TableCleanupTaskHandlerTest.java | Asserts purge/cleanup tasks carry catalog name in internal properties. |
| runtime/service/src/test/java/org/apache/polaris/service/task/PurgeTaskSessionTagsTest.java | End-to-end Quarkus test validating task-vended STS tags/session name during drop-with-purge. |
| runtime/service/src/test/java/org/apache/polaris/service/catalog/io/StorageAccessConfigProviderTest.java | Unit tests for context derivation (resolved path vs. task catalog+identifier). |
| runtime/service/src/test/java/org/apache/polaris/service/catalog/iceberg/AbstractLocalIcebergCatalogTest.java | Verifies drop-with-purge tasks include the catalog name internal property for table/view. |
| runtime/service/src/main/java/org/apache/polaris/service/task/TaskFileIOSupplier.java | Uses the new task-specific storage access config API, passing catalog name from task properties. |
| runtime/service/src/main/java/org/apache/polaris/service/catalog/io/StorageAccessConfigProvider.java | Adds getStorageAccessConfigForTask and refactors context building to avoid task-name derived tags. |
| runtime/service/src/main/java/org/apache/polaris/service/catalog/iceberg/LocalIcebergCatalog.java | Copies storage properties on purge and injects CATALOG_NAME for cleanup task credential context. |
| polaris-core/src/main/java/org/apache/polaris/core/entity/PolarisTaskConstants.java | Adds CATALOG_NAME constant for task internal properties. |
| CHANGELOG.md | Documents behavior change for STS tags/session name on purge cleanup task credentials. |
Review effort: Lite
orca-review-bot
left a comment
There was a problem hiding this comment.
Nice fix. The root cause is right: the task-only resolved path made the provider read the task name as both catalog and table. Child tasks pick up catalogName because TableCleanupTaskHandler already copies the parent's internal properties, so all three task types are covered. I checked the request path and it's unchanged, apart from not using a root that isn't a catalog as the catalog tag. One small test ask: in StorageAccessConfigProviderTest, can we verify that getStorageIntegration still gets the task path on the task path? That lookup is what keeps the task's storage config working.
🤖 AI-generated review comment — agents make mistakes, so please verify before acting on it.
b896ad5 to
4b21534
Compare
vigneshio
left a comment
There was a problem hiding this comment.
LGTM.. Clean fix & well tested.. Thanks for the contribution! @FillipdotS
4b21534 to
4add856
Compare
|
Heads-up: #5967 has been merged, so Could you rebase onto |
4add856 to
9c6fd27
Compare
|
Thanks @adutra! I've updated the PR to use the new changelog |
| @NonNull PolarisResolvedPathWrapper resolvedTaskPath) { | ||
| Optional<String> namespace = | ||
| tableIdentifier.hasNamespace() | ||
| ? Optional.of(String.join(".", tableIdentifier.namespace().levels())) |
There was a problem hiding this comment.
This doesn't look safe, what if the namespace contains a dot character? I think it would be better to avoid concatenating namespaces altogether; if that's not possible, try using org.apache.polaris.core.rest.NamespaceUtils#joinNamespace as a safer alternative.
9c6fd27 to
5cb77bb
Compare

Fixes #5875
After a drop with purge, the cleanup tasks vend storage credentials with the wrong credential vending context: the
polaris:catalogandpolaris:tablesession tags and the role session name contain the task'sname (e.g.
entityCleanup_6) instead, and the namespace tag isunknown.This breaks CloudTrail correlation. Additionally deployments whose IAM or bucket policies check
aws:PrincipalTag/polaris:catalogdeny the cleanup tasks, so purge even fails in those cases.Root cause:
TaskFileIOSupplierpassesStorageAccessConfigProvidera resolved path containingonly the task entity. The provider takes the catalog from the first path entity and the table from
the last, so both become the task's name, and the namespace is unset.
Changes:
LocalIcebergCatalog.dropTableLikerecords the catalog name in the purge task's internalproperties (
PolarisTaskConstants.CATALOG_NAME). This covers table and view purge, and childtasks inherit it
TaskFileIOSupplierpasses that catalog name and the dropped entity'sTableIdentifierto a newStorageAccessConfigProvider.getStorageAccessConfigForTask, which builds the context from them.The task path is still used to look up the storage integration.
CATALOG, and the now-unusedTASKleaf branch is removed.Behaviour notes:
Tests:
PurgeTaskSessionTagsTest: real task executor and cleanup handlers, mocked STS client. Asserts every AssumeRole call of a drop with purge carries the right catalog, namespace and table tags and role session name.It needs its own test profile to enable purge and session tags
Checklist
CHANGELOG.md(if needed)site/content/in-dev/unreleased(if needed)