Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 9 additions & 8 deletions changelog.d/cloudflare-r2-credential-vending.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,13 @@

- Cloudflare R2 credential vending. The new extension module
`polaris-extensions-storage-cloudflare-r2` provides the `R2_LOCAL_SIGNING` credential vending
mechanism for S3 catalogs that use an R2 endpoint. The server signs temporary R2 credentials
locally from a parent token in its configuration, scoped to the table's location. Set the default
parent token with `polaris.storage.cloudflare-r2.access-key` and `.secret-key`, and named tokens
with `polaris.storage.cloudflare-r2.<storage>.access-key` and `.secret-key`. Named tokens
apply only when `RESOLVE_CREDENTIALS_BY_STORAGE_NAME` is true. A realm opts in by adding
mechanism for S3-compatible storage that uses an R2 endpoint. The server signs temporary R2
credentials locally from a parent token in its configuration, scoped to the table's location. Set
the default parent token with `polaris.storage.cloudflare-r2.access-key` and `.secret-key`, and
named tokens with `polaris.storage.cloudflare-r2.<storage>.access-key` and `.secret-key`. Named
tokens apply only when `RESOLVE_CREDENTIALS_BY_STORAGE_NAME` is true. A realm opts in by adding
`R2_LOCAL_SIGNING` to `SUPPORTED_S3_CREDENTIAL_VENDING_MECHANISMS`, whose default stays `[STS]`.
Every realm that opts in shares the parent token, and anyone with `CATALOG_CREATE` or
`CATALOG_WRITE_PROPERTIES` can point a catalog at any bucket the token reaches, so scope the
parent token to only the buckets those realms need. The server bundle includes the module.
The parent tokens are server-wide: every realm that opts in can use them. Anyone with
`CATALOG_CREATE` or `CATALOG_WRITE_PROPERTIES` chooses a catalog's buckets within the reach of
its parent token, and its `storageName` when named tokens apply, so each parent token should
cover only the buckets its catalogs need. The server bundle includes the module.
Original file line number Diff line number Diff line change
Expand Up @@ -50,9 +50,8 @@
* credentialVendingMechanism} is {@code R2_LOCAL_SIGNING}, scoped to one bucket and the table's key
* prefixes. Credentials are minted locally from the server-side parent token (see {@link
* CloudflareR2TemporaryCredentialSigner}); no call to Cloudflare is made. The token subject and
* audience come from the catalog's {@code endpoint}. Each time a credential is minted, the
* integration checks the {@code endpoint}, {@code pathStyleAccess}, and {@code region} again, so a
* stored config that never passed the mechanism's {@code validate} fails closed.
* audience come from the catalog's {@code endpoint}. {@link #compute} states the checks that run
* again each time a credential is minted.
*/
public class CloudflareR2CredentialsStorageIntegration
extends CachingStorageIntegration<AwsStorageConfigurationInfo> {
Expand Down Expand Up @@ -247,11 +246,9 @@ private static String forTable(CredentialVendingContext context) {
* grant location's raw path minus exactly one leading slash, with one trailing slash added only
* when the path has none.
*
* <p>The module refuses a location with an empty path segment. This method throws an {@link
* IllegalArgumentException} for a grant location that holds {@code //} anywhere after the bucket,
* including right after it. It checks every location before it returns, so the result does not
* depend on the order of the locations. {@link R2LocalSigningCredentialVendingMechanism#validate}
* applies the same rule to the allowed locations at create and update.
* <p>Every grant location must pass {@link
* R2LocalSigningCredentialVendingMechanism#requireNoEmptyPathSegment}. This method checks every
* location before it returns, so the result does not depend on the order of the locations.
*
* <p>An empty result scopes the credential to the whole bucket, so it is returned only when a
* grant is literally at bucket root ({@code s3://bucket} or {@code s3://bucket/}). That path is
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ public PolarisStorageIntegration integrationFor(AwsStorageConfigurationInfo stor
* <ol>
* <li>The endpoint is an R2 endpoint ({@link CloudflareR2Endpoint#parse}).
* <li>{@code pathStyleAccess} is true and {@code region} is {@code auto}.
* <li>The STS and KMS fields are absent.
* <li>The STS and KMS fields are absent, and {@code stsUnavailable} is not true.
* <li>No allowed location holds an empty path segment.
* <li>A parent token resolves for the config.
* </ol>
Expand All @@ -120,7 +120,7 @@ public void validate(

/**
* The extension requires {@code pathStyleAccess=true} and {@code region=auto}. {@link #validate}
* applies the rule at create and update, and the integration applies it again at vend time.
* and {@link CloudflareR2CredentialsStorageIntegration#compute} both call this method.
*/
static void requirePathStyleAndAutoRegion(AwsStorageConfigurationInfo config) {
if (!Boolean.TRUE.equals(config.getPathStyleAccess())) {
Expand All @@ -132,9 +132,10 @@ static void requirePathStyleAndAutoRegion(AwsStorageConfigurationInfo config) {
}

/**
* An R2 catalog has no role, no STS, and no KMS. A true {@code stsUnavailable} would also make
* the access delegation resolver fall back to remote signing. The Python CLI sends false for both
* flags by default, so false is accepted.
* An R2 catalog has no role, no STS, and no KMS, so the fields that configure them must be
* absent. A true {@code stsUnavailable} is refused, because it makes the access delegation
* resolver fall back to remote signing. False is accepted, because the Python CLI sends it by
* default. {@code kmsUnavailable} is accepted with any value: the integration never reads it.
*
* <p>The management API cannot set {@code userArn}, {@code currentKmsKey}, or {@code
* allowedKmsKeys} on the stored config: core does not copy {@code userArn} from the request, and
Expand All @@ -150,18 +151,13 @@ private static void requireStsAndKmsFieldsAbsent(AwsStorageConfigurationInfo upd
requireAbsent("stsEndpoint", updated.getStsEndpoint());
requireNotTrue("stsUnavailable", updated.getStsUnavailable());
requireAbsent("endpointInternal", updated.getEndpointInternal());
requireNotTrue("kmsUnavailable", updated.getKmsUnavailable());
requireAbsent("currentKmsKey", updated.getCurrentKmsKey());
requireAbsentList("allowedKmsKeys", updated.getAllowedKmsKeys());
requireAbsentList(ENCRYPTION_KEY_FIELDS, updated.getEncryptionKeys());
requireAbsentList("decryptionKeys", updated.getDecryptionKeys());
}

/**
* An allowed location must not contain an empty path segment. The module refuses such a location
* at create and update (allowed locations) and at vend time (grant locations, in the storage
* integration). The rule sees only the allowed locations of the config.
*/
/** Applies {@link #requireNoEmptyPathSegment} to each allowed location of the config. */
private static void requireNoEmptyPathSegments(AwsStorageConfigurationInfo updated) {
for (String location : updated.getAllowedLocations()) {
requireNoEmptyPathSegment("allowed location", location);
Expand All @@ -170,9 +166,10 @@ private static void requireNoEmptyPathSegments(AwsStorageConfigurationInfo updat

/**
* Refuses a location with an empty path segment, which is {@code //} anywhere after the bucket,
* including right after it. {@code description} names the kind of location in the message: {@code
* "allowed location"} at create and update, and {@code "location"} at vend time. Both sites call
* this method, so the two messages use one wording.
* including right after it. {@link #validate} calls this method for the allowed locations at
* create and update, and {@link CloudflareR2CredentialsStorageIntegration#normalizedPrefixes}
* calls it for the grant locations at vend time. {@code description} names the kind of location
* in the message: {@code "allowed location"} or {@code "location"}.
*/
static void requireNoEmptyPathSegment(String description, String location) {
String path = location.substring(location.indexOf("://") + 3);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -196,7 +196,7 @@ void aCreateWithAnExplicitAllowedListIsAcceptedWhenOnlyTheDefaultBaseLocationHas

/**
* The Python CLI sends {@code stsUnavailable=false} and {@code kmsUnavailable=false} by default.
* Only {@code true} is refused, so a catalog created that way is accepted.
* A catalog created that way is accepted.
*/
@Test
void aCatalogWithTheCliDefaultFlagsIsAccepted() {
Expand All @@ -210,6 +210,18 @@ void aCatalogWithTheCliDefaultFlagsIsAccepted() {
assertThat(fetched.getCredentialVendingMechanism()).isEqualTo(R2_LOCAL_SIGNING);
}

/** R2 has no KMS, so a catalog may say so with {@code kmsUnavailable=true}. */
@Test
void aCatalogWithKmsUnavailableTrueIsAccepted() {
createCatalog(
services, catalog("r2-no-kms", r2Config(ALLOWED).setKmsUnavailable(true).build()));

AwsStorageConfigInfo fetched =
(AwsStorageConfigInfo) fetchCatalog(services, "r2-no-kms").getStorageConfigInfo();
assertThat(fetched.getCredentialVendingMechanism()).isEqualTo(R2_LOCAL_SIGNING);
assertThat(fetched.getKmsUnavailable()).isTrue();
}

@Test
void aCatalogUnderAnotherMechanismUpdatesToR2WithTheSameEndpoint() {
createCatalog(services, catalog("switch", otherMechanismConfig(ALLOWED).build()));
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -200,6 +200,13 @@ void acceptsExplicitFalseForStsUnavailableAndKmsUnavailable() {
.doesNotThrowAnyException();
}

/** R2 has no KMS and the integration never reads the flag, so true is accepted. */
@Test
void acceptsTrueForKmsUnavailable() {
assertThatCode(() -> defaultMechanism().validate(null, r2Config().kmsUnavailable(true).build()))
.doesNotThrowAnyException();
}

@Test
void acceptsEmptyKmsKeyLists() {
AwsStorageConfigurationInfo updated =
Expand Down Expand Up @@ -316,7 +323,6 @@ static Stream<Arguments> forbiddenFields() {
forbidden("stsEndpoint", "set", b -> b.stsEndpoint("https://sts.example.com")),
forbidden("stsUnavailable", "true", b -> b.stsUnavailable(true)),
forbidden("endpointInternal", "set", b -> b.endpointInternal(ENDPOINT)),
forbidden("kmsUnavailable", "true", b -> b.kmsUnavailable(true)),
forbidden("currentKmsKey", "set", b -> b.currentKmsKey(KMS_KEY)),
forbidden("allowedKmsKeys", "set", b -> b.allowedKmsKeys(List.of(KMS_KEY))),
// Core stores the request's currentKmsKey and allowedKmsKeys as encryptionKeys, so the
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -237,4 +237,18 @@ void aCatalogWithTheCliDefaultFlagsStillVendsCredentials() {
assertThat(loaded.config().keySet()).containsAll(CREDENTIAL_KEYS);
assertThat(loaded.credentials().size()).isEqualTo(1);
}

/** {@code kmsUnavailable=true} does not change vending: the integration never reads the flag. */
@Test
void aCatalogWithKmsUnavailableTrueStillVendsCredentials() {
TestServices svc = services(Map.of(), FIXED_TOKEN);
String cat = "r2nokms";
createStructuredTable(
svc, cat, r2Config(List.of(BASE + "/" + cat)).setKmsUnavailable(true).build());

LoadTableResponse loaded = loadTable(svc, cat, "ns", "t1", "vended-credentials");

assertThat(loaded.config().keySet()).containsAll(CREDENTIAL_KEYS);
assertThat(loaded.credentials().size()).isEqualTo(1);
}
}
Loading