Repository navigation
Reject non-IMPLICIT authentication for HADOOP, HIVE and BIGQUERY connections - #6145
Open
youngrae81 wants to merge 1 commit into
Open
youngrae81 wants to merge 1 commit into
youngrae81 wants to merge 1 commit into
Conversation
…ections The Hadoop, Hive and BigQuery Metastore federation factories only support IMPLICIT authentication and throw IllegalStateException for any other type when the catalog is first used. Catalog creation only checked that the connection type and the authentication type were each allowed, so such a catalog was created, its secret was stored, and every later request against it failed with a 500. A new CatalogConfigValidator rejects these combinations with a 400. Like the other validators it runs on create and update, before any secret is written. Fixes apache#6138
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Hadoop, Hive and BigQuery Metastore federation factories only support
IMPLICITauthentication and throwIllegalStateExceptionfor any other type when the catalog is first used.Catalog creation only checked that the connection type and the authentication type were each in their allow-lists, so a
HADOOPconnection with, for example,BEARERauthentication was created, its secret was stored, and every later request against the catalog failed with a 500.This adds
ImplicitOnlyConnectionValidator, aCatalogConfigValidatorthat rejects a non-IMPLICITauthentication type for these three connection types with a 400 (HADOOP connections only support IMPLICIT authentication, but BEARER was requested).Like the existing
BigLakeCatalogValidator, it is picked up byCatalogConfigValidatorsand runs on create and update, before any secret is written, in line with the dev@ thread "Applying CatalogConfigValidator to federated catalog and storage STS endpoint validation". A missingauthenticationParametersis left to the existing request validation (#5909), and other connection types are not affected.Since it also runs on update, a catalog that was already created with such a combination now gets a 400 on update as well (
UpdateCatalogRequestcannot change the connection, so this only affects pre-existing catalogs). Such a catalog cannot serve any request today, so the changelog entry says to drop and recreate it.Two things I considered and want to flag:
extensions/federation/*, and this validator restates it inruntime/service. Putting aCatalogConfigValidatornext to each factory would keep the rule in one place, but the three extensions would each need a bean and the rule is the same for all of them. I went with one validator here, matchingBigLakeCatalogValidator; happy to move it if you prefer the per-extension form.ENABLE_CATALOG_FEDERATIONcheck, so with federation disabled aHADOOP+BEARERrequest now reports the authentication problem rather than "feature not enabled".BigLakeCatalogValidatoralready behaves this way.Not included: the BigQuery factory also checks
warehouseandgcp.bigquery.project-idonly at first use. That could go into the same validator, but I kept this PR to the authentication type; happy to add it if preferred.Tests:
TestServicesnow installs the validator like production does.ManagementServiceTestgets a parameterized test forHADOOP,HIVEandBIGQUERYwithBEARERauthentication; all three fail onmain(the catalog is created) and pass with this change, which also checks that the catalog is not persisted.ImplicitOnlyConnectionValidatorTestcoversIMPLICIT,BEARER, missing authentication,ICEBERG_RESTand catalogs without a connection../gradlew format compileAlland:polaris-runtime-service:checkpass.Fixes #6138
Checklist
changelog.d/(if needed, seechangelog.d/README.md)site/content/in-dev/unreleased(if needed) — not needed, the federation docs already state that these connection types only supportIMPLICITAI assistance was used to prepare this change; I reviewed and tested it end to end.