Repository navigation
[NO-SNOW] Route Maven deploy scripts through Artifactory mirror - #1153
Merged
Merged
Conversation
Maven Central (repo.maven.apache.org) began returning HTTP 429 during the 4.4.3-unshaded release when resolving com.google.cloud:first-party-dependencies:3.18.0 (a transitive parent POM of com.google.cloud.opentelemetry:exporter-metrics). The existing settings.xml used <repository id="central"> to redirect to Artifactory, but that override does not intercept transitive parent-POM lookups, so those fell back to public Maven Central and got rate-limited. Replace the per-repository override with a <mirror mirrorOf="*,!ossrh,!snapshot"> block, which intercepts every outbound repository request at the transport layer. This forces transitive parent-POM resolution through Snowflake's internal Artifactory too. Uses the globally-trusted artifactory.ci1.us-west-2.aws-dev.app.snowflake.com URL (see Artifactory onboarding wiki guidance). Applied to all four deploy scripts: deploy.sh, snapshot_deploy.sh, unshaded_deploy.sh, unshaded_snapshot_deploy.sh. Also drops a stray <n> tag typo that was producing "[WARNING] Unrecognised tag: 'n'" at build time. Verified by re-cutting the 4.4.3-unshaded release: previously-failing artifact resolution now succeeds against the Artifactory mirror. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
sfc-gh-alhuang
force-pushed
the
alhuang-artifactory-mirror
branch
from
May 7, 2026 20:58
cff4587 to
02eb2cc
Compare
sfc-gh-ggeng
approved these changes
May 7, 2026
sfc-gh-alhuang
enabled auto-merge (squash)
May 7, 2026 21:43
sfc-gh-miqbal
added a commit
that referenced
this pull request
Jun 25, 2026
The Build & Test / e2e jobs resolve from Maven Central directly and hit Central's rate-limiting (HTTP 429 / read timeouts), causing recurring, artifact-rotating 'could not be resolved / Read timed out' failures (surefire-booter, surefire-junit47, file-management, etc.). The deploy scripts already mirror through Snowflake's internal Artifactory for the same reason (commit #1153); this applies the same anonymous mirror to the build: - add mvn_settings_ci.xml (mirrorOf=*,!ossrh,!snapshot -> Artifactory) - scripts/run_gh_actions.sh: pass --settings to its mvn invocations - e2e-jar-test/run_e2e_jar_test.sh: install it as ~/.m2/settings.xml so every mvn call (incl. those run from the subdirectory) uses it - End2EndTest.yml: pass --settings on the two inline Windows mvn commands Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
3 tasks
sfc-gh-aminyaylov
added a commit
to snowflakedb/snowflake-kafka-connector
that referenced
this pull request
Oct 2, 2026
…y.sh (#1575) ## Summary - `deploy.sh` now generates a Maven `<mirror>` so Releng `MavenPushKafkaConnector` resolves Central through Artifactory instead of `repo.maven.apache.org` (HTTP 429 during 4.2.0, and earlier on 3.5.4). - Same shape as the one-time Releng workaround ([jenkins_utils#33708](https://github.com/snowflake-eng/jenkins_utils/pull/33708)) and the existing CI job (`BuildKafkaConnectorArtifactory.groovy`): - `development-maven-clean-virtual` (plain `development-maven-virtual` 404s on cache miss for new versions) - `mirrorOf=central` only, so the pom's Confluent/Cloudera repos stay direct - mirror id `artifactory-central-mirror` so it does not collide with the Sonatype `<server><id>central</id>` publish credential - Publish path is unchanged: `central-publishing-maven-plugin` still uploads via its own HTTP client using those Sonatype credentials. ## Why this was never migrated Regular connector CI already uses Artifactory. The company Nexus→Artifactory work (SNOW-1005987) also did not assign a Drivers `deploy.sh` task, and Releng's Artifactory PRs for this job were Docker-pull changes, not Maven resolution. `deploy.sh` stayed a Sonatype-credentials-only settings file, so `MavenPush` kept downloading from public Central until the 429s. ingest-java hit the same class of failure and fixed it in [snowflake-ingest-java#1153](snowflakedb/snowflake-ingest-java#1153). Kafka never got the analogous repo change. After this merges, the Releng `mvn` shim in jenkins_utils#33708 can be removed. ## Test plan - [ ] Inspect the generated `mvn_settings_central_deploy.xml` (heredoc in `deploy.sh`): mirror URL is `development-maven-clean-virtual`, `mirrorOf` is `central`, mirror id is not `central` - [ ] Next `MavenPushKafkaConnector` / `ReleaseClientKafkaConnectorSelf` run: no `repo.maven.apache.org` 429s while resolving plugins/BOMs; Confluent package step still resolves `packages.confluent.io` / `repository.cloudera.com` - [ ] Maven Central publish still succeeds (plugin uses the `central` server credentials, not the mirror) Made with [Cursor](https://cursor.com)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
<repository id="central">override with a<mirror mirrorOf="*,!ossrh,!snapshot">block in the settings.xml generated bydeploy.sh,snapshot_deploy.sh,unshaded_deploy.sh, andunshaded_snapshot_deploy.sh.https://artifactory.ci1.us-west-2.aws-dev.app.snowflake.com/artifactory/development-maven-virtual(Snowflake's internal Artifactory, globally-trusted cert per onboarding wiki).<n>typo (should have been<name>) that was producing[WARNING] Unrecognised tag: 'n'at build time.Why
Releasing 4.4.3-unshaded failed with:
The existing
<repository id="central">redefinition only catches repos declared in the effective POM. Maven resolves transitive parent-POM lookups (likefirst-party-dependencies:3.18.0, a parent ofexporter-metrics) using the parent's declared repositories, bypassing the override and hitting public Maven Central directly — where we got rate-limited.A
<mirror>intercepts at the HTTP transport layer, so every outbound request (including transitive parent/BOM resolution) is rewritten to Artifactory.ossrhandsnapshotare excluded frommirrorOfdefensively — they're<server>ids (credentials forcentral-publishing-maven-plugin), not repository ids, so mirrors wouldn't match them anyway, but the exclusion makes publishing-intent explicit.Test plan
alhuang-release-4.4.3-unshaded: release pipeline re-ran successfully;first-party-dependencies:3.18.0now resolves via Artifactory, norepo.maven.apache.orgURLs in logs.bash deploy.sh/bash snapshot_deploy.sh/bash unshaded_deploy.sh/bash unshaded_snapshot_deploy.shthrough the heredoc step (or inspect the script source). The<mirror>block should contain the Artifactory URL andmirrorOf=*,!ossrh,!snapshot.central-publishing-maven-pluginupload path unaffected (it uploads via its own configured URL using server-id auth, not via a mapped repository id).🤖 Generated with Claude Code