Skip to content

ci: add a Dependabot configuration - #323

Draft
andygrove wants to merge 2 commits into
apache:mainfrom
andygrove:ci-dependabot
Draft

andygrove wants to merge 2 commits into
apache:mainfrom
andygrove:ci-dependabot

Conversation

@andygrove

Copy link
Copy Markdown
Member

Which issue does this PR close?

Part of #56: the "Add a Dependabot configuration" item.

Rationale for this change

Nothing keeps the Cargo dependencies or the GitHub Actions in our workflows up to date. apache/datafusion and apache/datafusion-iceberg both use Dependabot for this.

What changes are included in this PR?

A .github/dependabot.yml, based on apache/datafusion-iceberg's:

  • Cargo, checked weekly.
    • Minor and patch updates are grouped into one pull request.
    • Major updates of datafusion* and arrow* are ignored. Under the versioning policy proposed in docs: add a versioning and API compatibility policy #319, datafusion-spark 56.x stays on DataFusion 56 and the arrow version it uses, so those major versions change only when main moves to the next DataFusion major version.
    • DataFusion is a git dependency on branch-56 for now, so Dependabot can only update its locked commit.
  • GitHub Actions, checked weekly and grouped into one pull request.
  • pip is left out on purpose. crates/sqllogictest/scripts/requirements.txt pins pyspark==4.2.0, the Spark version the sqllogictest files are checked against. Changing it is a decision, not a routine update.

There are no labels, because this repository has no auto-dependencies label yet.

Are these changes tested?

The file is valid YAML and follows apache/datafusion-iceberg's working configuration. GitHub validates dependabot.yml once it is on the default branch, and reports any errors in the repository's Dependabot settings.

Are there any user-facing changes?

No.

This PR was written by an LLM (Claude) and reviewed before opening.

🤖 Generated with Claude Code

Check Cargo dependencies and GitHub Actions weekly, grouping minor and
patch updates. Ignore major updates of datafusion* and arrow*, which
change only when main moves to the next DataFusion major version, and
leave out the PySpark runner's pinned pyspark version.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mbutrovich

Copy link
Copy Markdown
Contributor

How do we want DataFusion updates to arrive? Minor and patch updates of datafusion* aren't ignored here, but line 32 excludes them from all-other-cargo-deps and no other group covers them. So Dependabot opens a separate PR for each of our 11 datafusion* crates (docs). That happens on each 56.x release, and on each weekly run while main follows branch-56 and the branch has moved. datafusion-iceberg has the same config and got separate PRs for its DataFusion crates on the same day (apache/datafusion-iceberg#77, apache/datafusion-iceberg#78, apache/datafusion-iceberg#79).

If we want Dependabot to propose DataFusion updates, a group puts them in one PR. Leaving out update-types lets the group also take the branch-56 commit bumps, which aren't semver versions. The ignore rule above still keeps majors out.

      datafusion:
        applies-to: version-updates
        patterns:
          - "datafusion*"

If we'd rather update DataFusion by hand, as Comet does in practice, we could drop update-types from this ignore entry so Dependabot skips datafusion* entirely.

@comphead

Copy link
Copy Markdown
Contributor

Besides the datafusion* grouping question above:

  1. Library version requirements. datafusion-spark is a library, so raising a minimum in Cargo.toml (for example tokio = "1.52" to "1.53") forces the same floor on downstream users such as Comet. Dependabot's default auto strategy tries to tell libraries from apps, but I haven't checked which way it classifies this workspace. Setting it explicitly makes minor and patch updates move only Cargo.lock when the current requirement already allows the new version:

    versioning-strategy: increase-if-necessary
  2. MSRV. The workspace declares rust-version = "1.95.0", but CI only builds on stable. A Dependabot update that needs a newer Rust would pass CI and quietly break the MSRV. A cargo check job on 1.95 would catch it. That could be a follow-up.

  3. Nit: target-branch: main is the default branch, so the line can go.

@andygrove
andygrove marked this pull request as draft November 17, 2026 03:40
@andygrove
andygrove requested a review from kosiew November 17, 2026 13:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants