Skip to content

ci: set timeout-minutes on every workflow job - #7679

Merged
andygrove merged 1 commit into
apache:mainfrom
andygrove:ci/job-timeouts
Nov 19, 2026
Merged

andygrove merged 1 commit into
apache:mainfrom
andygrove:ci/job-timeouts

Conversation

@andygrove

Copy link
Copy Markdown
Member

Which issue does this PR close?

There is no issue. This came up while investigating CI failures on the branch-1.2 backports.

Rationale for this change

No test or build job in our workflows sets timeout-minutes, so each one gets GitHub's six hour default. On #7645 the PR Build (Linux) / Spark 3.4, JDK 17, Scala 2.12 [shuffle] job hung in CometShuffleSuite at 21:54 and was cancelled at 03:49. The same job takes 10 to 14 minutes on other pull requests. For those six hours it held a runner, and nobody saw a failure.

What changes are included in this PR?

Every job that runs on a runner now sets timeout-minutes. Each limit is about two to three times the slowest successful run of that job across recent merge queue, nightly and pull request runs, rounded up.

Jobs Slowest seen Limit
macOS test matrix 83 min 180
Spark SQL shards, Iceberg suites, Delta contrib 46 to 54 min 120
Native builds, Linux PR test matrix, Miri 26 to 46 min 90
Rust tests, native executors, TPC-H and TPC-DS verify, PyArrow UDF, Delta build gate 11 to 26 min 60
Lint, Preflight, CodeQL, benchmark check under 10 min 30
Change detection, Required Checks, nightly report and other housekeeping under 1 min 10 to 15

A job that calls a reusable workflow cannot set timeout-minutes, so the limits are on the jobs inside pr_build_linux.yml, spark_sql_test_reusable.yml and the other reusable workflows. Every limit is well below the merge queue's check_response_timeout_minutes: 300.

A new "Job timeouts" section in .github/workflows/README.md explains how the limits were chosen. It also says that a pull request that makes a job slower should raise that job's limit, not remove it.

How are these changes tested?

Every workflow file still parses, and a script confirmed that every job with runs-on now has timeout-minutes. The README passes prettier --check. This pull request's own CI runs under the new limits. Some limits are based on only two successful runs, mostly with a warm cache, so the build limits are set generously to allow for a cold cache.

🤖 Generated with Claude Code

Without timeout-minutes a job runs for GitHub's six hour default. A hung
suite then holds a runner, and its pipeline, for six hours before it
reports. Set a limit on every job that runs on a runner, at about two to
three times its slowest recent successful run, and document how the
limits were chosen in the workflows README.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot added build Build environment enhancement New feature or request area:ci CI/CD, GitHub Actions, build tooling area:Iceberg area:udf area:Delta Delta Lake contrib (native scan, deletion vectors) labels Nov 18, 2026
@andygrove
andygrove requested a review from comphead November 18, 2026 19:53

@comphead comphead left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @andygrove I was actually thinking about the same last week

@andygrove
andygrove added this pull request to the merge queue Nov 18, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Nov 19, 2026
@andygrove
andygrove added this pull request to the merge queue Nov 19, 2026
Merged via the queue into apache:main with commit d187e68 Nov 19, 2026
25 checks passed
bvolpato pushed a commit to bvolpato/datafusion that referenced this pull request Nov 19, 2026
## Which issue does this PR close?

Related to apache#25148. This follows apache/datafusion-comet#7679.

## Rationale for this change

GitHub Actions defaults to six hours per job. Explicit job timeouts let
CI report a hung job sooner and free its runner.

## What changes are included in this PR?

- Add timeouts to the 49 runner jobs that lacked one, so all 52 runner
jobs have an explicit limit.
- Use recent successful Runs-On and GitHub-hosted jobs, including fork
runs, to size the limits. The Linux test took up to about 10 minutes
upstream and 25 minutes on a fork in the sample. Its 90-minute limit
leaves room for cold caches. [Upstream
run](https://github.com/apache/datafusion/actions/runs/44273230362/job/134695832557),
[fork
run](https://github.com/kumarUjjawal/datafusion/actions/runs/42294482708/job/128158781944).
- Use the existing 15-minute `ubuntu-slim` runner limit for the docs
jobs, Verify Vendored Code, and Detect Unused Dependencies.
- Document how to choose and adjust job timeouts in the contributor
guide.

## What is the testing strategy for this PR?

Parsed all 20 workflows and verified every runner job has a timeout.
Compared the parsed YAML against `main` to confirm the timeout additions
are the only workflow changes. The existing CI-script tests,
required-check mapping, and Prettier checks passed.

## Are there any user-facing changes?

CI jobs now stop when they reach their configured timeout.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:ci CI/CD, GitHub Actions, build tooling area:Delta Delta Lake contrib (native scan, deletion vectors) area:Iceberg area:udf build Build environment enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants