[CI][Roadmap] Near-term CI hardening and contributor workflow improvements

[Zhifu-Liu]

Summary

This issue tracks the near-term plan for improving RL-Kernel's CI and contributor workflow.

The goal is to build on the existing CI foundation, address the most immediate gaps, and gradually align the project with mature open-source CI practices. Follow-up pull requests should remain small and independently reviewable, and should link back to this issue. Release and publishing hardening is intentionally deferred and will be tracked in a separate follow-up issue.

Current state

RL-Kernel already provides:

• pre-commit, mypy, CPU tests, and documentation checks;
• label-gated GPU CI for selected NVIDIA architectures;
• documentation deployment;
• CI image builds;
• CODEOWNERS and stale pull request management.

GPU test workflows for the WS1/WS2 ROCm workstreams are currently in flight on open PRs (e.g., #356) and should be coordinated with.

The next stage should focus on contribution compliance, reliable required checks, workflow security, and GPU CI maintainability.

Scope

1. DCO and contributor documentation

Maintainer-side configuration:

• Install the DCO GitHub App.
• Add the DCO check to the repository status-check configuration.
• Enable automatic sign-off for commits created through the GitHub web interface.

Repository-side work:

• Add the unmodified DCO 1.1 text as /DCO.
• Add a root /CONTRIBUTING.md.
• Add .github/PULL_REQUEST_TEMPLATE.md.
• Link the contribution guide from the README and documentation.
• Document git commit -s and missing-signoff repair commands.
• Document author email matching, human co-author, bot, and AI-assistance policies.
• Define a transition policy for pull requests opened before enforcement.
• Add a new-PR welcome comment covering sign-off repair and how GPU tests are triggered, without checking out pull request code.

The policy should apply prospectively. Existing history on main does not need to be rewritten.

2. Fast and stable pull request checks

Keep the default PR path CPU-safe and reasonably fast.

• Confirm stable required checks for linting, unit tests, documentation, and DCO.
• Add PR title validation.
• Add actionlint for GitHub Actions workflows.
• Add shellcheck for CI and maintenance scripts.
• Add a clean package build/install smoke test.
• Decide the supported Python matrix (Python 3.10 reaches end-of-life in October 2026).
• Add concurrency cancellation for superseded PR commits.
• Remove avoidable duplicate work between validation and deployment workflows.
• Ensure failures include useful local reproduction commands.

GPU checks should remain separate from the unconditional fast path.

3. Workflow security and dependency maintenance

• Add explicit minimal permissions to every workflow.
• Pin third-party GitHub Actions to reviewed commit SHAs.
• Add automated update proposals for GitHub Actions dependencies.
• Audit the pull_request_target boundary across all workflows that use it, not only GPU CI.
• Ensure untrusted pull request code never runs with repository secrets.
• Pin or verify externally downloaded CI binaries and important container environments.
• Review CI logs and artifacts for possible credential exposure.

4. GPU CI reliability

Continue improving the existing label-gated GPU workflow.

• Coordinate the hardware matrix work with [P0] Cross-platform Docker matrix and CI workflows for hardware regression and automated tests #173.
• Coordinate architecture mismatch handling with [WS1][CI] The GPU architecture unmatch in GPU CI. #226.
• Clearly distinguish test failures from infrastructure or capacity failures.
• Record GPU model, compute capability, driver, CUDA, PyTorch, and commit SHA in CI output.
• Resolve the unused rl-kernel-ci:cuda CI image: either wire it into GPU CI via CI_IMAGE, or retire the build workflow.
• Improve timeout, cleanup, and failure-log behavior.
• Document when maintainers should apply the needs-gpu-ci label.
• Define a controlled retry policy for infrastructure failures.

Command-driven or bot-driven GPU test selection (as in vLLM's /ci run flow) is intentionally out of near-term scope; the needs-gpu-ci label remains the triggering mechanism.

Broader consistency and performance work should continue in their existing issues, including #137, #101, #106, and #113, rather than being duplicated here.

5. CI documentation and maintenance

• Add a concise CI guide for contributors.
• Document which checks are required and which are label-gated.
• Document common failures and local reproduction commands.
• Define how flaky tests and recurring infrastructure failures should be reported and tracked.
• Document operational triage ownership for CI failures (review ownership for workflow changes is already defined in CODEOWNERS).

Suggested implementation order

1. DCO files and contributor documentation.
2. Pull request template and title validation.
3. Fast-check improvements and package smoke testing.
4. Workflow permission and dependency hardening.
5. GPU CI reliability and contributor-facing CI documentation.

Each item should be implemented in a focused pull request. Pull requests should reference this issue and any more specific implementation issue that they address.

Responsibility boundary

Community contributors can implement:

• repository documentation;
• pull request templates;
• workflow and test changes;
• CI scripts;
• dependency update configuration.

Maintainer access is required for:

• GitHub App configuration;
• Rulesets and required status checks;
• web commit sign-off settings;
• secrets and protected environments.

Completion criteria

This near-term roadmap is complete when:

• DCO requirements are documented and enforced for new contributions;
• required PR checks are stable and clearly documented;
• workflow permissions and third-party dependencies follow a reviewed security policy;
• forked PR code cannot access repository secrets;
• GPU CI failures provide enough information to distinguish code and infrastructure problems;
• contributors have clear instructions for running and repairing CI checks;
• follow-up CI work is linked and tracked through focused issues and pull requests.

@Flink-ddd @bitborne @z1ying

[Zhifu-Liu]

This is a proposal — maintainers, please feel free to edit, re-scope, or tell me to adjust.