Create release docs automation#8925
Conversation
There was a problem hiding this comment.
Pull request overview
Adds a manually triggered GitHub Actions workflow intended to automate release documentation updates by running a Python script and opening a PR in this docs repository.
Changes:
- Introduces a new
workflow_dispatchworkflow with inputs (component, release type, version, release date, instructions, draft PR). - Sets up Python, installs the Anthropic SDK, runs a docs update script, and opens a PR via
peter-evans/create-pull-request.
|
Newest code from mattermost has been published to preview environment for Git SHA bf7c558 |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
|
Newest code from mattermost has been published to preview environment for Git SHA 836a331 |
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds a GitHub Actions workflow and a Python script that use Anthropic’s Messages API to regenerate selected docs from release metadata with truncation handling, response validation, write-safety guards, branch/PR creation, and per-file status reporting. ChangesAutomated Documentation Updates
Sequence DiagramsequenceDiagram
actor User
participant GitHub as GitHub Actions
participant Runner as update-docs job
participant Script as update_docs.py
participant Anthropic as Anthropic API
participant FileSystem as File System
participant Git as git/gh
User->>GitHub: Trigger workflow_dispatch (component, version, release_type, etc.)
GitHub->>Runner: Checkout repo, setup Python, install anthropic
Runner->>Script: Run with env vars (ANTHROPIC_API_KEY, metadata)
Script->>Script: get_files()
loop For each documentation file
Script->>FileSystem: Read file content
Script->>Script: Build user prompt (metadata + content)
Script->>Anthropic: Call Messages API (system + user prompt)
Anthropic-->>Script: Return response.content[0].text and stop_reason
Script->>Script: Validate (structure, stop_reason, non-empty, length >=50%, changed)
Script->>FileSystem: Write updated file (if safe)
end
Script-->>Git: Commit & push branch docs/{component}-v{safe_version} (if changes)
Git->>GitHub: Create PR via gh pr create (if no existing PR)
Git-->>User: PR created (draft or ready)
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
|
Newest code from mattermost has been published to preview environment for Git SHA 2eed2fe |
|
Newest code from mattermost has been published to preview environment for Git SHA cb4dc94 |
|
Newest code from mattermost has been published to preview environment for Git SHA 8b3f734 |
amyblais
added
1: Dev Review
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (1)
.github/workflows/update-docs.yml (1)
58-59: ⚡ Quick winPin the Anthropic SDK version for reproducible workflow runs.
Installing
anthropicwithout a version can pull breaking changes and make the automation flaky over time. The latest stable version is 0.97.0 and is compatible with Python 3.11.Suggested fix
- - name: Install dependencies - run: pip install anthropic + - name: Install dependencies + run: pip install -r .github/scripts/update-docs-requirements.txt# .github/scripts/update-docs-requirements.txt anthropic==0.97.0🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/update-docs.yml around lines 58 - 59, Replace the unpinned pip install in the "Install dependencies" workflow step by pinning the Anthropic SDK to 0.97.0: create a requirements file named .github/scripts/update-docs-requirements.txt containing anthopic==0.97.0 and change the step that currently runs "pip install anthropic" to "pip install -r .github/scripts/update-docs-requirements.txt" so the workflow uses the fixed anthropic==0.97.0 release for reproducible runs.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/scripts/update_docs.py:
- Around line 138-140: The current handlers in update_docs.py (including the
except FileNotFoundError block and the other early-return sites around where
errors/warnings are logged) log problems then return, allowing main() to exit
success; change those returns to fail-fast behavior by raising or exiting
non‑zero (e.g., raise RuntimeError or call sys.exit(1)) so the workflow fails
when safety checks reject an update; update the except FileNotFoundError handler
and the two locations that log error/warning (the blocks referenced around lines
152–163) to propagate a non‑zero failure instead of returning normally.
- Line 142: The print statements using f-strings with no placeholders (e.g.,
print(f" Sending to Claude...") and the similar print at line 183) should be
converted to plain strings; locate the print calls in update_docs.py (look for
the exact message " Sending to Claude..." and the other analogous message) and
change them from f-strings to normal string literals (remove the leading f) so
they are simple print(" Sending to Claude...") calls.
In @.github/workflows/update-docs.yml:
- Line 75: Sanitize the free-form inputs.version before using it in the branch
ref: replace or remove characters that can make invalid Git branch names
(spaces, slashes, colons, parentheses, and other non-alphanumeric characters
except dot and dash), collapse consecutive separators to a single dash, trim
leading/trailing separators, and normalize case; then use that sanitized value
instead of inputs.version in the branch expression (replace branch: docs/${{
inputs.component }}-v${{ inputs.version }} with branch: docs/${{
inputs.component }}-v${{ steps.sanitize-version.outputs.version }} and add a
short step named sanitize-version that computes and outputs the cleaned version
string).
---
Nitpick comments:
In @.github/workflows/update-docs.yml:
- Around line 58-59: Replace the unpinned pip install in the "Install
dependencies" workflow step by pinning the Anthropic SDK to 0.97.0: create a
requirements file named .github/scripts/update-docs-requirements.txt containing
anthopic==0.97.0 and change the step that currently runs "pip install anthropic"
to "pip install -r .github/scripts/update-docs-requirements.txt" so the workflow
uses the fixed anthropic==0.97.0 release for reproducible runs.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: dde1a4b0-2311-4c59-9766-35b752c87527
📒 Files selected for processing (2)
.github/scripts/update_docs.py.github/workflows/update-docs.yml
|
Newest code from mattermost has been published to preview environment for Git SHA 36da96b |
|
Newest code from mattermost has been published to preview environment for Git SHA 121da36 |
|
Newest code from mattermost has been published to preview environment for Git SHA 25408b2 |
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (1)
.github/scripts/update_docs.py (1)
28-32: 💤 Low valueBare
os.environ[...]at module scope emits a crypticKeyErrorwhen a required variable is absent.A missing
COMPONENTorVERSIONraisesKeyError: 'COMPONENT'with no indication that this is a required workflow input. A small validation block early inmain()(or a helper) would provide an actionable error message and keep side-effects out of module initialisation.♻️ Proposed refactor
-COMPONENT = os.environ["COMPONENT"] -RELEASE_TYPE = os.environ["RELEASE_TYPE"] -VERSION = os.environ["VERSION"] -RELEASE_DATE = os.environ["RELEASE_DATE"] -INSTRUCTIONS = os.environ.get("INSTRUCTIONS", "").strip() +def _require_env(name: str) -> str: + value = os.environ.get(name) + if not value: + print(f"ERROR: Required environment variable '{name}' is not set.") + sys.exit(1) + return value + +COMPONENT = _require_env("COMPONENT") +RELEASE_TYPE = _require_env("RELEASE_TYPE") +VERSION = _require_env("VERSION") +RELEASE_DATE = _require_env("RELEASE_DATE") +INSTRUCTIONS = os.environ.get("INSTRUCTIONS", "").strip()🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/scripts/update_docs.py around lines 28 - 32, Move environment access out of module scope and add explicit validation: stop reading COMPONENT, RELEASE_TYPE, VERSION, RELEASE_DATE, and INSTRUCTIONS at import time and instead read them inside main() (or a new helper like validate_env_vars()) and raise a clear RuntimeError or print an actionable message if any required variable (COMPONENT, VERSION, RELEASE_DATE) is missing; keep optional INSTRUCTIONS using os.environ.get(...).strip() inside the same helper, and update any references to use the local variables returned by main()/validate_env_vars() so module import no longer raises KeyError.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/scripts/update_docs.py:
- Around line 196-200: The try/except around update_file(client, filepath)
currently catches all Exception; change it to only catch expected recoverable
errors (e.g., OSError, anthropic.APIError, RuntimeError) and let fatal errors
propagate; update the except handler for the tuple (OSError, anthropic.APIError,
RuntimeError) to log the exception type and message (including type(e).__name__)
and append to errors as before, and remove or avoid a broad except Exception so
functions like update_file, client, or filepath-related fatal exceptions bubble
up instead of being swallowed.
- Around line 142-149: Guard against empty/invalid response.content and detect
truncated outputs before writing files: after calling client.messages.create
(the response object used as response.content[0].text), verify response.content
is a non-empty list and that response.content[0] has a text field (and/or type
== "text") before accessing .text to avoid IndexError/AttributeError;
additionally check response.stop_reason and refuse to accept outputs where
stop_reason == "max_tokens" (or any non-null truncation indicator) — in that
case either retry with a larger/max_tokens or surface an explicit error so the
file is not overwritten with truncated content; make max_tokens configurable
(e.g., increase to 32000) if needed and ensure any early-return paths raise or
log a clear error rather than letting the broad except() silently skip writing.
- Around line 42-69: SERVER_FILES currently includes
"source/product-overview/mattermost-desktop-releases.md" which should not be
updated for Server releases; remove that string from the SERVER_FILES list so it
only exists in DESKTOP_BASE_FILES, i.e., edit the SERVER_FILES definition to
delete the "source/product-overview/mattermost-desktop-releases.md" entry and
leave DESKTOP_BASE_FILES unchanged.
---
Nitpick comments:
In @.github/scripts/update_docs.py:
- Around line 28-32: Move environment access out of module scope and add
explicit validation: stop reading COMPONENT, RELEASE_TYPE, VERSION,
RELEASE_DATE, and INSTRUCTIONS at import time and instead read them inside
main() (or a new helper like validate_env_vars()) and raise a clear RuntimeError
or print an actionable message if any required variable (COMPONENT, VERSION,
RELEASE_DATE) is missing; keep optional INSTRUCTIONS using
os.environ.get(...).strip() inside the same helper, and update any references to
use the local variables returned by main()/validate_env_vars() so module import
no longer raises KeyError.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 28615292-da7e-4fa9-9990-46d9386a94cb
📒 Files selected for processing (2)
.github/scripts/update_docs.py.github/workflows/update-docs.yml
🚧 Files skipped from review as they are similar to previous changes (1)
- .github/workflows/update-docs.yml
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/update-docs.yml:
- Around line 58-60: The workflow step named "Install dependencies" uses an
unpinned run command `pip install anthropic`; change this to install a pinned
anthropic version (e.g., pin to a specific semantic version) to ensure
deterministic CI, and apply the same pinning to the identical `pip install
anthropic` steps in `generate-changelog.yml` and `pr-export.yml`; update the run
command in the "Install dependencies" step that currently contains `pip install
anthropic` to use the fixed version string instead.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: d630a996-a9e9-420c-9e89-71700d3b5a28
📒 Files selected for processing (2)
.github/scripts/update_docs.py.github/workflows/update-docs.yml
|
Newest code from mattermost has been published to preview environment for Git SHA d926a21 |
|
Newest code from mattermost has been published to preview environment for Git SHA 6896d53 |
|
Newest code from mattermost has been published to preview environment for Git SHA 7947f50 |
|
@NARSimoes @esarafianou - Are y'all able to help give this a dev review? |
|
Newest code from mattermost has been published to preview environment for Git SHA 8b98b8d |
| description: 'Release date (e.g., May 15, 2026)' | ||
| required: true | ||
| type: string | ||
| instructions: |
There was a problem hiding this comment.
What additional instruction are we planning to add? I wonder if we should strict the input instructions since current approach is a free form and inject anything to the model prompt.
There was a problem hiding this comment.
Addressed! Updated the description to "Keep concise — appended to the Claude prompt as-is" so it's clear what it does and nudges users toward short, specific input rather than arbitrary text.
There was a problem hiding this comment.
Thank you @amyblais, What additional instruction are we planning to add? Just trying to understand if we really need this or we can think about an alternative to avoid having a free form that allow injecting anything to the models.
There was a problem hiding this comment.
Replaced free-form instructions with a structured esr_end_date input (covers the known use case without the injection risk).
|
Newest code from mattermost has been published to preview environment for Git SHA c467bef |
|
Newest code from mattermost has been published to preview environment for Git SHA c46f9fb |
|
Newest code from mattermost has been published to preview environment for Git SHA 2958bad |
|
Newest code from mattermost has been published to preview environment for Git SHA a0e1d4f |
NARSimoes
left a comment
NARSimoes
left a comment
There was a problem hiding this comment.
Thanks for addressing the review, a few more clarifications.
| echo "Please review all changes carefully before merging." | ||
| } > pr_body.md | ||
|
|
||
| gh pr create \ |
There was a problem hiding this comment.
Consider adding --base / --head branch. For example in generate-changelog we use
--base master \
--head "changelog/${{ inputs.version }}"
There was a problem hiding this comment.
Added --base/--head to gh pr create.
| echo "No files were modified — skipping PR creation." | ||
| else | ||
| git commit -m "docs: update ${{ inputs.component }} files for v${{ inputs.version }} ${{ inputs.release_type }}" | ||
| git push origin "docs/${{ inputs.component }}-v${{ steps.vars.outputs.safe_version }}" |
There was a problem hiding this comment.
We might have some collisions here if we are planning to re-run this for same component / version. For example if the pull requests already exists or content already pushed. One possible option could be --force-with-release but would need to double check.
# Create new branch e.g. docs/server-v17
git checkout -b "docs/${{ inputs.component }}-v${{ steps.vars.outputs.safe_version }}"
...
# Push to e.g. docs/server-v17
git push origin "docs/${{ inputs.component }}-v${{ steps.vars.outputs.safe_version }}"
There was a problem hiding this comment.
Addressed - using git push --force-with-lease and check if a PR already exists before creating.
| print(f" {fp}: {err}") | ||
| sys.exit(1) | ||
| else: | ||
| print("All files processed successfully.") |
There was a problem hiding this comment.
Clarification: there are files being skipped just printing messages with Warnings / Errors and here it's printing All files processed successfully.. Is this expected?
There was a problem hiding this comment.
Refactored update_file() to return a status, and print an accurate summary that distinguishes updated/unchanged/skipped/not-found.
| # --------------------------------------------------------------------------- | ||
|
|
||
| SERVER_FILES = [ | ||
| "source/product-overview/mattermost-v11-changelog.md", |
There was a problem hiding this comment.
I wonder if this file is too heavy. If we are only interested in the ~ < 50 % of the file we could find some strategy to reduce the size of (e.g. just send part of the file, etc). Open for discussion.
There was a problem hiding this comment.
Removed from the list as we have a separate automation for the server changelog.
| description: 'Release date (e.g., May 15, 2026)' | ||
| required: true | ||
| type: string | ||
| instructions: |
There was a problem hiding this comment.
Thank you @amyblais, What additional instruction are we planning to add? Just trying to understand if we really need this or we can think about an alternative to avoid having a free form that allow injecting anything to the models.
|
Newest code from mattermost has been published to preview environment for Git SHA 46c0a09 |
|
Newest code from mattermost has been published to preview environment for Git SHA 68ef475 |
There was a problem hiding this comment.
🧹 Nitpick comments (1)
.github/workflows/update-docs.yml (1)
93-93: ⚡ Quick winUse environment variable for
inputs.versionin commit message to avoid template injection.While
inputs.componentandinputs.release_typeare choice inputs (safe),inputs.versionis a free-form string. Direct interpolation in a shell command can allow injection if the value contains shell metacharacters. The risk is mitigated by the write-access requirement, but using an environment variable is a low-effort hardening.Suggested fix
- name: Commit and push changes id: commit + env: + VERSION: ${{ inputs.version }} run: | git config user.name "github-actions[bot]" git config user.email "github-actions[bot]`@users.noreply.github.com`" git checkout -b "docs/${{ inputs.component }}-v${{ steps.vars.outputs.safe_version }}" git add source/ if git diff --cached --quiet; then echo "changed=false" >> "$GITHUB_OUTPUT" echo "No files were modified — skipping PR creation." else - git commit -m "docs: update ${{ inputs.component }} files for v${{ inputs.version }} ${{ inputs.release_type }}" + git commit -m "docs: update ${{ inputs.component }} files for v${VERSION} ${{ inputs.release_type }}" # --force-with-lease safely overwrites the remote branch on re-runs # without clobbering any concurrent pushes from other users. git push --force-with-lease origin "docs/${{ inputs.component }}-v${{ steps.vars.outputs.safe_version }}" echo "changed=true" >> "$GITHUB_OUTPUT" fi🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/update-docs.yml at line 93, The commit command directly interpolates inputs.version into a shell command (git commit -m "...${{ inputs.version }}..."), which can allow shell injection; instead, set an environment variable (e.g., VERSION) to ${{ inputs.version }} in the workflow step or job env and change the git commit invocation to reference that env var (use $VERSION) while leaving ${{ inputs.component }} and ${{ inputs.release_type }} as-is; update the step that contains the git commit -m line to read the environment variable instead of interpolating inputs.version inline.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In @.github/workflows/update-docs.yml:
- Line 93: The commit command directly interpolates inputs.version into a shell
command (git commit -m "...${{ inputs.version }}..."), which can allow shell
injection; instead, set an environment variable (e.g., VERSION) to ${{
inputs.version }} in the workflow step or job env and change the git commit
invocation to reference that env var (use $VERSION) while leaving ${{
inputs.component }} and ${{ inputs.release_type }} as-is; update the step that
contains the git commit -m line to read the environment variable instead of
interpolating inputs.version inline.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 7b3e26d8-240e-4cdb-a19c-fffca260e3cb
📒 Files selected for processing (2)
.github/scripts/update_docs.py.github/workflows/update-docs.yml
|
Newest code from mattermost has been published to preview environment for Git SHA edb07de |
Full spec doc: https://mattermost.atlassian.net/wiki/spaces/Security/pages/4518772760/Automation+-+Release+Docs+Update+GitHub+Actions+Workflow.
Adds a GitHub Actions workflow that automates docs updates for Mattermost releases. Instead of manually opening a GitHub issue, tagging claude, and pasting file URLs each time, you go to Actions → Update Docs → Run workflow and fill in four fields:
The workflow then automatically selects the right set of files for that component and release type, sends each one to Claude via the Anthropic API with the release context, and opens a PR with all the updated files — no issue, no copy-pasting URLs, no manual file list.