Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 10 additions & 8 deletions .github/workflows/codeql.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,14 +30,6 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
- name: Generate GitHub App token
id: app-token
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
with:
app-id: ${{ secrets.DOCS_BOT_APP_ID }}
private-key: ${{ secrets.DOCS_BOT_APP_PRIVATE_KEY }}
owner: github
repositories: docs-engineering

- uses: github/codeql-action/init@e296a935590eb16afc0c0108289f68c87e2a89a5 # v4.30.7
with:
Expand All @@ -50,6 +42,16 @@ jobs:
with:
slack_token: ${{ secrets.SLACK_DOCS_BOT_TOKEN }}

- name: Generate GitHub App token
if: ${{ failure() && github.event_name != 'pull_request' }}
id: app-token
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
with:
app-id: ${{ secrets.DOCS_BOT_APP_ID }}
private-key: ${{ secrets.DOCS_BOT_APP_PRIVATE_KEY }}
owner: github
repositories: docs-engineering

- uses: ./.github/actions/create-workflow-failure-issue
if: ${{ failure() && github.event_name != 'pull_request' }}
with:
Expand Down
15 changes: 15 additions & 0 deletions .github/workflows/purge-fastly.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ on:

permissions:
contents: read
deployments: read

# Serialize full-cache purges so two can't overlap and leave the cache in an
# unknown state. Every other run (per-deploy, per-language) gets a unique group
Expand Down Expand Up @@ -104,6 +105,20 @@ jobs:
fi
npm run purge-fastly -- "${args[@]}"

- name: Hard-purge changed English content URLs
# Prod deploys only. The soft purge above just marks `language:en` stale,
# so stale-while-revalidate can keep serving the pre-deploy copy of a
# just-changed page for a while. This evicts the specific English URLs
# whose content/ files changed in this deploy, so the next request fetches
# fresh. By the time the deploy succeeds the old pods are already gone, so
# the refill is deterministically the new content. data/ changes stay
# covered by the soft purge above (too many URLs to enumerate cheaply).
if: ${{ github.event_name == 'deployment_status' }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
HEAD_SHA: ${{ github.event.deployment.sha }}
run: npm run purge-fastly-changed-content

- uses: ./.github/actions/slack-alert
if: ${{ failure() && github.event_name != 'workflow_dispatch' }}
with:
Expand Down
4 changes: 4 additions & 0 deletions content/actions/get-started/understand-github-actions.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,10 @@ For a complete list of events that can be used to trigger workflows, see [Events

A **job** is a set of **steps** in a workflow that is executed on the same **runner**. Each step is either a shell script that will be executed, or an **action** that will be run. Steps are executed in order and are dependent on each other. Since each step is executed on the same runner, you can share data from one step to another. For example, you can have a step that builds your application followed by a step that tests the application that was built.

{% ifversion actions-nga %}
Steps run in order by default, but you can also run selected steps concurrently when your workflow benefits from parallel execution, such as starting a long-running service while later steps continue. For more information, see [AUTOTITLE](/actions/reference/workflows-and-actions/workflow-syntax#jobsjob_idstepsbackground).
{% endif %}

You can configure a job's dependencies with other jobs; by default, jobs have no dependencies and run in parallel. When a job takes a dependency on another job, it waits for the dependent job to complete before running.

You can also use a **matrix** to run the same job multiple times, each with a different combination of variables—like operating systems or language versions.
Expand Down
137 changes: 137 additions & 0 deletions content/actions/reference/workflows-and-actions/workflow-syntax.md
Original file line number Diff line number Diff line change
Expand Up @@ -903,6 +903,143 @@ The maximum number of minutes to run the step before killing the process. Maximu

Fractional values are not supported. `timeout-minutes` must be a positive integer.

{% ifversion actions-nga %}

## `jobs.<job_id>.steps[*].background`

Runs a step asynchronously so the job continues to the next step without waiting for it to finish. Use `background: true` for long-running processes, such as databases, servers, or monitoring tasks, that need to run alongside other steps. You synchronize with background steps later using [`wait`](#jobsjob_idstepswait) or [`wait-all`](#jobsjob_idstepswait-all) or stop them with [`cancel`](#jobsjob_idstepscancel).

You can use `background` on steps that use `run` or `uses`. To reference a background step from [`wait`](#jobsjob_idstepswait) or [`cancel`](#jobsjob_idstepscancel), give it an [`id`](#jobsjob_idstepsid). A maximum of 10 background steps can run concurrently in a single job; additional background steps are queued until a slot is free.

Outputs and environment changes from a background step are only available after you run a `wait` or `wait-all` step that includes it. If a background step fails, the job fails at the next `wait` or `wait-all` that includes it (unless [`continue-on-error`](#jobsjob_idstepscontinue-on-error) is set on that step). An implicit `wait-all` runs before any post-job cleanup.

Use `background` when you need fine-grained control: starting a long-running process (like a server or database) that stays up while later steps run, referencing a specific step with [`wait`](#jobsjob_idstepswait) or [`cancel`](#jobsjob_idstepscancel), or interleaving background work with other steps. If you instead have a self-contained group of steps that should all finish before the job continues, [`parallel`](#jobsjob_idstepsparallel) is a more convenient shorthand.

### Example: Running a step in the background

```yaml
steps:
- name: Start server
id: server
run: npm start
background: true

- name: Run tests against the server
run: npm test

- name: Wait for the server step to finish
wait: server
```

## `jobs.<job_id>.steps[*].wait`

Pauses the job until one or more background steps complete. A `wait` step performs no work itself, it only blocks until the referenced background steps finish. Provide a single step `id` as a string, or multiple step `id`s as an array.

After a `wait` step completes, the outputs of the referenced background steps become available to subsequent steps. If a referenced background step failed, the `wait` step fails too.

### Example: Waiting for specific background steps

```yaml
steps:
- name: Build frontend
id: build-frontend
run: npm run build:frontend
background: true

- name: Build backend
id: build-backend
run: npm run build:backend
background: true

- name: Run linter while builds run
run: npm run lint

- name: Wait for both builds to finish
wait: [build-frontend, build-backend]

- name: Run tests
run: npm test
```

## `jobs.<job_id>.steps[*].wait-all`

Pauses the job until all active background steps complete. This is useful when several background steps are running and you want them all to finish before continuing. Like `wait`, the `wait-all` step fails if any of the background steps it waits on failed, unless you set [`continue-on-error`](#jobsjob_idstepscontinue-on-error) to `true`.

The `wait-all` keyword takes no arguments.

### Example: Waiting for all background steps

```yaml
steps:
- name: Start database
id: db
run: docker run -d postgres:15
background: true

- name: Start cache
id: cache
run: docker run -d redis:7
background: true

- name: Run integration tests
run: npm run test:integration

- name: Wait for all services to stop
wait-all:
```

## `jobs.<job_id>.steps[*].cancel`

Gracefully terminates a running background step. The runner sends the step's process a termination signal (`SIGTERM`) so it can clean up, and forcibly stops it (`SIGKILL`) if it does not exit within a short grace period. The `cancel` keyword targets a single background step by its `id`.

### Example: Canceling a background step

```yaml
steps:
- name: Start long-running monitor
id: monitor
run: ./scripts/monitor.sh
background: true

- name: Run the main task
run: npm test

- name: Stop the monitor
cancel: monitor
```

## `jobs.<job_id>.steps[*].parallel`

Runs a group of steps concurrently, then waits for all of them to finish before continuing. The `parallel` keyword is shorthand: every step in the group runs as a background step, with an implicit `wait` at the end of the group. Use it when you have an independent group of steps that can run at the same time and you don't need to reference them individually.

Use `parallel` when you have a self-contained group of steps that should all finish before the job moves on, such as building several components at once. Use [`background`](#jobsjob_idstepsbackground) when you need finer control: starting a long-running process (like a server or database) that stays up while later steps run, referencing a specific step with [`wait`](#jobsjob_idstepswait) or [`cancel`](#jobsjob_idstepscancel), or interleaving background work with other steps. In short, `parallel` is more limited but more convenient for the "run this group at once" case, while `background` is the general-purpose primitive.

Each step in the group is subject to the same 10-step concurrency limit as other background steps.

### Example: Running steps in parallel

```yaml
steps:
- uses: {% data reusables.actions.action-checkout %}

- parallel:
- name: Build frontend
run: npm run build:frontend

- name: Build backend
run: npm run build:backend

- name: Build docs
run: npm run build:docs

- name: Run tests after all builds complete
run: npm test
```

The group above is equivalent to declaring each step with `background: true` followed by a `wait` step.

{% endif %}

## `jobs.<job_id>.timeout-minutes`

The maximum number of minutes to let a job run before {% data variables.product.prodname_dotcom %} automatically cancels it. Default: 360
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ Jenkins uses directives to manage _Declarative Pipelines_. These directives defi

### Parallel job processing

Jenkins can run the `stages` and `steps` in parallel, while {% data variables.product.prodname_actions %} currently only runs jobs in parallel.
{% ifversion actions-nga %}Jenkins can run the `stages` and `steps` in parallel. {% data variables.product.prodname_actions %} runs jobs in parallel and can also run steps concurrently within a job using step-level syntax. For more information, see [AUTOTITLE](/actions/reference/workflows-and-actions/workflow-syntax#jobsjob_idstepsbackground).{% else %}Jenkins can run the `stages` and `steps` in parallel, while {% data variables.product.prodname_actions %} currently only runs jobs in parallel.{% endif %}

| Jenkins Parallel | {% data variables.product.prodname_actions %} |
| ------------- | ------------- |
Expand Down
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,7 @@
"prettier-check": "prettier -c \"**/*.{ts,tsx,scss,yml,yaml}\"",
"prevent-pushes-to-main": "tsx src/workflows/prevent-pushes-to-main.ts",
"purge-fastly": "tsx src/workflows/purge-fastly.ts",
"purge-fastly-changed-content": "tsx src/workflows/purge-fastly-changed-content.ts",
"readability-report": "tsx src/workflows/experimental/readability-report.ts",
"ready-for-docs-review": "tsx src/workflows/ready-for-docs-review.ts",
"release-banner": "tsx src/ghes-releases/scripts/release-banner.ts",
Expand Down
78 changes: 78 additions & 0 deletions src/article-api/tests/secret-scanning-transformer.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
import { describe, expect, test, vi, beforeEach } from 'vitest'

import { SecretScanningTransformer } from '@/article-api/transformers/secret-scanning-transformer'
import shortVersionsMiddleware from '@/versions/middleware/short-versions'
import { allVersions } from '@/versions/lib/all-versions'
import enterpriseServerReleases from '@/versions/lib/enterprise-server-releases'
import { getSecretScanningData } from '@/secret-scanning/lib/get-secret-scanning-data'
import type { Context, ExtendedRequest, Page, SecretScanningData } from '@/types'

vi.mock('@/secret-scanning/lib/get-secret-scanning-data')
vi.mock('@/article-api/lib/load-template', () => ({
loadTemplate: () => '{{ content }}',
}))

const ghesConditional = '{% ifversion ghes %}false{% else %}true{% endif %}'

const makeEntry = (): SecretScanningData =>
({
provider: 'Example',
supportedSecret: 'Example Token',
secretType: 'example_token',
versions: {},
isPublic: true,
isPrivateWithGhas: true,
hasPushProtection: true,
hasValidityCheck: ghesConditional,
hasExtendedMetadata: ghesConditional,
base64Supported: false,
isduplicate: false,
}) as SecretScanningData

const stubPage = {
autogenerated: 'secret-scanning',
title: 'Test',
intro: '',
render: vi.fn().mockResolvedValue(''),
renderProp: vi.fn().mockResolvedValue(''),
} as unknown as Page

const buildContext = async (currentVersion: string): Promise<Context> => {
const req = { language: 'en', query: {} } as ExtendedRequest
req.context = { currentVersion, allVersions, enterpriseServerReleases } as Context
req.context.currentVersionObj = allVersions[currentVersion]
await shortVersionsMiddleware(req, null, () => {})
return req.context
}

describe('SecretScanningTransformer Liquid evaluation', () => {
const transformer = new SecretScanningTransformer()

beforeEach(() => {
vi.clearAllMocks()
})

const oldestGhes = enterpriseServerReleases.oldestSupported

test('resolves GHES conditionals to false on enterprise-server', async () => {
vi.mocked(getSecretScanningData).mockResolvedValue([makeEntry()])
const context = await buildContext(`enterprise-server@${oldestGhes}`)

await transformer.transform(stubPage, '/test', context)

expect(context.secretScanningData).toBeDefined()
expect(context.secretScanningData![0].hasValidityCheck).toBe(false)
expect(context.secretScanningData![0].hasExtendedMetadata).toBe(false)
})

test('resolves GHES conditionals to true on free-pro-team', async () => {
vi.mocked(getSecretScanningData).mockResolvedValue([makeEntry()])
const context = await buildContext('free-pro-team@latest')

await transformer.transform(stubPage, '/test', context)

expect(context.secretScanningData).toBeDefined()
expect(context.secretScanningData![0].hasValidityCheck).toBe(true)
expect(context.secretScanningData![0].hasExtendedMetadata).toBe(true)
})
})
12 changes: 11 additions & 1 deletion src/article-api/transformers/secret-scanning-transformer.ts
Original file line number Diff line number Diff line change
Expand Up @@ -40,14 +40,24 @@ export class SecretScanningTransformer implements PageTransformer {

// Process Liquid in values
for (const entry of data) {
// Only process Liquid for the hasValidityCheck field, as in the middleware
// Process Liquid for the hasValidityCheck field, as in the middleware
if (typeof entry.hasValidityCheck === 'string' && entry.hasValidityCheck.includes('{%')) {
// Render Liquid and parse as YAML to get correct boolean type
entry.hasValidityCheck = load(
await liquid.parseAndRender(entry.hasValidityCheck, context),
) as boolean
}

// Process Liquid for the hasExtendedMetadata field, as in the middleware
if (
typeof entry.hasExtendedMetadata === 'string' &&
entry.hasExtendedMetadata.includes('{%')
) {
entry.hasExtendedMetadata = load(
await liquid.parseAndRender(entry.hasExtendedMetadata, context),
) as boolean
}

if (entry.isduplicate) {
entry.secretType += ' <br/><a href="#token-versions">Token versions</a>'
}
Expand Down
8 changes: 6 additions & 2 deletions src/secret-scanning/middleware/secret-scanning.ts
Original file line number Diff line number Diff line change
Expand Up @@ -48,9 +48,13 @@ export default async function secretScanning(
// to execute that Liquid to get the actual value.
for (const entry of req.context.secretScanningData) {
for (const [key, value] of Object.entries(entry)) {
if (key === 'hasValidityCheck' && typeof value === 'string' && value.includes('{%')) {
if (
(key === 'hasValidityCheck' || key === 'hasExtendedMetadata') &&
typeof value === 'string' &&
value.includes('{%')
) {
const evaluated = yaml.load(await liquid.parseAndRender(value, req.context))
entry[key] = evaluated as string
entry[key] = evaluated as boolean | string
}
}
if (entry.isduplicate) {
Expand Down
Loading
Loading