Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 18 additions & 12 deletions flaky-tests/detection/flag-as-flaky.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: "Flag as Flaky"
description: "Manually mark a test as flaky from the test detail page"
og:title: "Flagging tests as flaky in Trunk"
---
Manually mark a test as flaky when you know it's unreliable but automated monitors haven't detected it yet — or when you want to override the system's assessment.
Manually mark a test as flaky when you know it's unreliable but automated monitors haven't detected it yet — or when you want to keep a test classified as flaky while monitors consider it healthy.

## When to Use It

Expand All @@ -22,10 +22,14 @@ Manually mark a test as flaky when you know it's unreliable but automated monito

Once flagged:

- The test is immediately marked as **flaky**, regardless of what automated monitors report.
- The test is immediately marked as **flaky**, even if no automated monitor has detected it.
- An amber banner appears below the header showing who flagged it, when, and the reason (if provided).
- The flag is additive — if automated monitors later detect the test as flaky too, both signals coexist.

<Info>
A manual flag does not override a **broken** status. Broken always takes precedence over flaky, so if an active monitor classifies the test as broken, the test shows as broken until that monitor resolves or is [muted](./index#muting-monitors). While a test is broken, the **Flag as Flaky** button is disabled — if you believe the broken classification is incorrect, [mute the monitor](./index#muting-monitors) or update its configuration instead. You can still remove an existing flag while the test is broken.
</Info>

### Removing the Flag

1. On the test detail page, find the amber "Manually flagged as flaky" banner.
Expand All @@ -39,16 +43,18 @@ After removing:

## Relationship to Monitors

The "Flag as Flaky" action is independent of automated monitors (threshold-based, pass-on-retry). It does not appear in the Monitors tab.

| Scenario | Test status |
| ----------------------------------- | ------------------------ |
| No monitors active, no flag | Healthy |
| Monitors active, no flag | Flaky (detected) |
| No monitors active, flag set | Flaky (manually flagged) |
| Monitors active, flag set | Flaky (both) |
| Flag removed, monitors still active | Flaky (detected) |
| Flag removed, monitors inactive | Healthy |
The "Flag as Flaky" action is separate from automated monitors (threshold-based, pass-on-retry) and does not appear in the Monitors tab, but it participates in the same [status resolution](./index#how-monitors-work) as health classification monitors: the most severe status wins, and broken outranks flaky. If a test's status looks wrong — for example, it shows as broken when you believe it's merely flaky — [mute](./index#muting-monitors) or reconfigure the monitor responsible rather than reaching for the flag.

| Scenario | Test status |
| --------------------------------------------- | -------------------------------- |
| No monitors active, no flag | Healthy |
| Flaky monitors active, no flag | Flaky (detected) |
| No monitors active, flag set | Flaky (manually flagged) |
| Flaky monitors active, flag set | Flaky (both) |
| Broken monitor active, flag set | Broken (broken takes precedence) |
| Broken monitor resolves or is muted, flag set | Flaky (manually flagged) |
| Flag removed, flaky monitors still active | Flaky (detected) |
| Flag removed, monitors inactive | Healthy |

## Flag History

Expand Down
1 change: 1 addition & 0 deletions flaky-tests/detection/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,7 @@ You can also unmute a monitor early from the test case view.
| --- | --- |
| Fix is in progress and you want to suppress noise temporarily | **Mute** the monitor for a few days |
| Test is flaky but no automated monitor has caught it | Use [**Flag as Flaky**](./flag-as-flaky) to mark it as flaky |
| A test shows as **Broken** (or Flaky) and you believe the classification is wrong | **Mute** the monitor that's driving the status, or update its configuration (thresholds, branch scope) so it stops misfiring — a [Flag as Flaky](./flag-as-flaky) can't override a broken status |
| You want to stop a monitor from evaluating a test permanently | Adjust the monitor's branch scope or thresholds instead |
| You want to suppress all flaky signals for a test | Mute each active monitor individually, or address the root cause |

Expand Down
2 changes: 1 addition & 1 deletion flaky-tests/management/managing-detected-flaky-tests.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ Muting suppresses the monitor's contribution to the test's status. If the muted

### Step 5: Flag flaky tests

If automated detection hasn't caught a test you know is flaky, you can manually [flag it as flaky](../detection/flag-as-flaky) from the test detail page. Flagged tests are treated as flaky regardless of automated detection state, and the flag can be removed at any time.
If automated detection hasn't caught a test you know is flaky, you can manually [flag it as flaky](../detection/flag-as-flaky) from the test detail page. Flagged tests are treated as flaky even when no monitor detects them, and the flag can be removed at any time. Note that active broken monitors will take precedence over the manual flaky flag.

### Step 6: Quarantine flaky tests

Expand Down