Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion docs/core/testing/microsoft-testing-platform-retry.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: Microsoft.Testing.Platform (MTP) retry
description: Learn about retrying failed tests with MTP.
author: evangelink
ms.author: amauryleve
ms.date: 06/01/2026
ms.date: 08/06/2026
ai-usage: ai-assisted
---

Expand Down Expand Up @@ -42,6 +42,10 @@ This extension is intended for integration tests where the test depends heavily

Both threshold options (`--retry-failed-tests-max-percentage` and `--retry-failed-tests-max-tests`) require `--retry-failed-tests` to also be set.

Starting with MTP 2.4.0, the retry summary reports separate `flaky` and `retried` counts and lists recovered test names under **Flaky tests**. To avoid misleading totals, MTP doesn't print a full-suite summary for filtered retry attempts after the first attempt. Use the terminal reporter's [`--show-flaky-tests`](microsoft-testing-platform-terminal-output.md#options) option to show or hide flaky details.

Each retry attempt writes its report artifacts under the `Retries` directory. For JUnit, the top-level report represents only the final filtered retry attempt, not the original full suite; collect the earlier attempt reports from `Retries` when your CI requires complete coverage. This layout applies to `--retry-failed-tests`. MSTest's in-process `[Retry]` instead collapses superseded attempts into one final JUnit or TRX result per test.

### Examples

Retry failed tests up to 3 times:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: Microsoft.Testing.Platform (MTP) terminal output
description: Learn about the built-in terminal test reporter in MTP, including output modes, ANSI support, and progress indicators.
author: evangelink
ms.author: amauryleve
ms.date: 07/17/2026
ms.date: 08/06/2026
ai-usage: ai-assisted
---

Expand Down Expand Up @@ -60,6 +60,7 @@ If your code must write directly to the console and you need that output to rema
| `--output` | — | Specifies the output verbosity when reporting tests. Valid values are `Normal` and `Detailed`. Default is `Normal`. |
| `--show-stdout` | 2.2.1 | Determines when to show captured standard output of a test. Valid values are `All`, `Failed`, and `None`. Default is `All`. |
| `--show-stderr` | 2.2.1 | Determines when to show captured error output of a test. Valid values are `All`, `Failed`, and `None`. Default is `All`. |
| `--show-flaky-tests` | 2.4.0 | Controls the `flaky:` summary and the **Flaky tests** list for tests that pass after a retry. Use `on` or `off`; the default is `on`. Applies to MSTest `[Retry]` and the [retry extension](microsoft-testing-platform-retry.md). |

> [!NOTE]
> A dash (—) in the **MTP version** column marks core options that aren't tied to a specific version because they've been available since the platform's initial releases.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: Microsoft.Testing.Platform (MTP) test reports
description: Learn about the MTP extensions that create test report files (TRX, HTML, JUnit, CTRF, Azure DevOps, GitHub Actions).
author: evangelink
ms.author: amauryleve
ms.date: 06/16/2026
ms.date: 08/06/2026
ai-usage: ai-assisted
---

Expand Down Expand Up @@ -90,6 +90,8 @@ The JUnit report creates a JUnit-compatible XML file for a test session. This ex

> [!NOTE]
> Available in MTP starting with version 2.3.0. This extension is experimental, and its options and output format might change in a future version.
>
> Starting with MSTest.Sdk 4.3, enable this extension with `<EnableMicrosoftTestingExtensionsJUnitReport>true</EnableMicrosoftTestingExtensionsJUnitReport>`. The extension isn't part of the `Default` or `AllMicrosoft` MSTest.Sdk profiles.

### Manual registration

Expand Down Expand Up @@ -126,6 +128,8 @@ builder.AddCtrfReportProvider();
| `--report-ctrf` | Generates the CTRF JSON report. |
| `--report-ctrf-filename` | The name of the generated CTRF JSON report. The value must end with `.json`. The default is `<UserName>_<MachineName>_<assembly>_<tfm>_<timestamp>.ctrf.json`. To customize the name, see [Report file names](#report-file-names). Requires `--report-ctrf`. |

Starting with MSTest 4.4, CTRF results for retried tests include the `retries` and `retryAttempts` fields. When a test passes after an earlier failed attempt, its result also includes `flaky: true`. The terminal summary identifies flaky and retried tests. TRX and JUnit reports keep one final result per test instead of recording every attempt.

## Azure DevOps reports

Azure DevOps report plugin enhances test running for developers that host their code on GitHub, but build on Azure DevOps build agents. It adds additional information to failures to show failure directly in GitHub PR.
Expand Down
7 changes: 5 additions & 2 deletions docs/core/testing/mstest-analyzers/mstest0024.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: "MSTEST0024: Do not store TestContext in a static member"
description: "Learn about code analysis rule MSTEST0024: Do not store TestContext in a static member"
ms.date: 03/19/2024
ms.date: 08/06/2026
f1_keywords:
- MSTEST0024
- DoNotStoreStaticTestContextAnalyzer
Expand All @@ -10,6 +10,7 @@ helpviewer_keywords:
- MSTEST0024
author: Evangelink
ms.author: amauryleve
ai-usage: ai-assisted
---
# MSTEST0024: Do not store TestContext in a static member

Expand All @@ -28,9 +29,11 @@ ms.author: amauryleve

This rule raises a diagnostic when an assignment to a `static` member of a `TestContext` parameter is done.

Starting with MSTest 4.4, the rule also detects coalescing assignments, such as `s_testContext ??= testContext`, and deconstruction assignments that store `TestContext` in a static member.

## Rule description

The `TestContext` parameter passed to each initialize method (`[AssemblyInitialize]` or `[ClassInitialize]`) is specific to the current context and is not updated on each test execution. Storing, for reuse, this `TextContext` object will most of the time lead to issues.
The `TestContext` parameter passed to each initialize method (`[AssemblyInitialize]` or `[ClassInitialize]`) is specific to the current context and is not updated on each test execution. Storing, for reuse, this `TestContext` object will most of the time lead to issues.

## How to fix violations

Expand Down
7 changes: 5 additions & 2 deletions docs/core/testing/mstest-analyzers/mstest0041.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: "MSTEST0041: Use 'ConditionBaseAttribute' on test classes"
description: "Learn about code analysis rule MSTEST0041: Use 'ConditionBaseAttribute' on test classes"
ms.date: 02/13/2025
ms.date: 08/06/2026
f1_keywords:
- MSTEST0041
- UseConditionBaseWithTestClassAnalyzer
Expand All @@ -10,6 +10,7 @@ helpviewer_keywords:
- MSTEST0041
author: Youssef1313
ms.author: ygerges
ai-usage: ai-assisted
---
# MSTEST0041: Use 'ConditionBaseAttribute' on test classes

Expand All @@ -22,7 +23,7 @@ ms.author: ygerges
| **Enabled by default** | Yes |
| **Default severity** | Warning |
| **Introduced in version** | 3.8.0 |
| **Is there a code fix** | No |
| **Is there a code fix** | Yes, starting with MSTest 4.4 |

## Cause

Expand All @@ -36,6 +37,8 @@ An attribute that derives from <xref:Microsoft.VisualStudio.TestTools.UnitTestin

Depending on the intent, either add the missing <xref:Microsoft.VisualStudio.TestTools.UnitTesting.TestClassAttribute> or remove the attribute that derives from <xref:Microsoft.VisualStudio.TestTools.UnitTesting.ConditionBaseAttribute>.

Starting with MSTest 4.4, the code fix adds `[TestClass]` to the affected type.

## When to suppress warnings

Do not suppress a warning from this rule.
5 changes: 3 additions & 2 deletions docs/core/testing/mstest-analyzers/mstest0050.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: "MSTEST0050: Global test fixture should be valid"
description: "Learn about code analysis rule MSTEST0050: Global test fixture should be valid"
ms.date: 07/29/2025
ms.date: 08/06/2026
f1_keywords:
- MSTEST0050
- GlobalTestFixtureShouldBeValidAnalyzer
Expand All @@ -10,6 +10,7 @@ helpviewer_keywords:
- MSTEST0050
author: Evangelink
ms.author: amauryleve
ai-usage: ai-assisted
---
# MSTEST0050: Global test fixture should be valid

Expand All @@ -20,7 +21,7 @@ ms.author: amauryleve
| **Category** | Usage |
| **Fix is breaking or non-breaking** | Non-breaking |
| **Enabled by default** | Yes |
| **Default severity** | Error |
| **Default severity** | Warning in MSTest 4.3 and later; error in earlier versions |
| **Introduced in version** | 3.10.0 |
| **Is there a code fix** | No |

Expand Down
4 changes: 3 additions & 1 deletion docs/core/testing/mstest-analyzers/mstest0065.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: "MSTEST0065: Avoid Assert.AreEqual on collection types"
description: "Learn about code analysis rule MSTEST0065: Avoid Assert.AreEqual on collection types"
ms.date: 06/04/2026
ms.date: 08/06/2026
f1_keywords:
- MSTEST0065
- AvoidAssertAreEqualOnCollectionsAnalyzer
Expand Down Expand Up @@ -29,6 +29,8 @@ dev_langs:

> [!NOTE]
> This rule is available starting with MSTest 4.3.
>
> Starting with MSTest 4.3.3, the rule doesn't report collection types that declare their own equality behavior.

## Cause

Expand Down
85 changes: 85 additions & 0 deletions docs/core/testing/mstest-analyzers/mstest0072.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
---
title: "MSTEST0072: '[AssemblyFixtureProvider]' isn't supported with ahead-of-time compilation"
description: "Learn about code analysis rule MSTEST0072: '[AssemblyFixtureProvider]' isn't supported with ahead-of-time compilation"
ms.date: 08/06/2026
f1_keywords:
- MSTEST0072
- AssemblyFixtureProviderNotSupportedWithNativeAotAnalyzer
helpviewer_keywords:
- AssemblyFixtureProviderNotSupportedWithNativeAotAnalyzer
- MSTEST0072
author: evangelink
ms.author: amauryleve
ai-usage: ai-assisted
dev_langs:
- CSharp
---
# MSTEST0072: '\[AssemblyFixtureProvider]' isn't supported with ahead-of-time compilation

| Property | Value |
|-------------------------------------|----------------------------------------------------|
| **Rule ID** | MSTEST0072 |
| **Title** | '\[AssemblyFixtureProvider]' isn't supported with ahead-of-time compilation |
| **Category** | Usage |
| **Fix is breaking or non-breaking** | Non-breaking |
| **Enabled by default** | Yes |
| **Default severity** | Warning |
| **Introduced in version** | 4.4.0 (preview) |
| **Is there a code fix** | No |

> [!IMPORTANT]
> This analyzer is planned for MSTest 4.4 and is available only in preview builds until MSTest 4.4.0 is released.

## Cause

A project applies `[AssemblyFixtureProvider]`, either directly or through a referenced library, while publishing with Native AOT (`PublishAot`) or Blazor WebAssembly AOT (`RunAOTCompilation`).

## Rule description

`[AssemblyFixtureProvider]` discovery walks the runtime assembly reference graph, which requires the runtime to generate dynamic code. Ahead-of-time compilation flavors such as Native AOT and Blazor WebAssembly AOT can't generate dynamic code, so the runtime silently skips discovery, and the fixture's `[AssemblyInitialize]`/`[AssemblyCleanup]` methods never run.

```csharp
// Violation: ignored at run time under Native AOT / Blazor WebAssembly AOT.
[assembly: AssemblyFixtureProvider(typeof(SharedFixtures))]
```

The analyzer reports this diagnostic whether the attribute is applied in the current compilation or on a referenced assembly, because either way the consuming Native AOT test project silently loses its assembly fixtures.

## How to fix violations

Declare the `[AssemblyInitialize]` and `[AssemblyCleanup]` methods directly in the test assembly instead of relying on a shared `[AssemblyFixtureProvider]` library.

```csharp
public static class Fixtures
{
[AssemblyInitialize]
public static void Init(TestContext context) { }
}
```

## When to suppress warnings

Don't suppress warnings from this rule for a project that actually publishes with Native AOT or Blazor WebAssembly AOT, because the fixture methods won't run and any state they set up won't exist for your tests. Suppressing is reasonable only if the AOT-published configuration doesn't run the affected tests.

## Suppress a warning

If you just want to suppress a single violation, add preprocessor directives to your source file to disable and then re-enable the rule.

```csharp
#pragma warning disable MSTEST0072
// The code that's violating the rule is on this line.
#pragma warning restore MSTEST0072
```

To disable the rule for a file, folder, or project, set its severity to `none` in the [configuration file](../../../fundamentals/code-analysis/configuration-files.md).

```ini
[*.{cs,vb}]
dotnet_diagnostic.MSTEST0072.severity = none
```

For more information, see [How to suppress code analysis warnings](../../../fundamentals/code-analysis/suppress-warnings.md).

## See also

- [Shared assembly fixtures with AssemblyFixtureProvider](../unit-testing-mstest-writing-tests-lifecycle.md#shared-assembly-fixtures-with-assemblyfixtureprovider)
84 changes: 84 additions & 0 deletions docs/core/testing/mstest-analyzers/mstest0073.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
---
title: "MSTEST0073: Prefer a constant for the '[ResourceLock]' resource key"
description: "Learn about code analysis rule MSTEST0073: Prefer a constant for the '[ResourceLock]' resource key"
ms.date: 08/06/2026
f1_keywords:
- MSTEST0073
- PreferConstantForResourceLockAnalyzer
helpviewer_keywords:
- PreferConstantForResourceLockAnalyzer
- MSTEST0073
author: evangelink
ms.author: amauryleve
ai-usage: ai-assisted
dev_langs:
- CSharp
---
# MSTEST0073: Prefer a constant for the '\[ResourceLock]' resource key

| Property | Value |
|-------------------------------------|----------------------------------------------------|
| **Rule ID** | MSTEST0073 |
| **Title** | Prefer a constant for the '\[ResourceLock]' resource key |
| **Category** | Usage |
| **Fix is breaking or non-breaking** | Non-breaking |
| **Enabled by default** | Yes |
| **Default severity** | Info |
| **Introduced in version** | 4.4.0 (preview) |
| **Is there a code fix** | No |

> [!IMPORTANT]
> `ResourceLockAttribute` is planned for MSTest 4.4 and is available only in preview builds until MSTest 4.4.0 is released.

## Cause

A `[ResourceLock]` attribute passes its resource key as a bare string literal instead of referencing a shared constant.

## Rule description

`[ResourceLock]` matches tests by exact, case-sensitive string equality of the resource key. A bare string literal fails open: a typo produces a different key, so the conflicting tests are no longer serialized and race silently instead of failing with a build error. Referencing a shared constant, such as a `WellKnownResources` member or your own `const`, makes typos a compile error and lets the compiler enforce that every test contending on the same resource uses the same key.

```csharp
[ResourceLock("database")] // Violation: bare string literal.
[TestMethod]
public void ReadsSharedSchema() { }
```

## How to fix violations

Reference a `WellKnownResources` member for process-global state, or declare and reference your own `const string`.

```csharp
private const string Database = "database";

[ResourceLock(Database)]
[TestMethod]
public void ReadsSharedSchema() { }
```

## When to suppress warnings

It's safe to suppress this warning if you intentionally use a literal resource key and are confident no other test in the assembly needs to coordinate on the same resource.

## Suppress a warning

If you just want to suppress a single violation, add preprocessor directives to your source file to disable and then re-enable the rule.

```csharp
#pragma warning disable MSTEST0073
// The code that's violating the rule is on this line.
#pragma warning restore MSTEST0073
```

To disable the rule for a file, folder, or project, set its severity to `none` in the [configuration file](../../../fundamentals/code-analysis/configuration-files.md).

```ini
[*.{cs,vb}]
dotnet_diagnostic.MSTEST0073.severity = none
```

For more information, see [How to suppress code analysis warnings](../../../fundamentals/code-analysis/suppress-warnings.md).

## See also

- [ResourceLockAttribute](../unit-testing-mstest-writing-tests-controlling-execution.md#resourcelockattribute)
Loading
Loading