Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ Check that your project uses one of the following testing frameworks (this list
- qunit
- testcafe
- vitest
- manual (Markdown-based manual tests)
- manual (Markdown-based manual tests) — [see documentation](docs/frameworks/markdown.md)

Add this action to your workflow file `.github/workflow/main.yml` and configure.

Expand Down Expand Up @@ -539,6 +539,8 @@ npx check-tests gauge "tests/**/*.spec"

#### Manual Tests (Markdown)

See [Markdown manual tests documentation](docs/frameworks/markdown.md) for full format reference, `push`/`pull` commands, and available options.

```bash
# Markdown-based manual test documentation
npx check-tests manual "docs/tests/**/*.md"
Expand Down
2 changes: 2 additions & 0 deletions bin/check.js
Original file line number Diff line number Diff line change
Expand Up @@ -209,6 +209,7 @@ program
.option('--dry-run', 'show what files would be created without actually creating them')
.option('--force', 'skip git checks and force pull files')
.option('--export-automated', 'include automated tests to markdown')
.option('--suite-ids <suite-ids>', 'comma-separated list of suite IDs to pull')
.description('Pull test files from Testomat.io')
.action(async opts => {
const Reporter = require('../src/reporter');
Expand All @@ -226,6 +227,7 @@ program
dryRun: opts.dryRun,
force: opts.force,
exportAutomated: opts.exportAutomated,
suiteIds: opts.suiteIds,
});
const files = await pull.pullFiles();

Expand Down
148 changes: 148 additions & 0 deletions docs/frameworks/markdown.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
# Markdown Manual Tests

Testomat.io supports manual tests written in Markdown files. Use the `push` and `pull` commands to sync them with your project.

## File Format

Each test file uses HTML comment blocks for metadata and Markdown headings for suite/test titles.

```markdown
<!-- suite
id: @S12345678
-->

# Suite Title

<!-- test
id: @T12345678
priority: high
type: manual
-->

## Test Title

Test description and steps go here.
```

### Metadata

Metadata is placed inside the HTML comment block before the heading. Available fields:

**Suite metadata:**

| Field | Description |
| ---------- | ------------------------------------------------------- |
| `id` | Suite ID assigned by Testomat.io (e.g. `@S12345678`) |
| `emoji` | Emoji icon for the suite |
| `tags` | Comma-separated tags (not already present in the title) |
| `labels` | Comma-separated labels |
| `assignee` | Email of the assigned user |
| `issues` | Linked issues |

**Test metadata:**

| Field | Description |
| ---------- | ------------------------------------------------------- |
| `id` | Test ID assigned by Testomat.io (e.g. `@T12345678`) |
| `type` | `manual` or `automated` |
| `priority` | `normal`, `high`, or `low` |
| `assignee` | Email of the assigned user |
| `creator` | Email of the test creator |
| `tags` | Comma-separated tags (not already present in the title) |
| `labels` | Comma-separated labels |
| `issues` | Linked issues |
| `shared` | `true` if the test is shared across suites |

> **Note:** Suite-level `issues` (Jira) are inherited by all tests inside that suite on push. Suite-level `assignee` is inherited only by tests that don't have their own `assignee` set.

> **Note:** `type` and `shared` are read-only fields exported from Testomat.io. Changing them locally has no effect on the test in Testomat.io after push.

> **Note:** The `id` field is used to match a local test with an existing test in Testomat.io. If you change the `id`, the test will be treated as a new test on next push and a new record will be created.

### Single-line comments

If no metadata is needed, use the single-line form:

```markdown
<!-- suite -->

# Suite Title

<!-- test -->

## Test Title
```

On first push, the single-line comment is expanded to a full block and the `id` is inserted.

## Commands

### push

Uploads manual tests from Markdown files to Testomat.io and writes back assigned IDs.

```
npx check-tests push [options]
```

| Option | Description |
| ------------------------ | ------------------------------------------------------------------------------- |
| `-d, --dir <dir>` | Directory to scan for markdown files |
| `-f, --files <files...>` | File paths or glob patterns (default: `**/*.test.md`) |
| `--no-empty` | Remove empty suites after import |
| `--keep-structure` | Prefer file structure over Testomat.io structure |
| `--clean-ids` | Remove IDs from test files (requires API key, removes only IDs known to server) |
| `--purge` | Remove all IDs from test files without server verification |
| `--force` | Skip git checks |

**Examples:**

```bash
# Push all *.test.md files from current directory
TESTOMATIO=api_key npx check-tests push

# Push from a specific directory
TESTOMATIO=api_key npx check-tests push -d ./tests/manual

# Push specific files
TESTOMATIO=api_key npx check-tests push --files "docs/**/*.md"
```

### pull

Downloads Markdown test files from Testomat.io to the local filesystem.

```
npx check-tests pull [options]
```

| Option | Description |
| -------------------- | ----------------------------------------------------------------- |
| `-d, --dir <dir>` | Target directory (default: `.`) |
| `--suite-ids <ids>` | Comma-separated suite IDs to pull (e.g. `@S12345678, @S456r4342`) |
| `--export-automated` | Include automated tests in exported Markdown |
| `--dry-run` | Preview files that would be created without writing them |
| `--force` | Skip git working tree checks |

**Examples:**

```bash
# Pull all manual test files
TESTOMATIO=api_key npx check-tests pull -d

# Pull into specific directory
TESTOMATIO=api_key npx check-tests pull -d ./tests/manual

# Pull specific suites only
TESTOMATIO=api_key npx check-tests pull --suite-ids "@S12345678,@S87654321"

# Preview what would be pulled
TESTOMATIO=api_key npx check-tests pull --dry-run
```

## Environment Variables

| Variable | Description |
| ---------------- | ----------------------------------------------------------- |
| `TESTOMATIO` | API key (required) |
| `TESTOMATIO_URL` | Custom Testomat.io URL (default: `https://app.testomat.io`) |
3 changes: 2 additions & 1 deletion src/pull.js
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ class Pull {
this.dryRun = options.dryRun || false;
this.force = options.force || false;
this.exportAutomated = options.exportAutomated || false;
this.suiteIds = options.suiteIds || null;
}

async pullFiles() {
Expand All @@ -21,7 +22,7 @@ class Pull {
}

try {
const data = await this.reporter.getFilesFromServer(this.exportAutomated);
const data = await this.reporter.getFilesFromServer(this.exportAutomated, this.suiteIds);

if (!data.files) {
console.log('No files received from server');
Expand Down
7 changes: 5 additions & 2 deletions src/reporter.js
Original file line number Diff line number Diff line change
Expand Up @@ -38,11 +38,14 @@ class Reporter {
}
}

getFilesFromServer(exportAutomated) {
getFilesFromServer(exportAutomated, suiteIds) {
return new Promise((res, rej) => {
debug('Getting files from Testomat.io...');
const suiteIdsParam = suiteIds ? `&suite_ids=${encodeURIComponent(suiteIds)}` : '';
const req = request(
`${URL.trim()}/api/test_data?with_files=true&api_key=${this.apiKey}&export_automated=${exportAutomated}`,
`${URL.trim()}/api/test_data?with_files=true&api_key=${
this.apiKey
}&export_automated=${exportAutomated}${suiteIdsParam}`,
{ method: 'GET' },
resp => {
// The whole response has been received. Print out the result.
Expand Down
Loading