Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/howto/set-up-continuous-integration-for-a-charm.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ If your charm is a Kubernetes charm, add the following job to `.github/workflows
if: ${{ !cancelled() }}
uses: actions/upload-artifact@v7
with:
name: juju-dump-logs
name: integration-test-logs
path: logs
```

Expand Down
145 changes: 139 additions & 6 deletions docs/howto/write-integration-tests-for-a-charm.md
Original file line number Diff line number Diff line change
Expand Up @@ -460,22 +460,155 @@ tox -e integration -- tests/integration/test_charm.py -k "not test_one"
> - [`pytest | How to invoke pytest`](https://docs.pytest.org/en/7.1.x/how-to/usage.html)
> - [](#validate-your-charm-with-every-change)

(write-integration-tests-for-a-charm-configure-jubilant-logs)=
## Configure Jubilant logs

(write-integration-tests-for-a-charm-view-juju-logs)=
## View Juju logs
Jubilant emits logs during the integration tests. These logs are captured and displayed by pytest. You can configure the logging behaviour in the `[tool.pytest.ini_options]` table in `pyproject.toml`.

### Logging modes

**Brief**

```toml
[tool.pytest.ini_options]
...
log_cli = true
log_cli_level = "INFO"
log_cli_format = "%(levelname)s %(name)s %(message)s"
```

While the test is running, logs with level `INFO` and above are emitted live to the console.

**Verbose**

This mode shows whatever is in Brief, and the verbose `DEBUG` level logs. Lowering the log level is all it needs to switch from Brief to Verbose mode.

```toml
[tool.pytest.ini_options]
...
log_cli = true
log_cli_level = "DEBUG"
log_cli_format = "%(asctime)s %(levelname)s %(name)s %(message)s"
log_cli_date_format = "%Y-%m-%dT%H:%M:%SZ"
```

We define the timestamps format with the `log_cli_date_format` key, following ISO 8601.

> See more:
> - [`datetime | strftime() and strptime() format codes`](https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes)

You can remove the timestamps without changing the mode, but we recommend leaving it in.

**Error**

Jubilant emits logs at `ERROR` level when an application or unit transition into `error` during `juju.wait()`. This mode is useful if you only care if any charm turns into error status.

```toml
[tool.pytest.ini_options]
...
log_cli = true
log_cli_level = "ERROR"
log_cli_format = "%(levelname)s %(name)s %(message)s"
```

These options can also be configured from the command line.

> See more:
> - [`pytest | How to manage logging`](https://docs.pytest.org/en/stable/how-to/logging.html)

(write-integration-tests-log-to-a-file)=
### Log to a file

This is an ideal configuration for long-running integration tests (for example, in CI). It uses Brief mode for logging to the console, and Verbose mode for logging to a local file.

```toml
[tool.pytest.ini_options]
...

log_level = "INFO"

log_cli = true
log_cli_level = "INFO"
log_cli_format = "%(levelname)s %(name)s %(message)s"

log_file = "logs/verbose.log"
log_file_level = "DEBUG"
log_file_format = "%(asctime)s %(levelname)s %(name)s %(message)s"
log_file_date_format = "%Y-%m-%dT%H:%M:%SZ"
```

When a test item fails, pytest prints all logs captured during the test to the terminal:

```
------- Captured log call -------
...
```

`log_level = "INFO"` is intentional to keep the `Captured log call` consistent with Brief mode. If you want Verbose mode in `Captured log call`, you can set `log_level` to `DEBUG`.

The local file where Verbose mode logs land is `logs/verbose.log`. It contains logs from one pytest session. If you want to run multiple test suites, and store Verbose logs in separated files. You can override `log_file` for each pytest invocation:

```
pytest --log-file "run1.log" ...
pytest --log-file "run2.log" ...
```

Alternatively, you can set `log-file-mode` to `append`. Pytest will put verbose logs from all invocations into one file:

If any tests fail, `pytest-jubilant` automatically prints the last 1000 lines of `juju debug-log` to stderr. You can also save the complete logs to disk with the `--juju-dump-logs` option.
```toml
[tool.pytest.ini_options]
log-file-mode = "a"
...
```

These options can also be configured from the command line.

> See more:
> - [`pytest | How to manage logging`](https://docs.pytest.org/en/stable/how-to/logging.html)

Use `--juju-dump-logs` in CI with `actions/upload-artifact` to make debug logs available as build artifacts:
Use this mode in CI with `actions/upload-artifact` to make `logs/verbose.log` available as build artifacts:

```yaml
# In your integration test job
- run: tox -e integration -- --juju-dump-logs logs
- run: tox -e integration
- name: Upload logs
if: ${{ !cancelled() }}
uses: actions/upload-artifact@v7
with:
name: integration-test-logs
path: logs
```

### Default behaviour

If no logging configuration is set by `tool.pytest.ini_options` or pytest CLI arguments, pytest captures all log messages at `WARNING` level or above and doesn't log to a file. You can still see messages from:

- pytest. For example: `tests/integration/test_charm.py::test_deploy PASSED`.
- other modules if they are at `WARNING` or above.
- `pytest-jubilant`. For example: `Models were torn down...`.

(write-integration-tests-for-a-charm-view-juju-logs)=
## View `juju debug-log` logs

If any tests fail, `pytest-jubilant` automatically logs the last 1000 lines of `juju debug-log` at `DEBUG` level. Log messages from `pytest-jubilant` are handled depending on how you configure pytest (see [](write-integration-tests-for-a-charm-configure-jubilant-logs)):

- For Brief mode and Error mode: They don't appear anywhere.
- For Verbose mode: They appear in the terminal.
- For Log to a file: They appear in `log_file`, and don't appear in the terminal.

You can also save the complete `juju debug-log` to disk with the `--juju-dump-logs` option.

Use `--juju-dump-logs` in CI with `actions/upload-artifact` to make `juju debug-log` files available as build artifacts. We can modify the GitHub Actions from [](write-integration-tests-log-to-a-file):

```diff
# In your integration test job
- - run: tox -e integration
+ - run: tox -e integration -- --juju-dump-logs logs
- name: Upload logs
if: ${{ !cancelled() }}
uses: actions/upload-artifact@v7
with:
name: juju-dump-logs
name: integration-test-logs

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I rename to keep it general, as this artifact now contain both the log file generated by pytest, and the files generated by pytest-jubilant.

path: logs
```

Expand Down
Loading