From 3287194e50df524ddb0039fec90cb5d1ec55d061 Mon Sep 17 00:00:00 2001 From: Trong Nhan Mai Date: Mon, 6 Jul 2026 10:48:39 +1000 Subject: [PATCH 1/6] docs: add howto configure jubilant logs --- .../write-integration-tests-for-a-charm.md | 141 +++++++++++++++++- 1 file changed, 135 insertions(+), 6 deletions(-) diff --git a/docs/howto/write-integration-tests-for-a-charm.md b/docs/howto/write-integration-tests-for-a-charm.md index 1be0b41cb..3d320fd64 100644 --- a/docs/howto/write-integration-tests-for-a-charm.md +++ b/docs/howto/write-integration-tests-for-a-charm.md @@ -460,25 +460,154 @@ tox -e integration -- tests/integration/test_charm.py -k "not test_one" > - [`pytest | How to invoke pytest`](https://docs.pytest.org/en/7.1.x/how-to/usage.html) > - [](#validate-your-charm-with-every-change) +(write-integration-tests-for-a-charm-configuring-jubilant-logs)= +## Configuring Jubilant logs -(write-integration-tests-for-a-charm-view-juju-logs)= -## View Juju logs +Jubilant emits logs during the integration tests. These logs are handled by pytest if you are using `pytest-jubilant`. We can configure this behavior through pytest's ini options in `pyproject.toml`. + +If you initialised the charm with `charmcraft init`, the default configuration should already be in the `[tool.pytest.ini_options]` table. For example: + +```toml +[tool.pytest.ini_options] +... +``` + +You can freely configure this to your needs. + +> See more: +> - [`pytest | How to manage loggings`](https://docs.pytest.org/en/stable/reference/reference.html#logging) + +This section covers the logging modes that we suggest. + +### Brief mode + +``` +[tool.pytest.ini_options] +... +log_cli = true +log_cli_level = "INFO" +log_cli_format = "%(levelname)s %(name)s %(message)s" +``` + +While the test is running, logs with level `INFO` and above are emitted live to the console. Each log message will contain the log level, the logger's name and the message. + +### Verbose mode -If any tests fail, `pytest-jubilant` automatically prints the last 1000 lines of `juju debug-log` to stderr. You can also save the complete logs to disk with the `--juju-dump-logs` option. +This mode shows whatever is in Brief, and the verbose `DEBUG` level logs. Lowering the log level is all it needs to switch from Brief to Verbose mode. -Use `--juju-dump-logs` in CI with `actions/upload-artifact` to make debug logs available as build artifacts: +```toml +[tool.pytest.ini_options] +... +log_cli = true +log_cli_level = "DEBUG" +log_cli_format = "%(asctime)s %(levelname)s %(name)s %(message)s" +log_cli_date_format = "%Y-%m-%dT%H:%M:%SZ" +``` + +As we show the timestamps, we need to define its format with the `log_cli_date_format` key. + +> See more: +> - [`datetime | strftime() and strptime() format codes`](https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes) + +You can remove the timestamps without changing the mode, but we recommend leaving it in. + +### Error mode + +Jubilant emits logs at `ERROR` level when an application or unit transition into `error` during `juju.wait()`. This mode is useful if you only care if any charm turns into error status. + +```toml +[tool.pytest.ini_options] +... +log_cli = true +log_cli_level = "ERROR" +log_cli_format = "%(levelname)s %(name)s %(message)s" +``` +(write-integration-tests-log-to-a-file-mode)= +### Log to a file + +This is an ideal configuration for long-running integration tests (for example, in CI). It uses Brief mode for logging to the console, and Verbose mode for logging to a local file. + +```toml +[tool.pytest.ini_options] +... + +log_level = "INFO" + +log_cli = true +log_cli_level = "INFO" +log_cli_format = "%(levelname)s %(name)s %(message)s" + +log_file = "logs/verbose.log" +log_file_level = "DEBUG" +log_file_format = "%(asctime)s %(levelname)s %(name)s %(message)s" +log_file_date_format = "%Y-%m-%dT%H:%M:%SZ" +``` + +When a test item fails, pytest prints all logs captured during the test to the terminal: + +``` +------- Captured log call ------- +... +``` + +`log_level = "INFO"` is intentional to keep the `Captured log call` consistent with Brief mode. If you still want Verbose mode in `Captured log call`, `log_level` can be removed. + +The local file where Verbose mode logs land is `logs/verbose.log`. It contains logs from one pytest session. If you want to run multiple test suites, and store Verbose logs in separated files. You can override `log_file` for each pytest invocation: + +``` +pytest --log-file "run1.log" ... +pytest --log-file "run2.log" ... +``` + +Alternatively, you can set `log-file-mode` to `append`. Pytest will put verbose logs from all invocations into one file: + +``` +[tool.pytest.ini_options] +log-file-mode = "a" +... +``` + +Use this mode in CI with `actions/upload-artifact` to make `logs/verbose.log` available as build artifacts: ```yaml # In your integration test job - - run: tox -e integration -- --juju-dump-logs logs + - run: tox -e integration - name: Upload logs if: ${{ !cancelled() }} uses: actions/upload-artifact@v7 with: - name: juju-dump-logs + name: integration-logs path: logs ``` + +### Behavior when no pytest's logging config is set + +This section covers the behavior when no logging configuration is set, either in config files, or from CLI arguments. In this case, pytest captures all log messages at `WARNING` level or above, and there is no logging to a file. You can still see messages from: + +- `pytest`. For example: `tests/integration/test_charm.py::test_deploy PASSED` +- Logs from other modules if they are at `WARNING` or above. +- Usage hints from pytest-jubilant. For example: `Models were torn down. To keep models available for subsequent test runs or manual debugging, pass the following: +--no-juju-teardown` + +(write-integration-tests-for-a-charm-view-juju-logs)= +## View `juju debug-log` logs + +If any tests fail, `pytest-jubilant` automatically logs the last 1000 lines of `juju debug-log`. Log messages from `pytest-jubilant` are handled depending on how you configure pytest (see [](write-integration-tests-for-a-charm-configuring-jubilant-logs): + +- For Brief mode and Error mode: They don't appear anywhere. +- For Verbose mode: They appear in the terminal. +- For Log to a file: They only land in `log_file`, and don't appear in the terminal. + +You can also save the complete `juju debug-log` to disk with the `--juju-dump-logs` option. + +Use `--juju-dump-logs` in CI with `actions/upload-artifact` to make `juju debug-log` files available as build artifacts. We can modify the Github Actions from [](write-integration-tests-log-to-a-file-mode): + +```yaml +- - run: tox -e integration ++ - run: tox -e integration -- --juju-dump-logs logs +``` + ## Generate crash dumps To generate crash dumps, you need the `juju-crashdump` tool . From 62518ca402da3eb268860f97d6df2d0b6c52d7a7 Mon Sep 17 00:00:00 2001 From: Trong Nhan Mai Date: Mon, 6 Jul 2026 11:01:11 +1000 Subject: [PATCH 2/6] docs: fix spelling errors --- docs/howto/write-integration-tests-for-a-charm.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/docs/howto/write-integration-tests-for-a-charm.md b/docs/howto/write-integration-tests-for-a-charm.md index 3d320fd64..d358a2403 100644 --- a/docs/howto/write-integration-tests-for-a-charm.md +++ b/docs/howto/write-integration-tests-for-a-charm.md @@ -463,7 +463,7 @@ tox -e integration -- tests/integration/test_charm.py -k "not test_one" (write-integration-tests-for-a-charm-configuring-jubilant-logs)= ## Configuring Jubilant logs -Jubilant emits logs during the integration tests. These logs are handled by pytest if you are using `pytest-jubilant`. We can configure this behavior through pytest's ini options in `pyproject.toml`. +Jubilant emits logs during the integration tests. These logs are handled by pytest if you are using `pytest-jubilant`. We can configure this behavior in `pyproject.toml`. If you initialised the charm with `charmcraft init`, the default configuration should already be in the `[tool.pytest.ini_options]` table. For example: @@ -472,12 +472,13 @@ If you initialised the charm with `charmcraft init`, the default configuration s ... ``` -You can freely configure this to your needs. +You can freely configure it to your needs. > See more: > - [`pytest | How to manage loggings`](https://docs.pytest.org/en/stable/reference/reference.html#logging) +> - [`pytest` | pyproject.toml](https://docs.pytest.org/en/stable/reference/customize.html#pyproject-toml) -This section covers the logging modes that we suggest. +The sections below cover the configuration that we suggest. ### Brief mode @@ -601,7 +602,7 @@ If any tests fail, `pytest-jubilant` automatically logs the last 1000 lines of ` You can also save the complete `juju debug-log` to disk with the `--juju-dump-logs` option. -Use `--juju-dump-logs` in CI with `actions/upload-artifact` to make `juju debug-log` files available as build artifacts. We can modify the Github Actions from [](write-integration-tests-log-to-a-file-mode): +Use `--juju-dump-logs` in CI with `actions/upload-artifact` to make `juju debug-log` files available as build artifacts. We can modify the GitHub Actions from [](write-integration-tests-log-to-a-file-mode): ```yaml - - run: tox -e integration From 85e4fec34fcc9ad94421dc62572540f7180bbe2a Mon Sep 17 00:00:00 2001 From: Trong Nhan Mai Date: Mon, 6 Jul 2026 14:25:57 +1000 Subject: [PATCH 3/6] minor adjustments --- ...t-up-continuous-integration-for-a-charm.md | 2 +- .../write-integration-tests-for-a-charm.md | 29 ++++++++++++------- 2 files changed, 19 insertions(+), 12 deletions(-) diff --git a/docs/howto/set-up-continuous-integration-for-a-charm.md b/docs/howto/set-up-continuous-integration-for-a-charm.md index 1bc127c55..c6c72591f 100644 --- a/docs/howto/set-up-continuous-integration-for-a-charm.md +++ b/docs/howto/set-up-continuous-integration-for-a-charm.md @@ -102,7 +102,7 @@ If your charm is a Kubernetes charm, add the following job to `.github/workflows if: ${{ !cancelled() }} uses: actions/upload-artifact@v7 with: - name: juju-dump-logs + name: integration-test-logs path: logs ``` diff --git a/docs/howto/write-integration-tests-for-a-charm.md b/docs/howto/write-integration-tests-for-a-charm.md index d358a2403..ed80e2f02 100644 --- a/docs/howto/write-integration-tests-for-a-charm.md +++ b/docs/howto/write-integration-tests-for-a-charm.md @@ -465,7 +465,7 @@ tox -e integration -- tests/integration/test_charm.py -k "not test_one" Jubilant emits logs during the integration tests. These logs are handled by pytest if you are using `pytest-jubilant`. We can configure this behavior in `pyproject.toml`. -If you initialised the charm with `charmcraft init`, the default configuration should already be in the `[tool.pytest.ini_options]` table. For example: +If you initialised the charm with `charmcraft init`, the `[tool.pytest.ini_options]` table already exist in `pyproject.toml`. For example: ```toml [tool.pytest.ini_options] @@ -490,7 +490,7 @@ log_cli_level = "INFO" log_cli_format = "%(levelname)s %(name)s %(message)s" ``` -While the test is running, logs with level `INFO` and above are emitted live to the console. Each log message will contain the log level, the logger's name and the message. +While the test is running, logs with level `INFO` and above are emitted live to the console. ### Verbose mode @@ -505,7 +505,7 @@ log_cli_format = "%(asctime)s %(levelname)s %(name)s %(message)s" log_cli_date_format = "%Y-%m-%dT%H:%M:%SZ" ``` -As we show the timestamps, we need to define its format with the `log_cli_date_format` key. +We define the timestamps format with the `log_cli_date_format` key, following ISO 8601. > See more: > - [`datetime | strftime() and strptime() format codes`](https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes) @@ -523,6 +523,7 @@ log_cli = true log_cli_level = "ERROR" log_cli_format = "%(levelname)s %(name)s %(message)s" ``` + (write-integration-tests-log-to-a-file-mode)= ### Log to a file @@ -551,7 +552,7 @@ When a test item fails, pytest prints all logs captured during the test to the t ... ``` -`log_level = "INFO"` is intentional to keep the `Captured log call` consistent with Brief mode. If you still want Verbose mode in `Captured log call`, `log_level` can be removed. +`log_level = "INFO"` is intentional to keep the `Captured log call` consistent with Brief mode. If you want Verbose mode in `Captured log call`, you can set `log_level` to `DEBUG`. The local file where Verbose mode logs land is `logs/verbose.log`. It contains logs from one pytest session. If you want to run multiple test suites, and store Verbose logs in separated files. You can override `log_file` for each pytest invocation: @@ -577,36 +578,42 @@ Use this mode in CI with `actions/upload-artifact` to make `logs/verbose.log` av if: ${{ !cancelled() }} uses: actions/upload-artifact@v7 with: - name: integration-logs + name: integration-test-logs path: logs ``` - ### Behavior when no pytest's logging config is set This section covers the behavior when no logging configuration is set, either in config files, or from CLI arguments. In this case, pytest captures all log messages at `WARNING` level or above, and there is no logging to a file. You can still see messages from: - `pytest`. For example: `tests/integration/test_charm.py::test_deploy PASSED` - Logs from other modules if they are at `WARNING` or above. -- Usage hints from pytest-jubilant. For example: `Models were torn down. To keep models available for subsequent test runs or manual debugging, pass the following: +- Usage hints from `pytest-jubilant`. For example: `Models were torn down. To keep models available for subsequent test runs or manual debugging, pass the following: --no-juju-teardown` (write-integration-tests-for-a-charm-view-juju-logs)= ## View `juju debug-log` logs -If any tests fail, `pytest-jubilant` automatically logs the last 1000 lines of `juju debug-log`. Log messages from `pytest-jubilant` are handled depending on how you configure pytest (see [](write-integration-tests-for-a-charm-configuring-jubilant-logs): +If any tests fail, `pytest-jubilant` automatically logs the last 1000 lines of `juju debug-log` at `DEBUG` level. Log messages from `pytest-jubilant` are handled depending on how you configure pytest (see [](write-integration-tests-for-a-charm-configuring-jubilant-logs): -- For Brief mode and Error mode: They don't appear anywhere. +- For Brief mode and Error mode: They don't appear anywhere. - For Verbose mode: They appear in the terminal. -- For Log to a file: They only land in `log_file`, and don't appear in the terminal. +- For Log to a file: They appear in `log_file`, and don't appear in the terminal. You can also save the complete `juju debug-log` to disk with the `--juju-dump-logs` option. Use `--juju-dump-logs` in CI with `actions/upload-artifact` to make `juju debug-log` files available as build artifacts. We can modify the GitHub Actions from [](write-integration-tests-log-to-a-file-mode): -```yaml +```diff + # In your integration test job - - run: tox -e integration + - run: tox -e integration -- --juju-dump-logs logs + - name: Upload logs + if: ${{ !cancelled() }} + uses: actions/upload-artifact@v7 + with: + name: integration-test-logs + path: logs ``` ## Generate crash dumps From eeb6ccbc7186d0db09511f66b5c4ffebfb0b5a22 Mon Sep 17 00:00:00 2001 From: Trong Nhan Mai Date: Thu, 9 Jul 2026 17:25:12 +1000 Subject: [PATCH 4/6] collapse logging modes into 1 subsection; add missing toml to code blocks; remove mentioning pytest-jubilant usage hints in Default Behaviour; and other minor changes --- .../write-integration-tests-for-a-charm.md | 57 +++++++++---------- 1 file changed, 26 insertions(+), 31 deletions(-) diff --git a/docs/howto/write-integration-tests-for-a-charm.md b/docs/howto/write-integration-tests-for-a-charm.md index ed80e2f02..f018b51ec 100644 --- a/docs/howto/write-integration-tests-for-a-charm.md +++ b/docs/howto/write-integration-tests-for-a-charm.md @@ -460,29 +460,16 @@ tox -e integration -- tests/integration/test_charm.py -k "not test_one" > - [`pytest | How to invoke pytest`](https://docs.pytest.org/en/7.1.x/how-to/usage.html) > - [](#validate-your-charm-with-every-change) -(write-integration-tests-for-a-charm-configuring-jubilant-logs)= -## Configuring Jubilant logs +(write-integration-tests-for-a-charm-configure-jubilant-logs)= +## Configure Jubilant logs -Jubilant emits logs during the integration tests. These logs are handled by pytest if you are using `pytest-jubilant`. We can configure this behavior in `pyproject.toml`. +Jubilant emits logs during the integration tests. These logs are captured and displayed by pytest. You can configure the logging behaviour in the `[tool.pytest.ini_options]` table in `pyproject.toml`. -If you initialised the charm with `charmcraft init`, the `[tool.pytest.ini_options]` table already exist in `pyproject.toml`. For example: +### Logging modes -```toml -[tool.pytest.ini_options] -... -``` - -You can freely configure it to your needs. +**Brief** -> See more: -> - [`pytest | How to manage loggings`](https://docs.pytest.org/en/stable/reference/reference.html#logging) -> - [`pytest` | pyproject.toml](https://docs.pytest.org/en/stable/reference/customize.html#pyproject-toml) - -The sections below cover the configuration that we suggest. - -### Brief mode - -``` +```toml [tool.pytest.ini_options] ... log_cli = true @@ -492,7 +479,7 @@ log_cli_format = "%(levelname)s %(name)s %(message)s" While the test is running, logs with level `INFO` and above are emitted live to the console. -### Verbose mode +**Verbose** This mode shows whatever is in Brief, and the verbose `DEBUG` level logs. Lowering the log level is all it needs to switch from Brief to Verbose mode. @@ -512,7 +499,7 @@ We define the timestamps format with the `log_cli_date_format` key, following IS You can remove the timestamps without changing the mode, but we recommend leaving it in. -### Error mode +**Error** Jubilant emits logs at `ERROR` level when an application or unit transition into `error` during `juju.wait()`. This mode is useful if you only care if any charm turns into error status. @@ -524,7 +511,12 @@ log_cli_level = "ERROR" log_cli_format = "%(levelname)s %(name)s %(message)s" ``` -(write-integration-tests-log-to-a-file-mode)= +These options can also be configured from the command line. + +> See more: +> - [`pytest | How to manage logging`](https://docs.pytest.org/en/stable/how-to/logging.html) + +(write-integration-tests-log-to-a-file)= ### Log to a file This is an ideal configuration for long-running integration tests (for example, in CI). It uses Brief mode for logging to the console, and Verbose mode for logging to a local file. @@ -563,12 +555,17 @@ pytest --log-file "run2.log" ... Alternatively, you can set `log-file-mode` to `append`. Pytest will put verbose logs from all invocations into one file: -``` +```toml [tool.pytest.ini_options] log-file-mode = "a" ... ``` +These options can also be configured from the command line. + +> See more: +> - [`pytest | How to manage logging`](https://docs.pytest.org/en/stable/how-to/logging.html) + Use this mode in CI with `actions/upload-artifact` to make `logs/verbose.log` available as build artifacts: ```yaml @@ -582,19 +579,17 @@ Use this mode in CI with `actions/upload-artifact` to make `logs/verbose.log` av path: logs ``` -### Behavior when no pytest's logging config is set +### Default behaviour -This section covers the behavior when no logging configuration is set, either in config files, or from CLI arguments. In this case, pytest captures all log messages at `WARNING` level or above, and there is no logging to a file. You can still see messages from: +If no logging configuration is set by `tool.pytest.ini_options` or pytest CLI arguments, pytest captures all log messages at `WARNING` level or above and doesn't log to a file. You can still see messages from: -- `pytest`. For example: `tests/integration/test_charm.py::test_deploy PASSED` -- Logs from other modules if they are at `WARNING` or above. -- Usage hints from `pytest-jubilant`. For example: `Models were torn down. To keep models available for subsequent test runs or manual debugging, pass the following: ---no-juju-teardown` +- pytest. For example: `tests/integration/test_charm.py::test_deploy PASSED`. +- other modules if they are at `WARNING` or above. (write-integration-tests-for-a-charm-view-juju-logs)= ## View `juju debug-log` logs -If any tests fail, `pytest-jubilant` automatically logs the last 1000 lines of `juju debug-log` at `DEBUG` level. Log messages from `pytest-jubilant` are handled depending on how you configure pytest (see [](write-integration-tests-for-a-charm-configuring-jubilant-logs): +If any tests fail, `pytest-jubilant` automatically logs the last 1000 lines of `juju debug-log` at `DEBUG` level. Log messages from `pytest-jubilant` are handled depending on how you configure pytest (see [](write-integration-tests-for-a-charm-configure-jubilant-logs)): - For Brief mode and Error mode: They don't appear anywhere. - For Verbose mode: They appear in the terminal. @@ -602,7 +597,7 @@ If any tests fail, `pytest-jubilant` automatically logs the last 1000 lines of ` You can also save the complete `juju debug-log` to disk with the `--juju-dump-logs` option. -Use `--juju-dump-logs` in CI with `actions/upload-artifact` to make `juju debug-log` files available as build artifacts. We can modify the GitHub Actions from [](write-integration-tests-log-to-a-file-mode): +Use `--juju-dump-logs` in CI with `actions/upload-artifact` to make `juju debug-log` files available as build artifacts. We can modify the GitHub Actions from [](write-integration-tests-log-to-a-file): ```diff # In your integration test job From ca9387a2dd5679cb256adad3bb9ed2a36dba6058 Mon Sep 17 00:00:00 2001 From: Trong Nhan Mai Date: Thu, 9 Jul 2026 17:30:22 +1000 Subject: [PATCH 5/6] mention of pytest-jubilant usage hints in the default behaviour section I removed it by mistake. --- docs/howto/write-integration-tests-for-a-charm.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/howto/write-integration-tests-for-a-charm.md b/docs/howto/write-integration-tests-for-a-charm.md index f018b51ec..b13b84581 100644 --- a/docs/howto/write-integration-tests-for-a-charm.md +++ b/docs/howto/write-integration-tests-for-a-charm.md @@ -585,6 +585,7 @@ If no logging configuration is set by `tool.pytest.ini_options` or pytest CLI ar - pytest. For example: `tests/integration/test_charm.py::test_deploy PASSED`. - other modules if they are at `WARNING` or above. +- `pytest-jubilant`. For example: `Models were torn down...`. (write-integration-tests-for-a-charm-view-juju-logs)= ## View `juju debug-log` logs From d69899c493b39f46c0ab5ff36396d828c5295cc8 Mon Sep 17 00:00:00 2001 From: Trong Nhan Mai Date: Fri, 10 Jul 2026 17:16:40 +1000 Subject: [PATCH 6/6] update the howto to remove pytest-jubilant 1000k lines logs to stderr --- docs/howto/write-integration-tests-for-a-charm.md | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/docs/howto/write-integration-tests-for-a-charm.md b/docs/howto/write-integration-tests-for-a-charm.md index b13b84581..c3e666c77 100644 --- a/docs/howto/write-integration-tests-for-a-charm.md +++ b/docs/howto/write-integration-tests-for-a-charm.md @@ -590,15 +590,13 @@ If no logging configuration is set by `tool.pytest.ini_options` or pytest CLI ar (write-integration-tests-for-a-charm-view-juju-logs)= ## View `juju debug-log` logs -If any tests fail, `pytest-jubilant` automatically logs the last 1000 lines of `juju debug-log` at `DEBUG` level. Log messages from `pytest-jubilant` are handled depending on how you configure pytest (see [](write-integration-tests-for-a-charm-configure-jubilant-logs)): +Save the complete `juju debug-log` logs to disk with the `--juju-dump-logs` option from `pytest-jubilant`. For example: -- For Brief mode and Error mode: They don't appear anywhere. -- For Verbose mode: They appear in the terminal. -- For Log to a file: They appear in `log_file`, and don't appear in the terminal. - -You can also save the complete `juju debug-log` to disk with the `--juju-dump-logs` option. +```text +tox -e integration -- --juju-dump-logs logs +``` -Use `--juju-dump-logs` in CI with `actions/upload-artifact` to make `juju debug-log` files available as build artifacts. We can modify the GitHub Actions from [](write-integration-tests-log-to-a-file): +Use `--juju-dump-logs` in CI with `actions/upload-artifact` to make `juju debug-log` files available as build artifacts. We can modify the GitHub Actions workflow from [](write-integration-tests-log-to-a-file): ```diff # In your integration test job