diff --git a/docs/tutorial/from-zero-to-hero-write-your-first-kubernetes-charm/create-a-minimal-kubernetes-charm.md b/docs/tutorial/from-zero-to-hero-write-your-first-kubernetes-charm/create-a-minimal-kubernetes-charm.md index 5a238d5ff..2cc0e9502 100644 --- a/docs/tutorial/from-zero-to-hero-write-your-first-kubernetes-charm/create-a-minimal-kubernetes-charm.md +++ b/docs/tutorial/from-zero-to-hero-write-your-first-kubernetes-charm/create-a-minimal-kubernetes-charm.md @@ -54,33 +54,20 @@ These files currently contain placeholder code and configuration. Open `~/k8s-tutorial/charmcraft.yaml` in your usual text editor or IDE, then change the values of `title`, `summary`, and `description` to: -```yaml -title: Web Server Demo -summary: A demo charm that operates a small Python FastAPI server. -description: | - This charm demonstrates how to write a Kubernetes charm with Ops. +```{literalinclude} ../../../examples/k8s-1-minimal/charmcraft.yaml +:language: yaml +:start-at: 'title: Web Server Demo' +:end-at: how to write a Kubernetes charm with Ops. ``` Next, describe the workload container and its OCI image. In `charmcraft.yaml`, replace the `containers` and `resources` blocks with: -```yaml -containers: - demo-server: - resource: demo-server-image - -resources: - # An OCI image resource for the container listed above. - demo-server-image: - type: oci-image - description: OCI image from GitHub Container Repository - # The upstream-source field is ignored by Charmcraft and Juju, but it can be - # useful to developers in identifying the source of the OCI image. It is also - # used by the 'canonical/charming-actions' GitHub action for automated releases. - # The test_deploy function in tests/integration/test_charm.py reads upstream-source - # to determine which OCI image to use when running the charm's integration tests. - upstream-source: ghcr.io/canonical/api_demo_server:1.0.4 +```{literalinclude} ../../../examples/k8s-1-minimal/charmcraft.yaml +:language: yaml +:start-at: 'containers:' +:end-at: 'upstream-source: ghcr.io/canonical/api_demo_server:1.0.4' ``` ### Define the charm class @@ -114,8 +101,11 @@ As you can see, a charm is a pure Python class that inherits from the [`CharmBas In the `__init__` function of your charm class, we'll tell Ops which method of your charm class to run for each event. Let's start with when the Juju controller tells us that the workload container's Pebble is up and running. -```python -framework.observe(self.on["demo-server"].pebble_ready, self._on_demo_server_pebble_ready) +```{literalinclude} ../../../examples/k8s-1-minimal/src/charm.py +:language: python +:start-at: framework.observe(self.on["demo-server"] +:end-at: framework.observe(self.on["demo-server"] +:dedent: ``` @@ -138,65 +128,39 @@ We'll use the `ActiveStatus` class to set the charm status to active. Note that Use `ActiveStatus` as well as further Ops constructs to define the event handler, as below. As you can see, what is happening is that, from the `event` argument, you extract the workload container object in which you add a custom layer. Once the layer is set you replan your service and set the charm status to active. -```python -def _on_demo_server_pebble_ready(self, event: ops.PebbleReadyEvent) -> None: - """Define and start a workload using the Pebble API.""" - # Get a reference the container attribute on the PebbleReadyEvent - container = event.workload - # Add initial Pebble config layer using the Pebble API - container.add_layer("fastapi_demo", self._get_pebble_layer(), combine=True) - # Make Pebble reevaluate its plan, ensuring any services are started if enabled. - container.replan() - # Learn more about statuses at - # https://documentation.ubuntu.com/juju/3.6/reference/status/ - self.unit.status = ops.ActiveStatus() +```{literalinclude} ../../../examples/k8s-1-minimal/src/charm.py +:language: python +:pyobject: FastAPIDemoCharm._on_demo_server_pebble_ready +:dedent: ``` The custom Pebble layer that you just added is defined in the `self._get_pebble_layer()` method. We'll now add this method. In the `__init__` method of your charm class, name your service to `fastapi-service` and add it as a class attribute: -```python -self.pebble_service_name = "fastapi-service" +```{literalinclude} ../../../examples/k8s-1-minimal/src/charm.py +:language: python +:start-at: self.pebble_service_name = "fastapi-service" +:end-at: self.pebble_service_name = "fastapi-service" +:dedent: ``` Finally, define the `_get_pebble_layer` function as below. The `command` variable represents a command line that should be executed in order to start our application. -```python -def _get_pebble_layer(self) -> ops.pebble.Layer: - """Pebble layer for the FastAPI demo services.""" - command = " ".join( - [ - "uvicorn", - "api_demo_server.app:app", - "--host=0.0.0.0", - "--port=8000", - ] - ) - pebble_layer: ops.pebble.LayerDict = { - "summary": "FastAPI demo service", - "description": "pebble config layer for FastAPI demo server", - "services": { - self.pebble_service_name: { - "override": "replace", - "summary": "fastapi demo", - "command": command, - "startup": "enabled", - } - }, - } - return ops.pebble.Layer(pebble_layer) +```{literalinclude} ../../../examples/k8s-1-minimal/src/charm.py +:language: python +:pyobject: FastAPIDemoCharm._get_pebble_layer +:dedent: ``` ### Add logger functionality In the imports section of `src/charm.py`, import the Python `logging` module and define a logger object, as below. This will allow you to read log data in `juju`. -```python -import logging - -# Log messages can be retrieved using juju debug-log -logger = logging.getLogger(__name__) +```{literalinclude} ../../../examples/k8s-1-minimal/src/charm.py +:language: python +:start-at: import logging +:end-at: logger = logging.getLogger(__name__) ``` ## Try your charm @@ -323,44 +287,9 @@ In this section we'll write a test to check that Pebble is configured as expecte Replace the contents of `tests/unit/test_charm.py` with: -```python -import ops -from ops import testing - -from charm import FastAPIDemoCharm - - -def test_pebble_layer(): - ctx = testing.Context(FastAPIDemoCharm) - container = testing.Container(name="demo-server", can_connect=True) - state_in = testing.State( - containers={container}, - leader=True, - ) - state_out = ctx.run(ctx.on.pebble_ready(container), state_in) - # Expected plan after Pebble ready with default config - expected_plan = { - "services": { - "fastapi-service": { - "override": "replace", - "summary": "fastapi demo", - "command": "uvicorn api_demo_server.app:app --host=0.0.0.0 --port=8000", - "startup": "enabled", - # Since the environment is empty, Layer.to_dict() will not - # include it. - } - } - } - - # Check that we have the plan we expected: - assert state_out.get_container(container.name).plan == expected_plan - # Check the unit is active: - assert state_out.unit_status == testing.ActiveStatus() - # Check the service was started: - assert ( - state_out.get_container(container.name).service_statuses["fastapi-service"] - == ops.pebble.ServiceStatus.ACTIVE - ) +```{literalinclude} ../../../examples/k8s-1-minimal/tests/unit/test_charm.py +:language: python +:start-at: import ops ``` This test checks the behaviour of the `_on_demo_server_pebble_ready` function that you set up earlier. The test simulates your charm receiving the pebble-ready event, then checks that the unit and workload container have the correct state. @@ -418,28 +347,9 @@ Let's write the simplest possible integration test, a [smoke test](https://en.wi Replace the contents of `tests/integration/test_charm.py` with: -```python -import logging -import pathlib - -import jubilant -import pytest -import yaml - -logger = logging.getLogger(__name__) - -METADATA = yaml.safe_load(pathlib.Path("charmcraft.yaml").read_text()) -APP_NAME = METADATA["name"] - - -@pytest.mark.juju_setup -def test_deploy(charm: pathlib.Path, juju: jubilant.Juju): - """Deploy the charm under test.""" - resources = { - "demo-server-image": METADATA["resources"]["demo-server-image"]["upstream-source"] - } - juju.deploy(charm, app=APP_NAME, resources=resources) - juju.wait(jubilant.all_active) +```{literalinclude} ../../../examples/k8s-1-minimal/tests/integration/test_charm.py +:language: python +:start-at: import logging ``` This test depends on two fixtures: diff --git a/docs/tutorial/from-zero-to-hero-write-your-first-kubernetes-charm/expose-operational-tasks-via-actions.md b/docs/tutorial/from-zero-to-hero-write-your-first-kubernetes-charm/expose-operational-tasks-via-actions.md index 60dc9c22c..a8cf05cc8 100644 --- a/docs/tutorial/from-zero-to-hero-write-your-first-kubernetes-charm/expose-operational-tasks-via-actions.md +++ b/docs/tutorial/from-zero-to-hero-write-your-first-kubernetes-charm/expose-operational-tasks-via-actions.md @@ -28,29 +28,19 @@ In this part of the tutorial we will follow this process to add an action that w Open the `charmcraft.yaml` file and add to it a block defining an action, as below. As you can see, the action is called `get-db-info` and it is intended to help the user access database authentication information. The action has a single parameter, `show-password`; if set to `True`, it will show the username and the password. -```yaml -actions: - get-db-info: - description: Fetches database authentication information - params: - show-password: - description: Show username and password in output information - type: boolean - default: false - additionalProperties: false +```{literalinclude} ../../../examples/k8s-4-action/charmcraft.yaml +:language: yaml +:start-at: 'actions:' +:end-at: 'additionalProperties: false' ``` ## Define an action class Open your `src/charm.py` file, and add an action class that matches the definition you used in `charmcraft.yaml`: -```python -@dataclasses.dataclass(frozen=True, kw_only=True) -class GetDbInfoAction: - """Fetches database authentication information.""" - - show_password: bool - """Show username and password in output information.""" +```{literalinclude} ../../../examples/k8s-4-action/src/charm.py +:language: python +:pyobject: GetDbInfoAction ``` We'll use [](ActionEvent.load_params) to create an instance of your config class from the Juju action event. This allows IDEs to provide hints when we are accessing the action parameter, and static type checkers are able to validate that we are using the parameter correctly. @@ -61,45 +51,21 @@ Open the `src/charm.py` file. In the charm `__init__` method, add an action event observer, as below. As you can see, the name of the event consists of the name defined in the `charmcraft.yaml` file (`get-db-info`) and the word `action`. -```python -# Events on charm actions that are run via 'juju run'. -framework.observe(self.on.get_db_info_action, self._on_get_db_info_action) +```{literalinclude} ../../../examples/k8s-4-action/src/charm.py +:language: python +:start-at: "# Events on charm actions that are run via 'juju run'." +:end-at: framework.observe(self.on.get_db_info_action +:dedent: ``` Now, define the action event handler, as below: First, read the value of the parameter defined in the `charmcraft.yaml` file (`show-password`). Then, use the `fetch_database_relation_data` method (that we defined in a previous chapter) to read the contents of the database relation data and, if the parameter value read earlier is `True`, add the username and password to the output. Finally, use `event.set_results` to attach the results to the event that has called the action; this will print the output to the terminal. If we are not able to get the data (for example, if the charm has not yet been integrated with the postgresql-k8s application) then we use the `fail` method of the event to let the user know. -```python -def _on_get_db_info_action(self, event: ops.ActionEvent) -> None: - """Return information about the integrated database. - - This method is called when "get_db_info" action is called. It shows information about - database access points by calling the `fetch_database_relation_data` method and creates - an output dictionary containing the host, port, if show_password is True, then include - username, and password of the database. - - If the PostgreSQL charm is not integrated, the output is set to "No database connected". - - Learn more about actions at https://documentation.ubuntu.com/ops/latest/howto/manage-actions/ - """ - params = event.load_params(GetDbInfoAction, errors="fail") - db_data = self.fetch_database_relation_data() - if not db_data: - event.fail("No database connected") - return - output = { - "db-host": db_data.get("db_host", None), - "db-port": db_data.get("db_port", None), - } - if params.show_password: - output.update( - { - "db-username": db_data.get("db_username", None), - "db-password": db_data.get("db_password", None), - } - ) - event.set_results(output) +```{literalinclude} ../../../examples/k8s-4-action/src/charm.py +:language: python +:pyobject: FastAPIDemoCharm._on_get_db_info_action +:dedent: ``` ## Validate your charm @@ -155,64 +121,16 @@ Congratulations, you now know how to expose operational tasks via actions! Let's add a test to check the behaviour of the `get_db_info` action that we just set up. Our test sets up the context, defines the input state with a relation, then runs the action and checks whether the results match the expected values: -```python -def test_get_db_info_action(): - ctx = testing.Context(FastAPIDemoCharm) - relation = testing.Relation( - endpoint="database", - interface="postgresql_client", - remote_app_name="postgresql-k8s", - remote_app_data={ - "endpoints": "example.com:5432", - "username": "foo", - "password": "bar", - }, - ) - container = testing.Container(name="demo-server", can_connect=True) - state_in = testing.State( - containers={container}, - relations={relation}, - leader=True, - ) - - ctx.run(ctx.on.action("get-db-info", params={"show-password": False}), state_in) - - assert ctx.action_results == { - "db-host": "example.com", - "db-port": "5432", - } +```{literalinclude} ../../../examples/k8s-4-action/tests/unit/test_charm.py +:language: python +:pyobject: test_get_db_info_action ``` Since the `get_db_info` action has a parameter `show-password`, let's also add a test to cover the case where the user wants to show the password: -```python -def test_get_db_info_action_show_password(): - ctx = testing.Context(FastAPIDemoCharm) - relation = testing.Relation( - endpoint="database", - interface="postgresql_client", - remote_app_name="postgresql-k8s", - remote_app_data={ - "endpoints": "example.com:5432", - "username": "foo", - "password": "bar", - }, - ) - container = testing.Container(name="demo-server", can_connect=True) - state_in = testing.State( - containers={container}, - relations={relation}, - leader=True, - ) - - ctx.run(ctx.on.action("get-db-info", params={"show-password": True}), state_in) - - assert ctx.action_results == { - "db-host": "example.com", - "db-port": "5432", - "db-username": "foo", - "db-password": "bar", - } +```{literalinclude} ../../../examples/k8s-4-action/tests/unit/test_charm.py +:language: python +:pyobject: test_get_db_info_action_show_password ``` Run `tox -e unit` to check that all tests pass. diff --git a/docs/tutorial/from-zero-to-hero-write-your-first-kubernetes-charm/integrate-your-charm-with-postgresql.md b/docs/tutorial/from-zero-to-hero-write-your-first-kubernetes-charm/integrate-your-charm-with-postgresql.md index 5ceb76822..a990cc8cb 100644 --- a/docs/tutorial/from-zero-to-hero-write-your-first-kubernetes-charm/integrate-your-charm-with-postgresql.md +++ b/docs/tutorial/from-zero-to-hero-write-your-first-kubernetes-charm/integrate-your-charm-with-postgresql.md @@ -22,10 +22,10 @@ A charm often requires or supports relations to other charms. For example, to ma In `charmcraft.yaml`, add a `charm-libs` section before the `containers` section: -```yaml -charm-libs: - - lib: data_platform_libs.data_interfaces - version: "0" +```{literalinclude} ../../../examples/k8s-3-postgresql/charmcraft.yaml +:language: yaml +:start-at: 'charm-libs:' +:end-at: 'version: "0"' ``` This tells Charmcraft that your charm requires the [`data_interfaces`](https://charmhub.io/data-platform-libs/libraries/data_interfaces) charm library from Charmhub. @@ -64,12 +64,10 @@ First, find out the name of the interface that PostgreSQL offers for other charm Next, open the `charmcraft.yaml` file of your charm and, before the `charm-libs` section, define a relation endpoint using a `requires` block, as below. This endpoint says that our charm is requesting a relation called `database` over an interface called `postgresql_client` with a maximum number of supported connections of 1. (Note: Here, `database` is a custom relation name, though in general we recommend sticking to default recommended names for each charm.) -```yaml -requires: - database: - interface: postgresql_client - limit: 1 - optional: false +```{literalinclude} ../../../examples/k8s-3-postgresql/charmcraft.yaml +:language: yaml +:start-at: 'requires:' +:end-at: 'optional: false' ``` That will tell `juju` that our charm can be integrated with charms that provide the same `postgresql_client` interface, for example, the official PostgreSQL charm. @@ -86,15 +84,10 @@ To do so, we need to update our charm `src/charm.py` to do all of the following: At the top of `src/charm.py`, import the database interfaces library: -```python -# Import the 'data_interfaces' library. -# The import statement omits the top-level 'lib' directory -# because 'charmcraft pack' copies its contents to the project root. -from charms.data_platform_libs.v0.data_interfaces import ( - DatabaseCreatedEvent, - DatabaseEndpointsChangedEvent, - DatabaseRequires, -) +```{literalinclude} ../../../examples/k8s-3-postgresql/src/charm.py +:language: python +:start-at: "# Import the 'data_interfaces' library." +:end-at: ')' ``` ````{important} @@ -129,28 +122,28 @@ export PYTHONPATH=lib:$PYTHONPATH In the `__init__` method, define a new instance of the 'DatabaseRequires' class. This is required to set the right permissions scope for the PostgreSQL charm. It will create a new user with a password and a database with the required name (below, `names_db`), and limit the user permissions to only this particular database (that is, below, `names_db`). -```python -# The 'relation_name' comes from the 'charmcraft.yaml file'. -# The 'database_name' is the name of the database that our application requires. -self.database = DatabaseRequires(self, relation_name="database", database_name="names_db") +```{literalinclude} ../../../examples/k8s-3-postgresql/src/charm.py +:language: python +:start-at: "# The 'relation_name' comes from the 'charmcraft.yaml file'." +:end-at: self.database = DatabaseRequires( +:dedent: ``` Next, add event observers for all the database events: -```python -# See https://charmhub.io/data-platform-libs/libraries/data_interfaces -framework.observe(self.database.on.database_created, self._on_database_endpoint) -framework.observe(self.database.on.endpoints_changed, self._on_database_endpoint) +```{literalinclude} ../../../examples/k8s-3-postgresql/src/charm.py +:language: python +:start-at: "# See https://charmhub.io/data-platform-libs/libraries/data_interfaces" +:end-at: framework.observe(self.database.on.endpoints_changed +:dedent: ``` Finally, define the method that is called on the database events: -```python -def _on_database_endpoint( - self, _: DatabaseCreatedEvent | DatabaseEndpointsChangedEvent -) -> None: - """Event is fired when the database is created or its endpoint is changed.""" - self._replan_workload() +```{literalinclude} ../../../examples/k8s-3-postgresql/src/charm.py +:language: python +:pyobject: FastAPIDemoCharm._on_database_endpoint +:dedent: ``` We now need to make sure that our application knows how to access the database. @@ -159,40 +152,18 @@ We now need to make sure that our application knows how to access the database. Our application consumes database authentication data in the form of environment variables. Let's define a method that prepares database authentication data in that form: -```python -def get_app_environment(self) -> dict[str, str]: - """Return a dictionary of environment variables for the application.""" - db_data = self.fetch_database_relation_data() - if not db_data: - return {} - return { - "DEMO_SERVER_DB_HOST": db_data["db_host"], - "DEMO_SERVER_DB_PORT": db_data["db_port"], - "DEMO_SERVER_DB_USER": db_data["db_username"], - "DEMO_SERVER_DB_PASSWORD": db_data["db_password"], - } +```{literalinclude} ../../../examples/k8s-3-postgresql/src/charm.py +:language: python +:pyobject: FastAPIDemoCharm.get_app_environment +:dedent: ``` This method depends on the following method, which extracts the database authentication data: -```python -def fetch_database_relation_data(self) -> dict[str, str]: - """Retrieve relation data from a database.""" - relations = self.database.fetch_relation_data() - logger.debug("Got following database data: %s", relations) - for data in relations.values(): - if not data: - continue - logger.info("New database endpoint is %s", data["endpoints"]) - host, port = data["endpoints"].split(":") - db_data = { - "db_host": host, - "db_port": port, - "db_username": data["username"], - "db_password": data["password"], - } - return db_data - return {} +```{literalinclude} ../../../examples/k8s-3-postgresql/src/charm.py +:language: python +:pyobject: FastAPIDemoCharm.fetch_database_relation_data +:dedent: ``` ### Share the authentication data with your application @@ -201,72 +172,20 @@ Let's change the Pebble service definition to include a dynamic `environment` ke First, update `_replan_workload()` to provide environment variables when creating the Pebble layer: -```python -def _replan_workload(self) -> None: - """Define and start a workload using the Pebble API. - - You'll need to specify the right entrypoint and environment - configuration for your specific workload. Tip: you can see the - standard entrypoint of an existing container using docker inspect - Learn more about interacting with Pebble at - https://documentation.ubuntu.com/ops/latest/reference/pebble/ - Learn more about Pebble layers at - https://documentation.ubuntu.com/pebble/how-to/use-layers/ - """ - # Learn more about statuses at - # https://documentation.ubuntu.com/juju/3.6/reference/status/ - self.unit.status = ops.MaintenanceStatus("Assembling Pebble layers") - try: - config = self.load_config(FastAPIConfig) - except ValueError as e: - logger.error("Configuration error: %s", e) - return - env = self.get_app_environment() - try: - self.container.add_layer( - "fastapi_demo", - self._get_pebble_layer(config.server_port, env), - combine=True, - ) - logger.info("Added updated layer 'fastapi_demo' to Pebble plan") - - # Tell Pebble to incorporate the changes, including restarting the - # service if required. - self.container.replan() - logger.info(f"Replanned with '{self.pebble_service_name}' service") - except (ops.pebble.APIError, ops.pebble.ConnectionError) as e: - logger.info("Unable to connect to Pebble: %s", e) +```{literalinclude} ../../../examples/k8s-3-postgresql/src/charm.py +:language: python +:pyobject: FastAPIDemoCharm._replan_workload +:dedent: ``` We removed three `self.unit.status = ` lines from this version of the method. We'll handle replacing those shortly. Next, update `_get_pebble_layer()` to put the environment variables in the Pebble layer: -```python -def _get_pebble_layer(self, port: int, environment: dict[str, str]) -> ops.pebble.Layer: - """Pebble layer for the FastAPI demo services.""" - command = " ".join( - [ - "uvicorn", - "api_demo_server.app:app", - "--host=0.0.0.0", - f"--port={port}", - ] - ) - pebble_layer: ops.pebble.LayerDict = { - "summary": "FastAPI demo service", - "description": "pebble config layer for FastAPI demo server", - "services": { - self.pebble_service_name: { - "override": "replace", - "summary": "fastapi demo", - "command": command, - "startup": "enabled", - "environment": environment, - } - }, - } - return ops.pebble.Layer(pebble_layer) +```{literalinclude} ../../../examples/k8s-3-postgresql/src/charm.py +:language: python +:pyobject: FastAPIDemoCharm._get_pebble_layer +:dedent: ``` With these changes, we've made sure that our application knows how to access the database. @@ -287,34 +206,19 @@ Now that the charm is getting more complex, there are many more cases where the In your charm's `__init__` add a new observer: -```python -# Report the unit status after each event. -framework.observe(self.on.collect_unit_status, self._on_collect_status) +```{literalinclude} ../../../examples/k8s-3-postgresql/src/charm.py +:language: python +:start-at: "# Report the unit status after each event." +:end-at: framework.observe(self.on.collect_unit_status +:dedent: ``` And define a method that does the various checks, adding appropriate statuses. The library will take care of selecting the 'most significant' status for you. -```python -def _on_collect_status(self, event: ops.CollectStatusEvent) -> None: - try: - self.load_config(FastAPIConfig) - except ValueError as e: - event.add_status(ops.BlockedStatus(str(e))) - if not self.model.get_relation("database"): - # We need the user to do 'juju integrate'. - event.add_status(ops.BlockedStatus("Waiting for database relation")) - elif not self.database.fetch_relation_data(): - # We need the charms to finish integrating. - event.add_status(ops.WaitingStatus("Waiting for database relation")) - try: - status = self.container.get_service(self.pebble_service_name) - except (ops.pebble.APIError, ops.pebble.ConnectionError, ops.ModelError): - event.add_status(ops.MaintenanceStatus("Waiting for Pebble in workload container")) - else: - if not status.is_running(): - event.add_status(ops.MaintenanceStatus("Waiting for the service to start up")) - # If nothing is wrong, then the status is active. - event.add_status(ops.ActiveStatus()) +```{literalinclude} ../../../examples/k8s-3-postgresql/src/charm.py +:language: python +:pyobject: FastAPIDemoCharm._on_collect_status +:dedent: ``` We also want to clean up the code to remove the places where we're setting the status outside of this method, other than anywhere we're wanting a status to show up *during* the event execution (such as `MaintenanceStatus`). If you missed doing so above, in `_replan_workload`, remove the lines: @@ -418,59 +322,25 @@ Congratulations, your relation with PostgreSQL is functional! Now that our charm uses `fetch_database_relation_data` to extract database authentication data and endpoint information from the relation data, we should write a test for the feature. Here, we're not testing the `fetch_database_relation_data` function directly, but rather, we're checking that the response to a Juju event is what it should be: -```python -def test_relation_data(): - ctx = testing.Context(FastAPIDemoCharm) - relation = testing.Relation( - endpoint="database", - interface="postgresql_client", - remote_app_name="postgresql-k8s", - remote_app_data={ - "endpoints": "example.com:5432", - "username": "foo", - "password": "bar", - }, - ) - container = testing.Container(name="demo-server", can_connect=True) - state_in = testing.State( - containers={container}, - relations={relation}, - leader=True, - ) - - state_out = ctx.run(ctx.on.relation_changed(relation), state_in) - - assert state_out.get_container(container.name).layers["fastapi_demo"].services[ - "fastapi-service" - ].environment == { - "DEMO_SERVER_DB_HOST": "example.com", - "DEMO_SERVER_DB_PORT": "5432", - "DEMO_SERVER_DB_USER": "foo", - "DEMO_SERVER_DB_PASSWORD": "bar", - } +```{literalinclude} ../../../examples/k8s-3-postgresql/tests/unit/test_charm.py +:language: python +:pyobject: test_relation_data ``` In this chapter, we also defined a new method `_on_collect_status` that checks various things, including whether the required database relation exists. If the relation doesn't exist, we wait and set the unit status to `blocked`. We can also add a test to cover this behaviour: -```python -def test_no_database_blocked(): - ctx = testing.Context(FastAPIDemoCharm) - container = testing.Container(name="demo-server", can_connect=True) - state_in = testing.State( - containers={container}, - leader=True, - ) # We've omitted relation data from the input state. - - state_out = ctx.run(ctx.on.collect_unit_status(), state_in) - - assert state_out.unit_status == testing.BlockedStatus("Waiting for database relation") +```{literalinclude} ../../../examples/k8s-3-postgresql/tests/unit/test_charm.py +:language: python +:pyobject: test_no_database_blocked ``` Then modify `test_pebble_layer`. Since `test_pebble_layer` doesn't arrange a database relation, the unit will be in `blocked` status instead of `active`. Replace the `assert state_out.unit_status` line by: -```python - # Check the unit is blocked: - assert state_out.unit_status == testing.BlockedStatus("Waiting for database relation") +```{literalinclude} ../../../examples/k8s-3-postgresql/tests/unit/test_charm.py +:language: python +:start-at: "# Check the unit is blocked:" +:end-at: 'assert state_out.unit_status == testing.BlockedStatus("Waiting for database relation")' +:dedent: ``` Now run `tox -e unit` to make sure all test cases pass. @@ -479,49 +349,19 @@ Now run `tox -e unit` to make sure all test cases pass. Now that our charm integrates with the database, if there's not a database relation, the app will be in `blocked` status instead of `active`. Let's tweak our existing integration test `test_deploy` accordingly, to expect blocked status in `juju.wait`. Replace the contents of `tests/integration/test_charm.py` with: -```python -import logging -import pathlib - -import jubilant -import pytest -import yaml - -logger = logging.getLogger(__name__) - -METADATA = yaml.safe_load(pathlib.Path("./charmcraft.yaml").read_text()) -APP_NAME = METADATA["name"] - - -@pytest.mark.juju_setup -def test_deploy(charm: pathlib.Path, juju: jubilant.Juju): - """Deploy the charm under test. - - Assert on the unit status before any relations/configurations take place. - """ - resources = { - "demo-server-image": METADATA["resources"]["demo-server-image"]["upstream-source"] - } - - # Deploy the charm and wait for it to report blocked, as it needs Postgres. - juju.deploy(charm, app=APP_NAME, resources=resources) - juju.wait(jubilant.all_blocked) +```{literalinclude} ../../../examples/k8s-3-postgresql/tests/integration/test_charm.py +:language: python +:start-at: import logging +:end-at: juju.wait(jubilant.all_blocked) ``` Then, let's add another test case to check the integration is successful. For that, we need to deploy a database to the test cluster and integrate both applications. If everything works as intended, the charm should report an active status. In your `tests/integration/test_charm.py` file add the following test case: -```python -@pytest.mark.juju_setup -def test_database_integration(charm: pathlib.Path, juju: jubilant.Juju): - """Verify that the charm integrates with the database. - - Assert that the charm is active if the integration is established. - """ - juju.deploy("postgresql-k8s", channel="14/stable", trust=True) - juju.integrate(APP_NAME, "postgresql-k8s") - juju.wait(jubilant.all_active) +```{literalinclude} ../../../examples/k8s-3-postgresql/tests/integration/test_charm.py +:language: python +:pyobject: test_database_integration ``` This test depends on the `charm` fixture so that the test fails immediately if a `.charm` file isn't available. diff --git a/docs/tutorial/from-zero-to-hero-write-your-first-kubernetes-charm/make-your-charm-configurable.md b/docs/tutorial/from-zero-to-hero-write-your-first-kubernetes-charm/make-your-charm-configurable.md index ddc54fa73..7d5b41bd6 100644 --- a/docs/tutorial/from-zero-to-hero-write-your-first-kubernetes-charm/make-your-charm-configurable.md +++ b/docs/tutorial/from-zero-to-hero-write-your-first-kubernetes-charm/make-your-charm-configurable.md @@ -30,13 +30,10 @@ To begin with, let's define the option that will be available for configuration. In `charmcraft.yaml`, replace the `config` block with: -```yaml -config: - options: - server-port: - default: 8000 - description: Default port on which FastAPI is available - type: int +```{literalinclude} ../../../examples/k8s-2-configurable/charmcraft.yaml +:language: yaml +:start-at: 'config:' +:end-at: 'type: int' ``` This defines a configuration option called `server-port`. The `default` value is `8000` -- this is the value you're trying to allow a charm user to configure. @@ -45,18 +42,9 @@ This defines a configuration option called `server-port`. The `default` value is Open your `src/charm.py` file, and add a configuration class that matches the configuration you added in `charmcraft.yaml`: -```python -@dataclasses.dataclass(frozen=True, kw_only=True) -class FastAPIConfig: - """Configuration for the FastAPI demo charm.""" - - server_port: int = 8000 - """Default port on which FastAPI is available.""" - - def __post_init__(self): - """Validate the configuration.""" - if self.server_port == 22: - raise ValueError("Invalid port number, 22 is reserved for SSH") +```{literalinclude} ../../../examples/k8s-2-configurable/src/charm.py +:language: python +:pyobject: FastAPIConfig ``` Then, still in `src/charm.py`, add `import dataclasses` in the imports at the top of the file. @@ -69,15 +57,19 @@ Open your `src/charm.py` file. In the `__init__` function, add an observer for the `config_changed` event and pair it with an `_on_config_changed` handler: -```python -framework.observe(self.on.config_changed, self._on_config_changed) +```{literalinclude} ../../../examples/k8s-2-configurable/src/charm.py +:language: python +:start-at: framework.observe(self.on.config_changed +:end-at: framework.observe(self.on.config_changed +:dedent: ``` Now, define the handler, as below. Since configuring something like a port affects the way we call our workload application, we need to update our Pebble configuration. -```python -def _on_config_changed(self, _: ops.ConfigChangedEvent) -> None: - self._replan_workload() +```{literalinclude} ../../../examples/k8s-2-configurable/src/charm.py +:language: python +:pyobject: FastAPIDemoCharm._on_config_changed +:dedent: ``` We'll define `_replan_workload` shortly. @@ -89,86 +81,37 @@ A charm does not know which configuration option has been changed. Thus, make su In the `__init__` function, add a new attribute to define a container object for your workload: -```python -# See 'containers' in charmcraft.yaml. -self.container = self.unit.get_container("demo-server") +```{literalinclude} ../../../examples/k8s-2-configurable/src/charm.py +:language: python +:start-at: "# See 'containers' in charmcraft.yaml." +:end-at: self.container = self.unit.get_container("demo-server") +:dedent: ``` Create a new method, as below. This method will get the current Pebble layer configuration and compare the new and the existing service definitions -- if they differ, it will update the layer and restart the service. -```python -def _replan_workload(self) -> None: - """Define and start a workload using the Pebble API. - - You'll need to specify the right entrypoint and environment - configuration for your specific workload. Tip: you can see the - standard entrypoint of an existing container using docker inspect - Learn more about interacting with Pebble at - https://documentation.ubuntu.com/ops/latest/reference/pebble/ - Learn more about Pebble layers at - https://documentation.ubuntu.com/pebble/how-to/use-layers/ - """ - # Learn more about statuses at - # https://documentation.ubuntu.com/juju/3.6/reference/status/ - self.unit.status = ops.MaintenanceStatus("Assembling Pebble layers") - try: - config = self.load_config(FastAPIConfig) - except ValueError as e: - logger.error("Configuration error: %s", e) - self.unit.status = ops.BlockedStatus(str(e)) - return - try: - self.container.add_layer( - "fastapi_demo", self._get_pebble_layer(config.server_port), combine=True - ) - logger.info("Added updated layer 'fastapi_demo' to Pebble plan") - - # Tell Pebble to incorporate the changes, including restarting the - # service if required. - self.container.replan() - logger.info(f"Replanned with '{self.pebble_service_name}' service") - - self.unit.status = ops.ActiveStatus() - except (ops.pebble.APIError, ops.pebble.ConnectionError) as e: - logger.info("Unable to connect to Pebble: %s", e) - self.unit.status = ops.MaintenanceStatus("Waiting for Pebble in workload container") +```{literalinclude} ../../../examples/k8s-2-configurable/src/charm.py +:language: python +:pyobject: FastAPIDemoCharm._replan_workload +:dedent: ``` When the config is loaded as part of creating the Pebble layer, if the config is invalid (in our case, if the port is set to 22), then a `ValueError` will be raised. The `_replan_workload` method handles that by logging the error and setting the status of the unit to blocked, letting the Juju user know that they need to take action. Now, crucially, update the `_get_pebble_layer` method to make the layer definition dynamic, as shown below. This will replace the static port `8000` with the port passed to the method. -```python -def _get_pebble_layer(self, port: int) -> ops.pebble.Layer: - """Pebble layer for the FastAPI demo services.""" - command = " ".join( - [ - "uvicorn", - "api_demo_server.app:app", - "--host=0.0.0.0", - f"--port={port}", - ] - ) - pebble_layer: ops.pebble.LayerDict = { - "summary": "FastAPI demo service", - "description": "pebble config layer for FastAPI demo server", - "services": { - self.pebble_service_name: { - "override": "replace", - "summary": "fastapi demo", - "command": command, - "startup": "enabled", - } - }, - } - return ops.pebble.Layer(pebble_layer) +```{literalinclude} ../../../examples/k8s-2-configurable/src/charm.py +:language: python +:pyobject: FastAPIDemoCharm._get_pebble_layer +:dedent: ``` As you may have noticed, the new `_replan_workload` method looks like a more advanced variant of the existing `_on_demo_server_pebble_ready` method. Remove the body of the `_on_demo_server_pebble_ready` method and replace it a call to `_replan_workload` like this: -```python -def _on_demo_server_pebble_ready(self, _: ops.PebbleReadyEvent) -> None: - self._replan_workload() +```{literalinclude} ../../../examples/k8s-2-configurable/src/charm.py +:language: python +:pyobject: FastAPIDemoCharm._on_demo_server_pebble_ready +:dedent: ``` ## Validate your charm @@ -241,40 +184,16 @@ Since we added a new feature to configure `server-port` and use it in the Pebble First, we'll add a test that sets the port in the input state and asserts that the port is used in the service's command in the container layer: -```python -def test_config_changed(): - ctx = testing.Context(FastAPIDemoCharm) - container = testing.Container(name="demo-server", can_connect=True) - state_in = testing.State( - containers={container}, - config={"server-port": 8080}, - leader=True, - ) - state_out = ctx.run(ctx.on.config_changed(), state_in) - command = ( - state_out.get_container(container.name) - .layers["fastapi_demo"] - .services["fastapi-service"] - .command - ) - assert "--port=8080" in command +```{literalinclude} ../../../examples/k8s-2-configurable/tests/unit/test_charm.py +:language: python +:pyobject: test_config_changed ``` In `_on_config_changed`, we specifically don't allow port 22 to be used. If port 22 is configured, we set the unit status to `blocked`. So, we can add a test to cover this behaviour by setting the port to 22 in the input state and asserting that the unit status is blocked: -```python -def test_config_changed_invalid_port(): - ctx = testing.Context(FastAPIDemoCharm) - container = testing.Container(name="demo-server", can_connect=True) - state_in = testing.State( - containers={container}, - config={"server-port": 22}, - leader=True, - ) - state_out = ctx.run(ctx.on.config_changed(), state_in) - assert state_out.unit_status == testing.BlockedStatus( - "Invalid port number, 22 is reserved for SSH" - ) +```{literalinclude} ../../../examples/k8s-2-configurable/tests/unit/test_charm.py +:language: python +:pyobject: test_config_changed_invalid_port ``` Run `tox -e unit` to check that all tests pass. diff --git a/docs/tutorial/from-zero-to-hero-write-your-first-kubernetes-charm/observe-your-charm-with-cos-lite.md b/docs/tutorial/from-zero-to-hero-write-your-first-kubernetes-charm/observe-your-charm-with-cos-lite.md index 2fb4fd6e2..519108e6d 100644 --- a/docs/tutorial/from-zero-to-hero-write-your-first-kubernetes-charm/observe-your-charm-with-cos-lite.md +++ b/docs/tutorial/from-zero-to-hero-write-your-first-kubernetes-charm/observe-your-charm-with-cos-lite.md @@ -31,18 +31,10 @@ Your charm will need several more libraries from Charmhub. Ensure you're in your Multipass Ubuntu VM, in your charm project directory. Then, in `charmcraft.yaml`, extend the `charm-libs` section: -```yaml -charm-libs: - - lib: data_platform_libs.data_interfaces - version: "0" - - lib: grafana_k8s.grafana_dashboard - version: "0" - - lib: loki_k8s.loki_push_api - version: "1" - - lib: observability_libs.juju_topology - version: "0" - - lib: prometheus_k8s.prometheus_scrape - version: "0" +```{literalinclude} ../../../examples/k8s-5-observe/charmcraft.yaml +:language: yaml +:start-at: 'charm-libs:' +:end-before: 'actions:' ``` Next, run the following command to download the libraries: @@ -101,11 +93,10 @@ Follow the steps below to make your charm capable of integrating with the existi In your `charmcraft.yaml` file, after the `requires` block, add a `provides` endpoint with relation name `metrics-endpoint` and interface name `prometheus_scrape`, as below. This declares that your charm can offer services to other charms over the `prometheus-scrape` interface. In short, that your charm is open to integrating with, for example, the official Prometheus charm. (Note: `metrics-endpoint` is the default relation name recommended by the `prometheus_scrape` interface library.) -```yaml -provides: - metrics-endpoint: - interface: prometheus_scrape - optional: true +```{literalinclude} ../../../examples/k8s-5-observe/charmcraft.yaml +:language: yaml +:start-at: 'provides:' +:end-before: ' grafana-dashboard:' ``` ## Import the Prometheus interface libraries and set up Prometheus scraping @@ -114,25 +105,19 @@ In your `src/charm.py` file, do the following: First, at the top of the file, import the `prometheus_scrape` library: -```python -from charms.prometheus_k8s.v0.prometheus_scrape import MetricsEndpointProvider +```{literalinclude} ../../../examples/k8s-5-observe/src/charm.py +:language: python +:start-at: from charms.prometheus_k8s.v0.prometheus_scrape +:end-at: from charms.prometheus_k8s.v0.prometheus_scrape ``` Now, in your charm's `__init__` method, initialise the `MetricsEndpointProvider` instance with the desired scrape target, as below. Note that this uses the relation name that you specified earlier in the `charmcraft.yaml` file. Also, reflecting the fact that you've made your charm's port configurable (see previous chapter {ref}`Make the charm configurable `), the target job is set to be consumed from config. The URL path is not included because it is predictable (defaults to /metrics), so the Prometheus library uses it automatically. The last line, which sets the `refresh_event` to the `config_change` event, ensures that the Prometheus charm will change its scraping target every time someone changes the port configuration. Overall, this code will allow your application to be scraped by Prometheus once they've been integrated. -```python -# Provide a metrics endpoint for Prometheus to scrape. -try: - config = self.load_config(FastAPIConfig) -except ValueError as e: - logger.warning("Unable to add metrics: invalid configuration: %s", e) -else: - self._prometheus_scraping = MetricsEndpointProvider( - self, - relation_name="metrics-endpoint", - jobs=[{"static_configs": [{"targets": [f"*:{config.server_port}"]}]}], - refresh_event=self.on.config_changed, - ) +```{literalinclude} ../../../examples/k8s-5-observe/src/charm.py +:language: python +:start-at: "# Provide a metrics endpoint for Prometheus to scrape." +:end-before: "# Provide grafana dashboards" +:dedent: ``` Congratulations, your charm is ready to be integrated with Prometheus! @@ -145,15 +130,10 @@ Follow the steps below to make your charm capable of integrating with the existi In your `charmcraft.yaml` file, beneath your existing `requires` endpoint, add another `requires` endpoint with relation name `logging` and interface name `loki_push_api`. This declares that your charm can optionally make use of services from other charms over the `loki_push_api` interface. In short, that your charm is open to integrating with, for example, the official Loki charm. (Note: `logging` is the default relation name recommended by the `loki_push_api` interface library.) -```yaml -requires: - database: - interface: postgresql_client - limit: 1 - optional: false - logging: - interface: loki_push_api - optional: true +```{literalinclude} ../../../examples/k8s-5-observe/charmcraft.yaml +:language: yaml +:start-at: 'requires:' +:end-before: 'provides:' ``` ## Import the Loki interface libraries and set up the Loki API @@ -162,15 +142,19 @@ In your `src/charm.py` file, do the following: First, import the `loki_push_api` lib: -```python -from charms.loki_k8s.v1.loki_push_api import LogForwarder +```{literalinclude} ../../../examples/k8s-5-observe/src/charm.py +:language: python +:start-at: from charms.loki_k8s.v1.loki_push_api +:end-at: from charms.loki_k8s.v1.loki_push_api ``` Then, in your charm's `__init__` method, initialise the `LogForwarder` instance as shown below. The `logging` relation name comes from the `charmcraft.yaml` file. Overall this code ensures that your application can push logs to Loki (or any other charms that implement the `loki_push_api` interface). -```python -# Enable pushing application logs to Loki. -self._logging = LogForwarder(self, relation_name="logging") +```{literalinclude} ../../../examples/k8s-5-observe/src/charm.py +:language: python +:start-at: "# Enable pushing application logs to Loki." +:end-at: self._logging = LogForwarder +:dedent: ``` Congratulations, your charm can now also integrate with Loki! @@ -183,14 +167,10 @@ Follow the steps below to make your charm capable of integrating with the existi In your `charmcraft.yaml` file, add another `provides` endpoint with relation name `grafana-dashboard` and interface name `grafana_dashboard`, as below. This declares that your charm can offer services to other charms over the `grafana-dashboard` interface. In short, that your charm is open to integrations with, for example, the official Grafana charm. (Note: Here `grafana-dashboard` endpoint is the default relation name recommended by the `grafana_dashboard` library.) -```yaml -provides: - metrics-endpoint: - interface: prometheus_scrape - optional: true - grafana-dashboard: - interface: grafana_dashboard - optional: true +```{literalinclude} ../../../examples/k8s-5-observe/charmcraft.yaml +:language: yaml +:start-at: 'provides:' +:end-before: 'charm-libs:' ``` ### Import the Grafana interface libraries and set up the Grafana dashboards @@ -199,17 +179,19 @@ In your `src/charm.py` file, do the following: First, at the top of the file, import the `grafana_dashboard` lib: -```python -from charms.grafana_k8s.v0.grafana_dashboard import GrafanaDashboardProvider +```{literalinclude} ../../../examples/k8s-5-observe/src/charm.py +:language: python +:start-at: from charms.grafana_k8s.v0.grafana_dashboard +:end-at: from charms.grafana_k8s.v0.grafana_dashboard ``` Now, in your charm's `__init__` method, initialise the `GrafanaDashboardProvider` instance, as below. The `grafana-dashboard` is the relation name you defined earlier in your `charmcraft.yaml` file. Overall, this code states that your application supports the Grafana interface. -```python -# Provide grafana dashboards over a relation interface. -self._grafana_dashboards = GrafanaDashboardProvider( - self, relation_name="grafana-dashboard" -) +```{literalinclude} ../../../examples/k8s-5-observe/src/charm.py +:language: python +:start-at: "# Provide grafana dashboards over a relation interface." +:end-at: ')' +:dedent: ``` Now, in your `src` directory, create a subdirectory called `grafana_dashboards` and, in this directory, create a file called `FastAPI-Monitoring.json.tmpl` with the following content: @@ -482,25 +464,18 @@ The existing integration tests use a model that is created by the `juju` fixture First, in `tests/integration/test_charm.py`, import `json` and `time` from the standard library. Then import `pytest`, `pytest_jubilant`, and `requests`. Your imports should now look like this: -```python -import json -import logging -import pathlib -import time - -import jubilant -import pytest -import pytest_jubilant -import requests -import yaml +```{literalinclude} ../../../examples/k8s-5-observe/tests/integration/test_charm.py +:language: python +:start-at: import json +:end-at: import yaml ``` Next, still in `tests/integration/test_charm.py`, define the new fixture: -```python -@pytest.fixture(scope="module") -def cos(juju_factory: pytest_jubilant.JujuFactory): - yield juju_factory.get_juju(suffix="cos") +```{literalinclude} ../../../examples/k8s-5-observe/tests/integration/test_charm.py +:language: python +:start-at: '@pytest.fixture(scope="module")' +:end-at: yield juju_factory.get_juju ``` `get_juju` creates a model called `jubilant--cos`. @@ -511,54 +486,28 @@ We're now ready to write the tests. Add two test functions to `tests/integration/test_charm.py`: -```python -@pytest.mark.juju_setup -def test_deploy_cos(cos: jubilant.Juju): - """Deploy COS Lite in a separate model.""" - cos.deploy("cos-lite", trust=True) - cos.wait(jubilant.all_active, timeout=10 * 60) # Allow time for the bundle to deploy. - +```{literalinclude} ../../../examples/k8s-5-observe/tests/integration/test_charm.py +:language: python +:pyobject: test_deploy_cos +``` -@pytest.mark.juju_setup -def test_integrate_loki(juju: jubilant.Juju, cos: jubilant.Juju): - """Integrate our app with Loki from COS Lite.""" - cos.offer("loki", endpoint="logging") - juju.integrate(APP_NAME, f"{cos.model}.loki") - juju.wait(jubilant.all_active) - cos.wait(jubilant.all_active) +```{literalinclude} ../../../examples/k8s-5-observe/tests/integration/test_charm.py +:language: python +:pyobject: test_integrate_loki ``` ### Request logs from Loki Add a test function to `tests/integration/test_charm.py`: -```python -def test_loki_data(cos: jubilant.Juju): - """Use Loki's HTTP API to verify that Loki has a label for our app. - - COS Lite exposes Loki's API through the Traefik load balancer. Traefik comes with an action - that tells us the base URL of Loki's API. - """ - task = cos.run("traefik/0", "show-proxied-endpoints") - results = json.loads(task.results["proxied-endpoints"]) - loki_url = results["loki/0"]["url"] - loki_api_url = f"{loki_url}/loki/api/v1/label/juju_application/values" - juju_applications = _get_loki_logs(loki_api_url) - assert juju_applications is not None, "No logs available from Loki" - assert APP_NAME in juju_applications - - -def _get_loki_logs(loki_api_url: str) -> list[str] | None: - """Wait for logs to be available from Loki and return them.""" - for attempt in range(60): - if attempt: # If not the first attempt, wait before retrying. - time.sleep(1) - response = requests.get(loki_api_url) - if response.status_code == 200: - response_decoded = response.json() - if "data" in response_decoded: - return response_decoded["data"] - return None +```{literalinclude} ../../../examples/k8s-5-observe/tests/integration/test_charm.py +:language: python +:pyobject: test_loki_data +``` + +```{literalinclude} ../../../examples/k8s-5-observe/tests/integration/test_charm.py +:language: python +:pyobject: _get_loki_logs ``` For more information, see: diff --git a/docs/tutorial/write-your-first-machine-charm.md b/docs/tutorial/write-your-first-machine-charm.md index 68809fa62..8982ac764 100644 --- a/docs/tutorial/write-your-first-machine-charm.md +++ b/docs/tutorial/write-your-first-machine-charm.md @@ -190,11 +190,10 @@ These files currently contain placeholder code and configuration. It would be po Open `~/tinyproxy-tutorial/charmcraft.yaml` in your usual text editor or IDE, then change the values of `title`, `summary`, and `description` to: -```yaml -title: Reverse Proxy Demo -summary: A demo charm that configures tinyproxy as a reverse proxy. -description: | - This charm demonstrates how to write a machine charm with Ops. +```{literalinclude} ../../examples/machine-tinyproxy/charmcraft.yaml +:language: yaml +:start-at: 'title: Reverse Proxy Demo' +:end-at: how to write a machine charm with Ops. ``` ### Write a helper module @@ -219,104 +218,9 @@ This adds the following Python packages to the `dependencies` list in `pyproject Next, replace the contents of `src/tinyproxy.py` with: -```python -"""Functions for interacting with tinyproxy.""" - -import logging -import os -import shutil -import signal -import subprocess - -from charmlibs import apt, pathops - -logger = logging.getLogger(__name__) - -CONFIG_FILE = pathops.LocalPath("/etc/tinyproxy/tinyproxy.conf") -PID_FILE = pathops.LocalPath("/var/run/tinyproxy.pid") - - -def ensure_config(port: int, slug: str) -> bool: - """Ensure that tinyproxy is configured. Return True if any changes were made.""" - # For the config file format, see https://manpages.ubuntu.com/manpages/noble/en/man5/tinyproxy.conf.5.html - config = f"""\ -PidFile "{PID_FILE}" -Port {port} -Timeout 600 -ReverseOnly Yes -ReversePath "/{slug}/" "http://www.example.com/" -""" - return pathops.ensure_contents(CONFIG_FILE, config) - - -def get_version() -> str: - """Get the version of tinyproxy that is installed.""" - result = subprocess.run(["tinyproxy", "-v"], check=True, capture_output=True, text=True) - return result.stdout.removeprefix("tinyproxy").strip() - - -def install() -> None: - """Use APT to install the tinyproxy executable.""" - apt.update() - # Install a specific package from ubuntu@24.04 - # See https://packages.ubuntu.com/noble/tinyproxy-bin - # In general, it's good practice for charms to pin workload versions. - apt.add_package("tinyproxy-bin", "1.11.1-3") - # If this call fails, the charm will go into error status. The Juju logs will show the error: - # charmlibs.apt.PackageError: Failed to install packages: tinyproxy-bin - - -def is_installed() -> bool: - """Return whether the tinyproxy executable is available.""" - return shutil.which("tinyproxy") is not None - - -def is_running() -> bool: - """Return whether tinyproxy is running.""" - return bool(_get_pid()) - - -def reload_config() -> None: - """Ask tinyproxy to reload config.""" - pid = _get_pid() - if not pid: - raise RuntimeError("tinyproxy is not running") - # Sending signal SIGUSR1 doesn't terminate the process. It asks the process to reload config. - # See https://manpages.ubuntu.com/manpages/noble/en/man8/tinyproxy.8.html#signals - os.kill(pid, signal.SIGUSR1) - - -def start() -> None: - """Start tinyproxy.""" - subprocess.run(["tinyproxy"], check=True, capture_output=True, text=True) - - -def stop() -> None: - """Stop tinyproxy.""" - pid = _get_pid() - if pid: - os.kill(pid, signal.SIGTERM) - - -def uninstall() -> None: - """Uninstall the tinyproxy executable and remove files.""" - apt.remove_package("tinyproxy-bin") - PID_FILE.unlink(missing_ok=True) - CONFIG_FILE.unlink(missing_ok=True) - CONFIG_FILE.parent.rmdir() - - -def _get_pid() -> int | None: - """Return the PID of the tinyproxy process, or None if the process can't be found.""" - if not PID_FILE.exists(): - return None - pid = int(PID_FILE.read_text()) - try: - # Sending signal 0 doesn't terminate the process. It just checks whether the PID exists. - os.kill(pid, 0) - except ProcessLookupError: - return None - return pid +```{literalinclude} ../../examples/machine-tinyproxy/src/tinyproxy.py +:language: python +:start-at: '"""Functions for interacting with tinyproxy."""' ``` Notice that the helper module is stateless. In fact, your charm as a whole will be stateless. The main logic of your charm will: @@ -337,13 +241,9 @@ We'll now write the charm code that handles events from Juju. Charmcraft created In `src/charm.py`, replace the `_on_install` method of the charm class with: -```python - def _on_install(self, event: ops.InstallEvent) -> None: - """Install tinyproxy on the machine.""" - if not tinyproxy.is_installed(): - tinyproxy.install() - version = tinyproxy.get_version() - self.unit.set_workload_version(version) +```{literalinclude} ../../examples/machine-tinyproxy/src/charm.py +:language: python +:pyobject: TinyproxyCharm._on_install ``` When your charm receives the "install" event from Juju, Ops runs this method and tells Juju the version of tinyproxy that's installed on the machine. Juju shows the version in its status output. @@ -363,32 +263,25 @@ After deploying your charm, you'll use the `juju config` command to change the p In `charmcraft.yaml`, replace the `config` block with: -```yaml -config: - options: - slug: - description: "Configures the path of the reverse proxy. Must match the regex [a-z0-9-]+" - default: example - type: string +```{literalinclude} ../../examples/machine-tinyproxy/charmcraft.yaml +:language: yaml +:start-at: 'config:' +:end-at: 'type: string' ``` Then add the following class to `src/charm.py`: -```python -class TinyproxyConfig(pydantic.BaseModel): - """Schema for the charm's config options.""" - - slug: str = pydantic.Field( - "example", - pattern=r"^[a-z0-9-]+$", - description="Configures the path of the reverse proxy. Must match the regex [a-z0-9-]+", - ) +```{literalinclude} ../../examples/machine-tinyproxy/src/charm.py +:language: python +:pyobject: TinyproxyConfig ``` Also add the following line at the beginning of `src/charm.py`: -```python -import pydantic +```{literalinclude} ../../examples/machine-tinyproxy/src/charm.py +:language: python +:start-at: import pydantic +:end-at: import pydantic ``` The `TinyproxyConfig` class uses [Pydantic](https://docs.pydantic.dev) to specify how to validate values of `slug`. The class doesn't actually load the configured value of `slug`; we'll do that elsewhere in the charm code. @@ -409,65 +302,53 @@ Your charm now needs a way to load the value of the `slug` configuration option, In `src/charm.py`, add the following methods to the charm class: -```python - def configure_and_run(self) -> None: - """Ensure that tinyproxy is running with the correct config.""" - try: - config = self.load_config(TinyproxyConfig) - except pydantic.ValidationError: - # The collect-status handler will run next and will set status for the user to see. - return - if not tinyproxy.is_installed(): - return - changed = tinyproxy.ensure_config(PORT, config.slug) - if not tinyproxy.is_running(): - tinyproxy.start() - self.wait_for_running() - elif changed: - logger.info("Config changed while tinyproxy is running. Updating tinyproxy config") - tinyproxy.reload_config() - - def wait_for_running(self) -> None: - """Wait for tinyproxy to be running.""" - for _ in range(3): - if tinyproxy.is_running(): - return - time.sleep(1) - raise RuntimeError("tinyproxy was not running within the expected time") - # Raising a runtime error will put the charm into error status. - # The Juju logs will show the error message, to help you debug the error. +```{literalinclude} ../../examples/machine-tinyproxy/src/charm.py +:language: python +:pyobject: TinyproxyCharm.configure_and_run +``` + +```{literalinclude} ../../examples/machine-tinyproxy/src/charm.py +:language: python +:pyobject: TinyproxyCharm.wait_for_running ``` Then add the following lines at the beginning of `src/charm.py`: -```python -import time +```{literalinclude} ../../examples/machine-tinyproxy/src/charm.py +:language: python +:start-at: import time +:end-at: import time +``` -PORT = 8000 +```{literalinclude} ../../examples/machine-tinyproxy/src/charm.py +:language: python +:start-at: PORT = 8000 +:end-at: PORT = 8000 ``` The `configure_and_run` method ensures that tinyproxy is running and correctly configured, regardless of whether tinyproxy was already running. We can therefore use this method to handle two different Juju events: "start" and "config-changed". Replace the `_on_start` method of the charm class with: -```python - def _on_start(self, event: ops.StartEvent) -> None: - """Handle start event.""" - self.configure_and_run() +```{literalinclude} ../../examples/machine-tinyproxy/src/charm.py +:language: python +:pyobject: TinyproxyCharm._on_start ``` Next, add the following line to the `__init__` method of the charm class: -```python - framework.observe(self.on.config_changed, self._on_config_changed) +```{literalinclude} ../../examples/machine-tinyproxy/src/charm.py +:language: python +:start-at: framework.observe(self.on.config_changed +:end-at: framework.observe(self.on.config_changed +:dedent: ``` Then add the following method to the charm class: -```python - def _on_config_changed(self, event: ops.ConfigChangedEvent) -> None: - """Handle config-changed event.""" - self.configure_and_run() +```{literalinclude} ../../examples/machine-tinyproxy/src/charm.py +:language: python +:pyobject: TinyproxyCharm._on_config_changed ``` ### Report status to Juju @@ -478,27 +359,18 @@ One option would be to modify each method in the charm class to report an approp Add the following line to the `__init__` method of the charm class: -```python - framework.observe(self.on.collect_unit_status, self._on_collect_status) +```{literalinclude} ../../examples/machine-tinyproxy/src/charm.py +:language: python +:start-at: framework.observe(self.on.collect_unit_status +:end-at: framework.observe(self.on.collect_unit_status +:dedent: ``` Then add the following method to the charm class: -```python - def _on_collect_status(self, event: ops.CollectStatusEvent) -> None: - """Report the status of tinyproxy (runs after each event).""" - try: - self.load_config(TinyproxyConfig) - except pydantic.ValidationError as e: - (slug_error,) = e.errors() # 'slug' is the first and only option validated. - slug_value = slug_error["input"] - message = f"Invalid slug: '{slug_value}'. Slug must match the regex [a-z0-9-]+" - event.add_status(ops.BlockedStatus(message)) - if not tinyproxy.is_installed(): - event.add_status(ops.MaintenanceStatus("Waiting for tinyproxy to be installed")) - if not tinyproxy.is_running(): - event.add_status(ops.MaintenanceStatus("Waiting for tinyproxy to start")) - event.add_status(ops.ActiveStatus()) +```{literalinclude} ../../examples/machine-tinyproxy/src/charm.py +:language: python +:pyobject: TinyproxyCharm._on_collect_status ``` Ops runs this method after each Juju event, regardless of whether the charm code handles the event. After running the method, Ops decides which status to report to Juju, choosing the highest priority status that was proposed with `event.add_status`. @@ -511,30 +383,28 @@ If Juju wants to remove a unit of the application, your charm (on that unit) rec To handle these events, add the following lines to the `__init__` method of the charm class: -```python - framework.observe(self.on.stop, self._on_stop) - framework.observe(self.on.remove, self._on_remove) +```{literalinclude} ../../examples/machine-tinyproxy/src/charm.py +:language: python +:start-at: framework.observe(self.on.stop +:end-at: framework.observe(self.on.remove +:dedent: ``` Then add the following methods to the charm class: -```python - def _on_stop(self, event: ops.StopEvent) -> None: - """Handle stop event.""" - tinyproxy.stop() - self.wait_for_not_running() +```{literalinclude} ../../examples/machine-tinyproxy/src/charm.py +:language: python +:pyobject: TinyproxyCharm._on_stop +``` - def _on_remove(self, event: ops.RemoveEvent) -> None: - """Handle remove event.""" - tinyproxy.uninstall() +```{literalinclude} ../../examples/machine-tinyproxy/src/charm.py +:language: python +:pyobject: TinyproxyCharm._on_remove +``` - def wait_for_not_running(self) -> None: - """Wait for tinyproxy to not be running.""" - for _ in range(3): - if not tinyproxy.is_running(): - return - time.sleep(1) - raise RuntimeError("tinyproxy was still running after the expected time") +```{literalinclude} ../../examples/machine-tinyproxy/src/charm.py +:language: python +:pyobject: TinyproxyCharm.wait_for_not_running ``` That's all the charm code! If you'd like, you can [inspect the full code in GitHub](https://github.com/canonical/operator/tree/main/examples/machine-tinyproxy). @@ -684,24 +554,9 @@ When writing a charm, it's good practice to write unit tests for the charm code Create a file `tests/unit/test_tinyproxy.py` containing: -```python -import pytest - -from charm import tinyproxy - - -class MockVersionProcess: - """Mock object that represents the result of calling 'tinyproxy -v'.""" - - def __init__(self, version: str): - self.stdout = f"tinyproxy {version}" - - -def test_version(monkeypatch: pytest.MonkeyPatch): - """Test that the helper module correctly returns the version of tinyproxy.""" - version_process = MockVersionProcess("1.0.0") - monkeypatch.setattr("subprocess.run", lambda *args, **kwargs: version_process) - assert tinyproxy.get_version() == "1.0.0" +```{literalinclude} ../../examples/machine-tinyproxy/tests/unit/test_tinyproxy.py +:language: python +:start-at: import pytest ``` We'll run all the tests later in the tutorial. But if you'd like to see whether this test passes, run: @@ -724,154 +579,9 @@ It's helpful to think of each test this way: Replace the contents of `tests/unit/test_charm.py` with: -```python -import pytest -from ops import testing - -from charm import PORT, TinyproxyCharm - - -class MockTinyproxy: - """Mock object that represents tinyproxy.""" - - def __init__( - self, - config: None | tuple[int, str] = None, - installed: bool = False, - reloaded_config: bool = False, - running: bool = False, - ): - self.config = config - self.installed = installed - self.reloaded_config = reloaded_config - self.running = running - - def ensure_config(self, port: int, slug: str) -> bool: - old_config = self.config - self.config = (port, slug) - return self.config != old_config - - def get_version(self) -> str: - return "1.0.0" - - def install(self) -> None: - self.installed = True - - def is_installed(self) -> bool: - return self.installed - - def is_running(self) -> bool: - return self.running - - def start(self) -> None: - self.running = True - - def stop(self) -> None: - self.running = False - - def reload_config(self) -> None: - self.reloaded_config = True - - def uninstall(self) -> None: - self.installed = False - - -def test_install(monkeypatch: pytest.MonkeyPatch): - """Test that the charm correctly handles the install event.""" - # A state-transition test has three broad steps: - # Step 1. Arrange the input state. - tinyproxy = MockTinyproxy() - monkeypatch.setattr("charm.tinyproxy", tinyproxy) - ctx = testing.Context(TinyproxyCharm) - state_in = testing.State() - # Step 2. Simulate an event, in this case an install event. - state_out = ctx.run(ctx.on.install(), state_in) - # Step 3. Check the output state. - assert state_out.workload_version is not None - assert state_out.unit_status == testing.MaintenanceStatus("Waiting for tinyproxy to start") - assert tinyproxy.is_installed() - - -# For convenience, define a reusable fixture that provides a MockTinyproxy object -# and patches the helper module in the charm. -@pytest.fixture -def tinyproxy_installed(monkeypatch: pytest.MonkeyPatch): - tinyproxy = MockTinyproxy(installed=True) - monkeypatch.setattr("charm.tinyproxy", tinyproxy) - return tinyproxy - - -def test_start(tinyproxy_installed: MockTinyproxy): - """Test that the charm correctly handles the start event.""" - ctx = testing.Context(TinyproxyCharm) - state_out = ctx.run(ctx.on.start(), testing.State()) - assert state_out.unit_status == testing.ActiveStatus() - assert tinyproxy_installed.is_running() - assert tinyproxy_installed.config == (PORT, "example") - - -# Define another fixture, this time representing an installed, configured, and running tinyproxy. -@pytest.fixture -def tinyproxy_configured(monkeypatch: pytest.MonkeyPatch): - tinyproxy = MockTinyproxy(config=(PORT, "example"), installed=True, running=True) - monkeypatch.setattr("charm.tinyproxy", tinyproxy) - return tinyproxy - - -def test_config_changed(tinyproxy_configured: MockTinyproxy): - """Test that the charm correctly handles the config-changed event.""" - ctx = testing.Context(TinyproxyCharm) - state_in = testing.State(config={"slug": "foo"}) - state_out = ctx.run(ctx.on.config_changed(), state_in) - assert state_out.unit_status == testing.ActiveStatus() - assert tinyproxy_configured.is_running() - assert tinyproxy_configured.config == (PORT, "foo") - assert tinyproxy_configured.reloaded_config - - -@pytest.mark.parametrize("invalid_slug", ["", "foo_bar", "foo/bar"]) -def test_start_invalid_config(tinyproxy_installed: MockTinyproxy, invalid_slug: str): - """Test that the charm fails to start if the config is invalid.""" - ctx = testing.Context(TinyproxyCharm) - state_in = testing.State(config={"slug": invalid_slug}) - state_out = ctx.run(ctx.on.start(), state_in) - assert state_out.unit_status == testing.BlockedStatus( - f"Invalid slug: '{invalid_slug}'. Slug must match the regex [a-z0-9-]+" - ) - assert not tinyproxy_installed.is_running() - assert tinyproxy_installed.config is None - - -@pytest.mark.parametrize("invalid_slug", ["", "foo_bar", "foo/bar"]) -def test_config_changed_invalid_config(tinyproxy_configured: MockTinyproxy, invalid_slug: str): - """Test that the charm fails to change config if the config is invalid.""" - ctx = testing.Context(TinyproxyCharm) - state_in = testing.State(config={"slug": invalid_slug}) - state_out = ctx.run(ctx.on.config_changed(), state_in) - assert state_out.unit_status == testing.BlockedStatus( - f"Invalid slug: '{invalid_slug}'. Slug must match the regex [a-z0-9-]+" - ) - assert tinyproxy_configured.is_running() # tinyproxy should still be running... - assert tinyproxy_configured.config == (PORT, "example") # ...with the original config. - assert not tinyproxy_configured.reloaded_config - - -def test_stop(tinyproxy_configured: MockTinyproxy): - """Test that the charm correctly handles the stop event.""" - ctx = testing.Context(TinyproxyCharm) - state_out = ctx.run(ctx.on.stop(), testing.State()) - assert state_out.unit_status == testing.MaintenanceStatus("Waiting for tinyproxy to start") - assert not tinyproxy_configured.is_running() - - -def test_remove(tinyproxy_installed: MockTinyproxy): - """Test that the charm correctly handles the remove event.""" - ctx = testing.Context(TinyproxyCharm) - state_out = ctx.run(ctx.on.remove(), testing.State()) - assert state_out.unit_status == testing.MaintenanceStatus( - "Waiting for tinyproxy to be installed" - ) - assert not tinyproxy_installed.is_installed() +```{literalinclude} ../../examples/machine-tinyproxy/tests/unit/test_charm.py +:language: python +:start-at: import pytest ``` ### Run the tests @@ -907,16 +617,22 @@ When you created the initial version of your charm, Charmcraft included integrat In `tests/integration/test_charm.py`, change `juju.wait(jubilant.all_active)` to: -```python - juju.wait(jubilant.all_active, timeout=600) +```{literalinclude} ../../examples/machine-tinyproxy/tests/integration/test_charm.py +:language: python +:start-at: juju.wait(jubilant.all_active, timeout=600) +:end-at: juju.wait(jubilant.all_active, timeout=600) +:dedent: ``` This extends the duration that Jubilant waits for your charm to deploy, in case the integration tests run slowly in your virtual machine. The default duration would be sufficient if the integration tests were running in a continuous integration environment. Next, remove the `@pytest.mark.skip` decorator from `test_workload_version_is_set`. Then change `assert version == ...` to: -```python - assert version == "1.11.1" # The version installed by tinyproxy.install. +```{literalinclude} ../../examples/machine-tinyproxy/tests/integration/test_charm.py +:language: python +:start-at: 'assert version == "1.11.1"' +:end-at: 'assert version == "1.11.1"' +:dedent: ``` You should now have the following tests: @@ -928,12 +644,9 @@ Before running the tests, let's add a test to check that an invalid value of `sl Add the following function at the end of `tests/integration/test_charm.py`: -```python -def test_block_on_invalid_config(charm: pathlib.Path, juju: jubilant.Juju): - """Check that the charm goes into blocked status if slug is invalid.""" - juju.config("tinyproxy", {"slug": "foo/bar"}) - juju.wait(jubilant.all_blocked) - juju.config("tinyproxy", reset="slug") +```{literalinclude} ../../examples/machine-tinyproxy/tests/integration/test_charm.py +:language: python +:pyobject: test_block_on_invalid_config ``` Each test depends on two fixtures: