From a4da05f00dfcde1e53ea1dbfb03eb68384006188 Mon Sep 17 00:00:00 2001 From: Jan Pazdziora Date: Fri, 29 May 2026 09:37:52 +0200 Subject: [PATCH 1/3] Document the imports/drop-in approach for configuring containerd (#566) Signed-off-by: Jan Pazdziora --- docs/advanced.md | 53 +++++++++++++++++++++++++++++++++--------------- 1 file changed, 37 insertions(+), 16 deletions(-) diff --git a/docs/advanced.md b/docs/advanced.md index 31c0cbe42c..e89495780a 100644 --- a/docs/advanced.md +++ b/docs/advanced.md @@ -168,28 +168,36 @@ sudo etcdctl version \ ## Configuring containerd -:::info Version Gate -K3s includes containerd 2.0 as of the February 2025 releases: v1.31.6+k3s1 and v1.32.2+k3s1. -Be aware that containerd 2.0 prefers config version 3, while containerd 1.7 prefers config version 2. -::: +K3s will generate a configuration file for containerd at `/var/lib/rancher/k3s/agent/etc/containerd/config.toml` using values specific to the current cluster and node configuration. + +Starting with February 2025 releases, K3s includes containerd 2+ and the configuration file declares and uses the `version = 3` format. +Starting with April 2026 K3s releases, the default generated `config.toml` includes `imports` directive with the `config-v3.toml.d/*.toml` path. -K3s will generate a configuration file for containerd at `/var/lib/rancher/k3s/agent/etc/containerd/config.toml`, using values specific to the current cluster and node configuration. +So with recent K3s versions, the simplest way to tweak specific configuration values while keeping the K3s-provided defaults is to create the `/var/lib/rancher/k3s/agent/etc/containerd/config-v3.toml.d` directory and put a drop-in file with just the values you wish to change there: + +```toml title="/var/lib/rancher/k3s/agent/etc/containerd/config-v3.toml.d/cgroups.toml" +version = 3 +[plugins.'io.containerd.cri.v1.runtime'.containerd.runtimes.runc] + cgroup_writable = true +``` -For advanced customization, you can create a containerd config template in the same directory: -* For containerd 2.0, place a version 3 configuration template in `config-v3.toml.tmpl` - See the [containerd 2.0 documentation](https://github.com/containerd/containerd/blob/release/2.0/docs/cri/config.md) for more information. -* For containerd 1.7 and earlier, place a version 2 configuration template in `config.toml.tmpl` - See the [containerd 1.7 documentation](https://github.com/containerd/containerd/blob/release/1.7/docs/cri/config.md) for more information. +To quote the [containerd-config.toml man page(5)](https://containerd.io/docs/2.3/man/containerd-config.toml.5/): -Containerd 2.0 is backwards compatible with prior config versions, and k3s will continue to render legacy version 2 configuration from `config.toml.tmpl` if `config-v3.toml.tmpl` is not found. +> Imported files will overwrite simple fields like `int` or `string` (if not empty) and will append `array` and `map` fields. +If you wish to provide a full containerd configuration file instead of the default one, you can create a containerd config template `config-v3.toml.tmpl` in `/var/lib/rancher/k3s/agent/etc/containerd`. The template file is rendered into the containerd config using the [`text/template`](https://pkg.go.dev/text/template) library. -See `ContainerdConfigTemplateV3` and `ContainerdConfigTemplate` in [`templates.go`](https://github.com/k3s-io/k3s/blob/main/pkg/agent/templates/templates.go) for the default template content. +Search for `ContainerdConfigTemplateV3` in [`templates.go`](https://github.com/k3s-io/k3s/blob/main/pkg/agent/templates/templates.go) for the default template content. The template is executed with a [`ContainerdConfig`](https://github.com/k3s-io/k3s/blob/main/pkg/agent/templates/templates.go#L22-L33) struct as its dot value (data argument). +For best results and to get your future cluster and node configuration changes reflected properly, do NOT simply copy a prerendered `config.toml` with your changes into the template location, provide the template file with template markup. + +:::info Older K3s versions +With older K3s releases which contain containerd 1.7, the expected config version is 2 and the template file used is `config.toml.tmpl` instead of `config-v3.toml.tmpl`. +::: ### Base template -You can extend the K3s base template instead of copy-pasting the complete stock template out of the K3s source code. This is useful if you only need to build on the existing configuration by adding a few extra lines before or after the defaults. +If you have K3s version which does not include the `imports` directive by default and the drop-in config files cannot be used, you can extend the K3s base template instead of copy-pasting the complete stock template out of the K3s source code. This is useful if you only need to build on the existing configuration by adding a few extra lines before or after the defaults. ```toml title="/var/lib/rancher/k3s/agent/etc/containerd/config-v3.toml.tmpl" {{ template "base" . }} @@ -201,9 +209,22 @@ You can extend the K3s base template instead of copy-pasting the complete stock SystemdCgroup = true ``` -:::warning -For best results, do NOT simply copy a prerendered `config.toml` into the template and make your desired changes. Use the base template, or provide a full template based on the k3s defaults linked above. -::: +This approach however can only be used to add sections and values that do not exist in the base template at all. For example, attempts to configure values in `[plugins.'io.containerd.cri.v1.runtime'.containerd.runtimes.runc]` will cause containerd to fail with +``` +containerd: failed to unmarshal TOML: toml: table runc already exists +``` +error in `/var/lib/rancher/k3s/agent/containerd/containerd.log`. + +You can amend the default base template with the `imports` directive to enable the drop-in config file setup even on older K3s versions: +```toml title="/var/lib/rancher/k3s/agent/etc/containerd/config-v3.toml.tmpl" +imports = ["/var/lib/rancher/k3s/agent/etc/containerd/config-v3.toml.d/*.toml"] +{{ template "base" . }} +``` +Note however that once you upgrade to newer K3s version which has the `imports` in the base template by default, you will get an error +``` +containerd: failed to unmarshal TOML: toml: key imports is already defined +``` +and will need to remove that directive (or the whole template file and just use the default drop-in setup). ## Alternative Container Runtime Support From b24738232646a79a73c0b12aece904f7dc92e1f7 Mon Sep 17 00:00:00 2001 From: Jan Pazdziora Date: Wed, 3 Jun 2026 18:50:15 +0200 Subject: [PATCH 2/3] Add back the Version Gate infobox. --- docs/advanced.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/advanced.md b/docs/advanced.md index e89495780a..59618c5456 100644 --- a/docs/advanced.md +++ b/docs/advanced.md @@ -170,8 +170,10 @@ sudo etcdctl version \ K3s will generate a configuration file for containerd at `/var/lib/rancher/k3s/agent/etc/containerd/config.toml` using values specific to the current cluster and node configuration. +:::info Version Gate Starting with February 2025 releases, K3s includes containerd 2+ and the configuration file declares and uses the `version = 3` format. Starting with April 2026 K3s releases, the default generated `config.toml` includes `imports` directive with the `config-v3.toml.d/*.toml` path. +::: So with recent K3s versions, the simplest way to tweak specific configuration values while keeping the K3s-provided defaults is to create the `/var/lib/rancher/k3s/agent/etc/containerd/config-v3.toml.d` directory and put a drop-in file with just the values you wish to change there: From e3f903b8a162f34493df56e3746ff4caf58d887f Mon Sep 17 00:00:00 2001 From: Jan Pazdziora Date: Wed, 3 Jun 2026 18:54:15 +0200 Subject: [PATCH 3/3] Use the original wording. --- docs/advanced.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/docs/advanced.md b/docs/advanced.md index 59618c5456..21cfc818c4 100644 --- a/docs/advanced.md +++ b/docs/advanced.md @@ -191,7 +191,10 @@ If you wish to provide a full containerd configuration file instead of the defau The template file is rendered into the containerd config using the [`text/template`](https://pkg.go.dev/text/template) library. Search for `ContainerdConfigTemplateV3` in [`templates.go`](https://github.com/k3s-io/k3s/blob/main/pkg/agent/templates/templates.go) for the default template content. The template is executed with a [`ContainerdConfig`](https://github.com/k3s-io/k3s/blob/main/pkg/agent/templates/templates.go#L22-L33) struct as its dot value (data argument). -For best results and to get your future cluster and node configuration changes reflected properly, do NOT simply copy a prerendered `config.toml` with your changes into the template location, provide the template file with template markup. + +:::warning +For best results, do NOT simply copy a prerendered `config.toml` into the template and make your desired changes. Use the base template, or provide a full template based on the k3s defaults linked above. +::: :::info Older K3s versions With older K3s releases which contain containerd 1.7, the expected config version is 2 and the template file used is `config.toml.tmpl` instead of `config-v3.toml.tmpl`.