Newlines may be rendered as literal \n for multi-line scalar variables in some circumstances

[kartsm]

Template file bar.jinja:

{% for k, v in site.quux.items() %}
{{ k }}
{{ v.garply }}
{% endfor %}

State file bar.sls:

/tmp/bar:
  file.managed:
    - user: root
    - group: root
    - mode: 644
    - source: salt://bar.jinja
    - template: jinja
    - context:
        site:
          quux:
            blurfl:
              garply: |
                corge
                wibble
                wobble

State file baz.sls:

{% import_yaml "baz.yaml" as site %}
/tmp/baz:
  file.managed:
    - user: root
    - group: root
    - mode: 644
    - source: salt://bar.jinja
    - template: jinja
    - context:
        site: {{ site }}

And imitation pillar data baz.yaml:

quux:
  blurfl:
    garply: |
      corge
      wibble
      wobble

So let's run these two states:

[root@l-tsmsalt-101 states]# salt '*salt*' state.sls bar
[root@l-tsmsalt-101 states]# salt '*salt*' state.sls baz

And compare output.

[root@l-tsmsalt-101 states]# diff -u /tmp/bar /tmp/baz
--- /tmp/bar    2016-01-27 23:16:01.839356994 +0000
+++ /tmp/baz    2016-01-27 23:16:10.282133493 +0000
@@ -1,7 +1,4 @@

 blurfl
-corge
-wibble
-wobble
-
+corge\nwibble\nwobble

It looks like it makes a rather large difference how the variable site is passed into the template. Since import_yaml is a Salt thing, and not a Jinja thing, I'm wondering if the newlines are getting mangled there.

I'm trying to take advantage of the public apache-formula. It encourages you to store information about all your sites in the pillar (thus "imitation pillar data" above). It allows arbitrary bits of config to be passed as multi-line scalars, for when the provided templates don't do what you want. Look for Formula_Append. I'm not interested in Salting every possible bit of Apache configuration syntax when I only need to use this feature occasionally. Only 3 sites out of several dozen need it, and only for about 4 short lines at a time.

About the machine where my tests were run:

[root@l-tsmsalt-101 states]# uname -a
Linux l-tsmsalt-101.bhyve.local 2.6.32-573.7.1.el6.x86_64 #1 SMP Tue Sep 22 22:00:00 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
[root@l-tsmsalt-101 states]# cat /etc/centos-release
CentOS release 6.7 (Final)
[root@l-tsmsalt-101 states]# cat /etc/system-release-cpe
cpe:/o:centos:linux:6:GA
[root@l-tsmsalt-101 states]# salt --version
salt 2015.8.3 (Beryllium)
[root@l-tsmsalt-101 states]# python --version
Python 2.6.6

Salt was installed via salt-bootstrap on 19 January 2016, if that helps any.

[jfindlay]

@kartsm, thanks for the report.

[stefansundin]

I'm seeing this issue in 2014.7.0rc2 as well.

[JonathanThorpe]

Still appears to be happening with 2016.3.3+ds-1 and Jinja 2.7.3-1 .

Gist of how I reproduce: https://gist.github.com/JonathanThorpe/c5b5ad5853c3465e1bbc9765030a4b5d

[ppmathis]

Experiencing the same issue with Salt 2016.3.3 as @JonathanThorpe
A temporary, but ugly workaround: {{ variable|replace('\\n', '\n') }}

[Armadill0]

Still present in 2016.11.

@ppmathis Thanks for the workaround.

[feinu]

Still present in 2017.7.0.

An alternative workaround is to use {%- set foo = salt.pillar.get('foo') -%} in the template itself. It's still ugly as it's not using context, but at least it won't accidentally mangle anything.

[hkbakke]

To avoid messing with the template I do this instead:

- context:
    # Pipe via json to avoid issue where newlines are mangled to literal \n
    # for multiline strings (https://github.com/saltstack/salt/issues/30690)
    config: {{ config|json }}

[ppmathis]

I've discovered that even using {{ config|yaml }} is sufficient within defaults or context to not mess with any new lines - all the content ends up correctly within the template.

[SEJeff]

@ppmathis I was literally about the say the exact same thing as I ran into this problem as well.

The correct solution is to use the |yaml jinja filter. If you want the exact formatting you can also disable the flow control via |yaml(None), depending on your use case.

cc: @cachedout this issue should be closed with some user documentation or something.