Skip to content

Use latin1 encoding for layer names - #3

Open
ErikWegner wants to merge 1 commit into
ducflair:mainfrom
ErikWegner:main
Open

ErikWegner wants to merge 1 commit into
ducflair:mainfrom
ErikWegner:main

Conversation

@ErikWegner

Copy link
Copy Markdown

No description provided.

@jorgedanisc

jorgedanisc commented Aug 22, 2026

Copy link
Copy Markdown
Contributor

Thanks for submitting this fix, @ErikWegner! Sorry for the delay, for some reason this didn't got into my inbox

You correctly identified that passing raw UTF-8 bytes to Object::string_literal causes encoding issues/mojibake in PDF viewers. PDF text strings are expected to use either PDFDocEncoding or a Unicode encoding recognized by the PDF specification, rather than raw UTF-8.

However, encode_latin1_lossy has two drawbacks:

  1. It is lossy for characters outside ISO-8859-1, so Eastern European, Cyrillic, Greek, CJK, Arabic, emojis, etc. can become ?.
  2. It adds an extra encoding_rs dependency.

According to ISO 32000-1 §7.9.2.2, Unicode PDF text strings can be encoded as UTF-16BE with a FE FF byte-order marker.

We can therefore support arbitrary Unicode without an additional dependency:

fn encode_pdf_text_string(s: &str) -> Vec<u8> {
    if s.is_ascii() {
        s.as_bytes().to_vec()
    } else {
        let mut bytes = Vec::with_capacity(2 + s.len() * 2);
        bytes.extend_from_slice(&[0xFE, 0xFF]);

        for code_unit in s.encode_utf16() {
            bytes.extend_from_slice(&code_unit.to_be_bytes());
        }

        bytes
    }
}

Would you be open to updating the PR to use the UTF-16BE BOM approach and dropping encoding_rs from Cargo.toml?

@jorgedanisc

Copy link
Copy Markdown
Contributor

We should also add a small test for this so we can validate the behavior and prevent regressions.

Something covering a few representative names such as Café, Łódź, 日本語, and 😀 should be enough.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants