# PDF metadata from translations \GetTranslation produces incorrect chararcter encoding?

While coding my own class I use \GetTranslation from the translations package to allow using different languages and \hypersetup from hyperref to set PDF metadata.

If I use \GetTranslation in the document, utf8 special characters are encoded correctly. However if I use \GetTranslation for PDF metadata, utf8 special characters appear as incorrectly interpreted as ISO 8859-1 characters, e.g. Ã¤ instead of ä. If I directly use utf8 special characters for PDF metadata, they are correctly encoded.

The following MWE produces Töst or Täst in the document but Töst or TÃ¤st in the PDF title.

Why is that and how can I fix it so that special characters from \GetTranslation are correctly encoded in PDF metadata?

Using \"a instead of ä in \NewTranslation produces a warning (“Token not allowed in a PDF string (Unicode), removing \" ”) and Töst or Tast remains as the PDF title.

MWE

\documentclass[ngerman]{article}

\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}

\usepackage{babel}

\usepackage{translations}
\DeclareTranslationFallback{test}{Test}
\NewTranslation{ngerman}{test}{Täst}

\usepackage[pdfencoding=auto,psdextra,pdftex]{hyperref}

\AtBeginDocument{
\hypersetup{
pdftitle = {\mytitle}
}
}

\newcommand{\mytitle}{Töst or \GetTranslation{test}}

\begin{document}
\mytitle
\end{document}

• Did you try the unicode option of hyperref package? Nov 15, 2019 at 13:00
• @PaulGaborit does not seem to work, the pdf meta seems to see the ä as two chars. Nov 15, 2019 at 13:07
• Yes, adding the unicode option does not change anything in the output (as far as I can see) Nov 15, 2019 at 13:08

the translations package surrounds the output by a \unexpanded and this stops the hyperref processing:

\newcommand*\@trnslt@translation@value[2]{%
\unexpanded\expandafter\expandafter\expandafter{%
\csname @trnslt@word@\detokenize{#1}@#2\endcsname
}%
}


You can try an variant of the \GetTranslation that avoid this (I'm not quite sure if I got the language part right, it could perhaps fails for unknown translations). If the text of the translation contains fragile commands you will perhaps to use \texorpdfstring if you use it in section titles:

\documentclass[ngerman]{article}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}

\usepackage{babel}

\usepackage{translations}
\DeclareTranslationFallback{test}{Test}
\NewTranslation{ngerman}{test}{Täst}

\usepackage[pdfencoding=auto,psdextra]{hyperref}

\makeatletter
\newcommand\GetTranslationPDF[1]{%
\csname @trnslt@word@\detokenize{#1}@\@trnslt@language{\@trnslt@current@language}\endcsname
}
\makeatother
\AtBeginDocument{
\hypersetup{
pdftitle = {\mytitle}
}
}

\newcommand{\mytitle}{\GetTranslationPDF{test}}

\begin{document}
\tableofcontents
\section{\GetTranslationPDF{test}}

\end{document}


## Edit

the following could work als for language without translation. But it would be better to make a feature request to the package author

\documentclass[ngerman]{article}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}

\usepackage{babel}

\usepackage{translations}

\DeclareTranslationFallback{test}{Test}
\NewTranslation{ngerman}{test}{Täst}

\usepackage[pdfencoding=auto,psdextra,pdftex]{hyperref}

\makeatletter
\newcommand*\@trnslt@checkandget@translation@forPDF[2]{%
\@trnslt@if@translation{#1}{#2}
{\@trnslt@get@translation@forPDF{#1}{#2}}
{%
\@trnslt@if@translation{#1}{fallback}
{\@trnslt@translation@valuePDF{#1}{fallback}}
{\@trnslt@translation@valuePDF{#1}{literal}}%
}%
}

\newcommand*\@trnslt@get@translation@forPDF[2]{%
\@trnslt@if@dialect{#2}
{%
\@trnslt@if@translation@valuePDF{#1}{\@trnslt@dialect{#2}}
{\@trnslt@translation@valuePDF{#1}{\@trnslt@dialect{#2}}}
{\@trnslt@translation@valuePDF{#1}{\@trnslt@dialect@of{#2}}}%
}
{\@trnslt@translation@valuePDF{#1}{\@trnslt@language{#2}}}%
}
\newcommand*\@trnslt@translation@valuePDF[2]{%
\csname @trnslt@word@\detokenize{#1}@#2\endcsname
}

\newcommand*\GetTranslationPDF[1]{%
\@trnslt@checkandget@translation@forPDF{#1}{\@trnslt@current@language}}

\makeatother
\AtBeginDocument{
\hypersetup{
pdftitle = {\mytitle}
}
}

\newcommand{\mytitle}{\GetTranslationPDF{test}}

\begin{document}
\tableofcontents
\section{\GetTranslationPDF{test}}

\end{document}

• Thank you very much for this answer! Your solution appears to work as long as there is an explicit translation. Unfortunately \GetTranslationPDF produces no output (neither in the document nor in the PDF metadata) if the default translation is used (e.g. by requiring french without let’s say \NewTranslation{french}{test}{Tèst}), because hyperref removes the result (“Package hyperref Warning: Token not allowed in a PDF string (Unicode): (hyperref) removing \@trnslt@word@test@french' ”) and \@trnslt@word@test@french is empty Nov 15, 2019 at 13:32
• As I said, I haven't really looked at the language part. The best would be to make a feature request so that the package adds a version without the unexpanded. Nov 15, 2019 at 14:31
• Sure, and it isn’t a problem if the \csname … \endcsname is wrapped with \@trnslt@if@translation and you require fallback or literal otherwise. Which leads me to the question why there is an \unexpanded in the first place? In what situations would you need it? Nov 15, 2019 at 14:35
• Can you make an example where it fails so that I can test another code?# Nov 15, 2019 at 14:40
• I added an edit - perhaps it works. Nov 15, 2019 at 14:49