# Latex3: Unicode (utf8) string, weird output with pdflatex

I want to store text that contains non-ascii characters in latex3 string variables. With xelatex and lualatex it works well but With pdflatex next mwe produces incorrect output. For legacy reasons switching to xelatex or lualatex isn't that easy on the project that I'm working with, so would it be possible to make next mwe work correctly?

MWE:

\documentclass{article}

\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{xparse}

\ExplSyntaxOn

\NewDocumentCommand{\savestring}{mm}{%
\str_clear_new:c{g_mwe_#1_str}
\str_gset:cn{g_mwe_#1_str}{#2}
}

\NewDocumentCommand{\printstring}{m}{%
\str_use:c{g_mwe_#1_str}
}

\ExplSyntaxOff

\begin{document}
\savestring{umlaut}{hääyöaie}

Hääyöaie

\printstring{umlaut}
\end{document}


Result is:

• You can't store UTF-8 characters as strings, as you discovered. Why not token lists? – egreg Jul 8 '18 at 18:50
• latex3 strings are catcode12 characters (like punctuation) and spaces. inputenc needs active (catcode13) characters to decode UTF-8. – David Carlisle Jul 8 '18 at 19:06
• There's no problem if you use token lists - I do this all the time. (And I only use pdfTeX.) – cfr Jul 8 '18 at 21:18

Strings are “detokenized”: every character in them has category code 12, except for spaces that have catcode 10.

You can rescan the string:

\documentclass{article}

\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{xparse}

\ExplSyntaxOn

\NewDocumentCommand{\savestring}{mm}
{
\str_clear_new:c{g_mwe_#1_str}
\str_gset:cn{g_mwe_#1_str}{#2}
}

\NewDocumentCommand{\printstring}{m}
{
\tl_rescan:nv {} {g_mwe_#1_str}
}
\cs_generate_variant:Nn \tl_rescan:nn { nv }

\ExplSyntaxOff

\begin{document}
\savestring{umlaut}{hääyöaie}

Hääyöaie

\printstring{umlaut}
\end{document}


but what you really want is saving the tokens in token list variables.

\documentclass{article}

\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{xparse}

\ExplSyntaxOn

\NewDocumentCommand{\savestring}{mm}
{
\tl_clear_new:c {g_mwe_#1_tl}
\tl_gset:cn {g_mwe_#1_tl}{#2}
}

\NewDocumentCommand{\printstring}{m}
{
\tl_use:c {g_mwe_#1_tl}
}

\ExplSyntaxOff

\begin{document}
\savestring{umlaut}{hääyöaie}

Hääyöaie

\printstring{umlaut}
\end{document}