# Write accented characters to file using \write

I am trying to output some information to an auxiliary file using \write and among other things I have authors names that often include accented letters. If I use the string explicitly, the accented letters are expanded into TeX commands, but I can solve this by using \unexpanded. However, if the string is contained in a variable, and I use \unexpanded, I get the name of the variable instead of its content. The command \unexpanded works fine again if the variable is a command parameter, for example #1.

I don't have an understanding of expansion that is clear enough to sort this out, so any help is appreciated. All I want to do is to write the content of a variable holding a string with accented letters to the text file.

Here is a minimal example:

\documentclass{article}
\usepackage[utf8]{inputenc}

\newwrite\file
\immediate\openout\file=\jobname_extra.txt
\AtEndDocument{\closeout\file}

\begin{document}
\immediate\write\file{Test 1}                 % works fine
\immediate\write\file{José}                   % expands into TeX code
\immediate\write\file{\unexpanded{José}}      % works fine
\def\mystring{José}
\immediate\write\file{\mystring}              % expands into TeX code
\immediate\write\file{\unexpanded{\mystring}} % outputs \mystring literally
\newcommand{\writeaccented}[1]{
\immediate\write\file{\unexpanded{#1}}
}
\writeaccented{José}                          % works fine
\end{document}


EDIT: I received two answers to use either \expandafter\unexpanded\expandafter{\mystring} or \detokenize\expandafter{\mystring}. These work fine on the minimal example above, but not on the real example, making me think that the expansion in my case is happening earlier. Here is the new minimal example that is closer to my real case:

\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{xstring}

\newwrite\file
\immediate\openout\file=\jobname_extra.txt
\AtEndDocument{\closeout\file}

\begin{document}
\def\listofauthors{José Saramago, Luigi Pirandello and Władysław Reymont}
\makeatletter
\StrSubstitute[0]{\listofauthors}{, }{,}[\@cpauthors]
\StrSubstitute[0]{\@cpauthors}{ and }{,}[\@cpauthors]
\@for \@cpauthor:=\@cpauthors\do{%
\immediate\write\file{<author>\expandafter\unexpanded\expandafter{\@cpauthor}</author>}
}
\makeatother
\end{document}


EDIT 2: adding \usepackage[T1]{fontenc} prevents the expansion into TeX commands, but outputs accented letters in ISO-8859 encoding instead of unicode. I tried both with and without the proposed solutions.

EDIT 3: the solution that worked for me is in https://tex.stackexchange.com/a/390119/36823. Thank you!

• You can try \protected@write{\file}{}{content} but that will only be done after a shipout of page. – user31729 Sep 6 '17 at 15:18
• Welcome to TeX.SX! Usually, we don't put a greeting or a “thank you” in our posts. While this might seem strange at first, it is not a sign of lack of politeness, but rather part of our trying to keep everything very concise. Accepting and upvoting answers is the preferred way here to say “thank you” to users who helped you. – dexteritas Sep 6 '17 at 15:19
• Crosspost – Johannes_B Sep 6 '17 at 16:29
• 'Of course' \StrSubstitute must expand something before it can store it to another macro. – user31729 Sep 6 '17 at 16:53
• I adde a more general solution based on the edit – egreg Sep 7 '17 at 8:45

You can use \detokenize:

\documentclass{article}
\usepackage[utf8]{inputenc}

\newwrite\file
\immediate\openout\file=\jobname_extra.txt
\AtEndDocument{\closeout\file}

\begin{document}

\def\mystring{José}

\immediate\write\file{\detokenize\expandafter{\mystring}} % outputs \mystring literally

\end{document}


The contents of the _extra file is

José


The hex dump:

0000000 4a 6f 73 c3 a9 0a
0000006


The UTF-8 representation of é is indeed C3 A9.

A different approach for the more complicated situation:

\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{xparse}

\ExplSyntaxOn
\iow_new:N \g_giampiero_output_stream
\tl_new:N \l__giampiero_write_temp_tl

\AtBeginDocument
{
\iow_open:Nn \g_giampiero_output_stream { \c_sys_jobname_str _extra.txt }
}
\AtEndDocument
{
\iow_close:N \g_giampiero_output_stream
}

\NewDocumentCommand{\writeauthors}{sm}
{% *-version for doing a variable
\IfBooleanTF{#1}
{
\giampiero_writeauthors:V #2
}
{
\giampiero_writeauthors:n { #2 }
}
}

\cs_new_protected:Nn \giampiero_writeauthors:n
{
\tl_set:Nn \l__giampiero_write_temp_tl { #1 }
\tl_replace_all:Nnn \l__giampiero_write_temp_tl { ~and~ } { , }
\seq_set_split:NnV \l__giampiero_write_temp_seq { , } \l__giampiero_write_temp_tl
\seq_map_inline:Nn \l__giampiero_write_temp_seq
{
\giampiero_write_generic:nn { author } { ##1 }
}
}
% This can be used for other similar write operations
\cs_new_protected:Nn \giampiero_write_generic:nn
{
\iow_now:Nn \g_giampiero_output_stream { <#1>#2</#1> }
}

\cs_generate_variant:Nn \seq_set_split:Nnn {NnV}
\cs_generate_variant:Nn \giampiero_writeauthors:n { V }
\ExplSyntaxOff

\newcommand\listofauthors{José Saramago, Luigi Pirandello and Władysław Reymont}

\begin{document}

\writeauthors{José Saramago, Luigi Pirandello and Władysław Reymont}

\writeauthors*{\listofauthors}

\end{document}


Here is the contents of the generated file:

<author>José Saramago</author>
<author>Luigi Pirandello</author>
<author>José Saramago</author>
<author>Luigi Pirandello</author>


Expansion is a weird beast in TeX ;-)

\immediate\write\file{\expandafter\unexpanded\expandafter{\mystring}} %


will expand the content of \mystring first before \unexpanded can 'protect' it again.

\documentclass{article}
\usepackage[utf8]{inputenc}

\newwrite\file
\immediate\openout\file=\jobname_extra.txt
\AtEndDocument{\closeout\file}

\begin{document}
\immediate\write\file{Test 1}                 % works fine
\immediate\write\file{José}                   % expands into TeX code
\immediate\write\file{\unexpanded{José}}      % works fine
\def\mystring{José}
\immediate\write\file{\mystring}              % expands into TeX code
\immediate\write\file{\expandafter\unexpanded\expandafter{\mystring}} % outputs \mystring literally
\newcommand{\writeaccented}[1]{
\immediate\write\file{\unexpanded{#1}}
}
\writeaccented{José}                          % works fine
\end{document}


Output: (%%%% annotation by me)

Test 1
Jos\unhbox \voidb@x \bgroup \let \unhbox \voidb@x \setbox \@tempboxa \hbox {e\global \mathchardef \accent@spacefactor \spacefactor }\accent 19 e\egroup \spacefactor \accent@spacefactor
José
Jos\unhbox \voidb@x \bgroup \let \unhbox \voidb@x \setbox \@tempboxa \hbox {e\global \mathchardef \accent@spacefactor \spacefactor }\accent 19 e\egroup \spacefactor \accent@spacefactor
José%%%%% This one is from \expandafter\unexpanded\expandafter{\mystring}
José

• \expandafter\unexpanded\expandafter can be shortened to \unexpanded\expandafter (i.e. the first one is not needed). Same with \detokenize. – user4686 Sep 6 '17 at 17:22

# Answer to example with xstring

Package xstring knows several expansion modes, see section "3.1 Expansion of arguments" of the documentation.

In this case, \listofauthors should contain the plain author names as given in the example. Then, mode \expandarg just expands the first token, macro \listofauthors, but without further expansion.

The example limits the effect of \expandarg by a group. It uses \detokenize and removes a unnecessary \expandafter.

\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{xstring}

\newwrite\file
\immediate\openout\file=\jobname_extra.txt
\AtEndDocument{\closeout\file}

\begin{document}
\def\listofauthors{José Saramago, Luigi Pirandello and Władysław Reymont}
\makeatletter
\begingroup
\expandarg
\StrSubstitute[0]{\listofauthors}{, }{,}[\@cpauthors]
\StrSubstitute[0]{\@cpauthors}{ and }{,}[\@cpauthors]
\@for \@cpauthor:=\@cpauthors\do{%
\immediate\write\file{<author>\detokenize\expandafter{\@cpauthor}</author>}
}
\endgroup
\makeatother
\end{document}


File \jobname_extra.txt:

<author>José Saramago</author>
<author>Luigi Pirandello</author>