6

I'm attempting to automatically generate a BiBTeX file from within LaTeX. I'd like to expand all control sequences, but disable any active character expansion in the output. The idea is that the following pseudo-code:

\documentclass{article}
\usepackage[utf8]{inputenc}
\begin{document}
  \def\macro{ěščř}
  \output{\jobname.output}{\macro žýáíé}
\end{document}

should generate a file containing ěščřžýáíé in UTF-8.

Plain TeX

I first arrived at this Plain TeX solution:

\documentclass{article}
\usepackage[utf8]{inputenc}
\begin{document}
  \def\macro{ěščř}
  \immediate\newwrite\fd
  \immediate\openout\fd=\jobname.output
  \immediate\write\fd{\macro žýáíé}
  \immediate\closeout\fd
\end{document}

This generated a file containing

\unhbox \voidb@x \bgroup \let \unhbox \voidb@x \setbox \@tempboxa \hbox {e\global \mathchardef \accent@spacefactor \spacefactor }\accent 20 e\egroup \spacefactor \accent@spacefactor \unhbox \voidb@x \bgroup \let \unhbox \voidb@x \setbox \@tempboxa \hbox {s\global \mathchardef \accent@spacefactor \spacefactor }\accent 20 s\egroup \spacefactor \accent@spacefactor \unhbox \voidb@x \bgroup \let \unhbox \voidb@x \setbox \@tempboxa \hbox {c\global \mathchardef \accent@spacefactor \spacefactor }\accent 20 c\egroup \spacefactor \accent@spacefactor \unhbox \voidb@x \bgroup \let \unhbox \voidb@x \setbox \@tempboxa \hbox {r\global \mathchardef \accent@spacefactor \spacefactor }\accent 20 r\egroup \spacefactor \accent@spacefactor \unhbox \voidb@x \bgroup \let \unhbox \voidb@x \setbox \@tempboxa \hbox {z\global \mathchardef \accent@spacefactor \spacefactor }\accent 20 z\egroup \spacefactor \accent@spacefactor \unhbox \voidb@x \bgroup \let \unhbox \voidb@x \setbox \@tempboxa \hbox {y\global \mathchardef \accent@spacefactor \spacefactor }\accent 19 y\egroup \spacefactor \accent@spacefactor \unhbox \voidb@x \bgroup \let \unhbox \voidb@x \setbox \@tempboxa \hbox {a\global \mathchardef \accent@spacefactor \spacefactor }\accent 19 a\egroup \spacefactor \accent@spacefactor \unhbox \voidb@x \bgroup \let \unhbox \voidb@x \setbox \@tempboxa \hbox {\OT1\i \global \mathchardef \accent@spacefactor \spacefactor }\accent 19 \OT1\i \egroup \spacefactor \accent@spacefactor \unhbox \voidb@x \bgroup \let \unhbox \voidb@x \setbox \@tempboxa \hbox {e\global \mathchardef \accent@spacefactor \spacefactor }\accent 19 e\egroup \spacefactor \accent@spacefactor

Whoa. Inserting \input{\jobname.output} typesets the expected output, so this is clearly an expanded form of the ěščřžýáíé string, but not what I wanted.

The newfile LaTeX package

The \addtostream command

The next thing I tried was the \addtostream command provided by the newfile LaTeX package.

\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{newfile}
\begin{document}
  \newoutputstream{out}
  \openoutputfile{\jobname.output}{out}
  \def\macro{ěščř}
  \addtostream{out}{\macro žýáíé}
  \closeoutputstream{out}
\end{document}

This generated a file containing

\IeC {\v e}\IeC {\v s}\IeC {\v c}\IeC {\v r}\IeC {\v z}\IeC {\'y}\IeC {\'a}\IeC {\'\i }\IeC {\'e}

Again, not what I wanted.

The writeverbatim environment

Next I tried the writeverbatim environment provided by the package.

\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{newfile}
\begin{document}
  \newoutputstream{out}
  \openoutputfile{\jobname.output}{out}
  \begin{writeverbatim}{out}ěščřžýáíé\end{writeverbatim}
  \closeoutputstream{out}
\end{document}

This generated a file containing ěščřžýáíé in UTF-8, just as I wanted, but I need control sequence expansion. Any ideas?

9
  • 1
    Would you be prepared to switch to xelatex or lualatex? They treat UTF8 characters in the same way as a,...,z, A, ..., Z. Mar 22, 2015 at 18:30
  • Sadly, that's not an option. The package needs to work with pdfLaTeX.
    – Witiko
    Mar 22, 2015 at 18:34
  • 1
    Okay. Perhaps \@onelevel@sanitize\macro before writing? Mar 22, 2015 at 18:37
  • 1
    @Witiko I just replace \usepackage[utf8]{inputenc} by \usepackage[T1]{fontenc} in plain solution and everything is good
    – touhami
    Mar 22, 2015 at 19:08
  • @NicolaTalbot: That works ok, but it doesn't work for multiple indirection.
    – Witiko
    Mar 22, 2015 at 19:31

3 Answers 3

7

You can disable interpretation of UTF-8 characters using the same mechanism that's used for interpreting them, that is redefine \UTF@two@octets and similar macros to produce the string representation of the following characters.

\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}

\makeatletter
\newcommand{\disable@UTF}{%
  \def\UTFviii@two@octets##1##2{\string##1\string##2}%
  \def\UTFviii@three@octets##1##2##3{\string##1\string##2\string##3}%
  \def\UTFviii@four@octets##1##2##3##4{\string##1\string##2\string##3\string##4}%
}
\newwrite\witiko@out
\immediate\openout\witiko@out=\jobname.dat
\newcommand{\witikowrite}[1]{%
  \protected@write\witiko@out{\disable@UTF}{#1}%
}
\makeatother

\begin{document}

Something for activating \texttt{\string\write}

\witikowrite{\emph{ěščřžýáíé}}

\end{document}

This will write in the .dat file

\emph  {ěščřžýáíé}

Depending on what you need, you might want to add other “neutralizing” assignments in the second argument to \protected@write.

5
  • Thank you, this works just fine in the general case. However in my case, the simpler solution seems to be the delayed inclusion of the inputenc package, as described in @touhami's comment.
    – Witiko
    Mar 22, 2015 at 19:38
  • @Witiko Sorry to inform you that not including inputenc will lead you to disasters; try printing ß and you'll see.
    – egreg
    Mar 22, 2015 at 20:32
  • @Witiko moreover, as this is a package you can not know whether the user will load inputenc or not.
    – touhami
    Mar 22, 2015 at 20:45
  • True, but I'm only delaying its inclusion. This is a class and I'm generating the output file at the beginning of its definition. The inputenc package gets loaded afterwards. (edit: Sorry for the confusion, this is a class, not a package)
    – Witiko
    Mar 22, 2015 at 22:12
  • Although to ensure robustness (another class with inputenc loaded could hypothetically LoadClass my class), it's perhaps better to locally reset the UTFviii macro definitions, when generating the file. Just in case.
    – Witiko
    Mar 22, 2015 at 22:15
5

If I use csplain or pdfcsplain (i.e. plain TeX with a little modification) with the source (in UTF8):

\newwrite\fout
\immediate\openout\fout=test.txt
\immediate\write\fout{ěščřžýáíé}
\def\macro{žřů}
\immediate\write\fout{\macro}
\bye

then I get the result in the file test.txt which reads in UTF8:

ěščřžýáíé
žřů

Just simple. The effect is here due to csplain is the format which activates encTeX extension (an extension for UTF8 encoding for pdfTeX).

5
  • encTeX is neat, but I don't think I will be adding it to the class as a dependancy.
    – Witiko
    Mar 22, 2015 at 22:17
  • @Witiko LaTeX uses another principle than encTeX: when 8bit TeX engine (pdftex) and UTF8 input are used then LaTeX user must use [utf8]{inputenc} which activates the UTF8 prefixes. EncTeX needs to be activated when format is generated and LaTeX doesn't activate it. I know several people who activate encTeX in their home-made LaTeX and don't use inputenc. But this never will be official LaTeX and the users must accept that the documents for this home-made LaTeX are not simply interchangeable.
    – wipet
    Mar 23, 2015 at 7:02
  • Afaik, encTeX is a part of the TeXLive distribution, so distributing the class along with a Makefile specifying the necessary pdflatex flags should do the trick.
    – Witiko
    Mar 23, 2015 at 12:10
  • @Witiko Yes. But still there is a potential space for confusions: users cannot use both: encTeX-ed LaTeX plus \usepackage[utf8]{inputenc}. And maybe, there are many LaTeX tricks based on inputenc internals.
    – wipet
    Mar 23, 2015 at 12:22
  • 1
    And another important and potential problem in LaTeX: encTeX totally avoids the LIRC (LaTeX internal representation of characters). LaTeX uses inputenc -> LIRC -> fontenc. On the other hand, encTeX maps UTF8 codes directly to control sequences or 8bit codes of chosen font. If you need to avoid 50 % of LaTeX, OK, use encTeX. But it seems to be better to avoid whole LaTeX and to use plain TeX in such situation.
    – wipet
    Mar 23, 2015 at 13:46
0

Have a look at the following (taken from one of my blog posts), I use it to be able to store also \\ in external files to generate address files. From what I can see it does work for your characters as well.

\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{xpatch}

\makeatletter
% get a copy of `\protected@write
\let\protected@iwrite\protected@write
% patch the copy to add \immediate
\xpatchcmd{\protected@iwrite}{\write}{\immediate\write}{}{}
\makeatother


\newwrite\tempfile
\newcommand{\Anschrift}{ěščř}
\immediate\openout\tempfile=Anschrift.txt
\makeatletter
\protected@iwrite\tempfile{\let\\\relax}{\Anschrift žýáíé}
\immediate\closeout\tempfile
\makeatother

\begin{document}

\input{Anschrift.txt}

\end{document}

screenshot

4
  • The file will contain \IeC {\v e}\IeC {\v s}\IeC {\v c}\IeC {\v r}...
    – egreg
    Mar 22, 2015 at 18:59
  • You are right, this won't work then :-( Mar 22, 2015 at 19:07
  • By the way, I seem to have already seen that code for \protected@iwrite. ;-)
    – egreg
    Mar 22, 2015 at 19:12
  • Good memory :-) Mar 22, 2015 at 19:18

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .