1

I'm trying to pass data that contains (and needs to keep) newlines and all kinds of special chars (like ä, $, ", \, ...) to a command. The idea is to then write the data to a file, run it through an external parser which creates a picture and to finally include the picture. This would allow all kinds of languages (that can be parsed to a image or PDF or so) to be written directly in a LaTeX file.

I've got it to work except for the special characters which got me wondering:

  • how can I pass and use data as is (unless of course it contains something like \end)? What's the best/easiest/most fool-proof/most widespread way to approach this?
8
  • 2
    \write\zzz{\detokenize{......}} perhaps. Feb 17, 2020 at 17:27
  • 2
    The package filecontents might be of interest. The +V-argument-type of xparse might be of interest. How to handle indentation in the .tex-file when the argument of \command spans several lines? Shall indentation at the beginnings of lines within the argument also yield indentation in the external file? By the way: When TeX reads input, it always does some pre-processing which cannot be prevented: All space-characters at the right get removed and a character according to \enlinechar gets appended. Feb 17, 2020 at 21:02
  • Thanks a lot for these inputs. If the \detokenize-method works that would by very simple. filecontents sounds promising for the intended purpose. I'll give those two a try tomorrow.
    – ritt0r
    Feb 17, 2020 at 22:19
  • 2
    @UlrichDiez A text file whose last line isn't empty is anomalous, you know. :-)
    – frougon
    Feb 17, 2020 at 23:28
  • 1
    @frougon It would indeed be anomalous. Because each line of text should end with the characters which with the computer-platform in question denote the end of a line (CR/LF/CR+LF/whatsoever). It is about "copying" a macro-argument to external file. I intended to point out: The end of the macro-argument will be at a closing brace } and thus without things that denote the end of a line while "copying" to file will in the file yield these things that denote the end of a line. Feb 18, 2020 at 0:01

1 Answer 1

2

I assume you wish a macro/a command which can produce within an external text file an exact copy of that portion of the .tex-input-file, which in the .tex-input-file denotes that macro's argument.


If you were not asking for a command/if an environment would be suficient, I'd suggest the filecontents*-environment from the LaTeX 2ε-kernel.

With LaTeX-releases prior to Fall 2019 the filecontents*-environment could only be used in the document's preamble and did not overwrite existing files.

But there is the package filecontents which removes these restrictions by providing the same environment(s) in a way where usage of the environments is not restricted to the preamble and where existing files will be overwritten.

So you can use the filecontents*-environment for creating temporary files which can be overwritten right after reading them back via \input/\include/\read/whatsoever or after having them post-processed by other programs whereby post-processing can probably be initiated from within the (La)TeX-run by means of TeX's \write18-functionality.

Some explanation of the \write18-feature can be found on https://www.texdev.net, written and maintained by Joseph Wright.

When it comes to using more recent LuaTeX-based engines where the \write18-feature is not available by default as these engines provide means for implementing it, the shellesc-package might be of interest.

If you wish to create files with images in .eps-format and to use the .eps-files in \includegraphics of the graphicx-package when running pdflatex/xelatex for creating a .pdf-file, the package epstopdf may be of interest. That package converts .eps-files to .pdf-files "on the fly" during the pdflatex-run. With more recent TeXLive-distributions pdflatex will invoke epstopdf under the hood when \includegraphics is used for including an .eps-image. With other TeX-distributions epstopdf may need to be loaded explicitly after graphicx .

Here is an example where the filecontents*-environment is used for creating an .eps-file (whose name will be the file name of the .tex-file + the phrase Picture, with extension .eps) and a text file containing a latex picture environment (whose name will be the file name of the .tex-file + the phrase Picture, with extension .lpe—lpe being my abbreviation for LaTeX-Picture-Environment).

I currently use pdflatex from TeXLive 2019 where .eps-to-.pdf-conversion is done under the hood on the fly automatically, so on my system the example compiles without problems.

If you save the example as test.tex, then (besides the usual auxiliary-files and .log-file) the files testPicture.eps and testPicture.lpe will be created during compilation with pdflatex. testPicture.eps is an .eps-image which can be viewed, e.g., with GSview or gv. testPicture.lpe is a text-file containing a picture-environment. Conversion of the .eps-file to .pdf will deliver a file with a .pdf-image whose name is testPicture-eps-converted-to.pdf.

\documentclass[a4paper]{article}
%\usepackage{shellesc} % <- implements the \write18-interface (which is
                       %     needed by epstopdf) for more recent LuaTeX-
                       %     based engines
\usepackage{graphicx}
%\usepackage{epstopdf}


% An .eps graphic by means of filecontents:
\begin{filecontents*}{\jobname Picture.eps}
%!PS-Adobe-3.0 EPSF-3.0
%%BoundingBox: 0 0 50 50
%%Title: (EXAMPLE.EPS)
%%CreationDate: 18/02/2020 18:09:07 +0200
%%EndComments
newpath 25 25 24 0 360 arc stroke
newpath 25 25 18 225 315 arc stroke
newpath 18 33 2 0 360 arc fill stroke
newpath 32 33 2 0 360 arc fill stroke
showpage
%%EOF
\end{filecontents*}


% A LaTeX 2e picture-environment by means of filecontents:
\begin{filecontents*}{\jobname Picture.lpe}
\begingroup
\unitlength=1pt\relax
\thicklines
\begin{picture}(51,51)(0,0)
\put(26,26){\circle{40}}
\put(19,34){\circle*{4}}
\put(33,34){\circle*{4}}
\put(26,22){\oval(24,24)[b]}
\end{picture}
\endgroup
\end{filecontents*}


\begin{document}

\section{A picture by LaTeX's picture-environment}
\begin{figure}[h]
\centering
\input{\jobname Picture.lpe}
\caption{A picture by LaTeX's picture-environment}
\end{figure}

\section{An .eps-graphic (probably auto-converted to .pdf via the pstopdf-package)}
\begin{figure}[h]
\centering
\includegraphics{\jobname Picture.eps}%
\caption{An .eps-graphic (probably auto-converted to .pdf via the pstopdf-package)}
\end{figure}

\end{document}

enter image description here

By the way:

By default the filecontents/filecontents*-environment will convert horizontal-tab-characters (code-point number 9 both in ASCII and in Unicode, ^^I in TeX's ^^-notation) into spaces. If you don't want this, on many TeX-platforms it suffices to just temporarily set the category code of the horizontal-tab to 12(other) before applying filecontents/filecontents*.

This could look like this:

\begingroup
\catcode`\^^I=12 %
\begin{filecontents*}
[...the content of the filecontents*-environment...]
\end{filecontents*}
\endgroup

By now I elaborated on filecontents* which is an environment.
But you did not ask for an environment.
You asked for a command.

Some preliminary remarks about TeX before delivering an example of how I might go at implementing a command:

TeX takes .tex-input-files for directives for producing tokens. E.g., control-sequence-tokens. E.g., character-tokens. These tokens are appended to the token-stream. This happens in TeX's mouth. The token-stream goes down TeX's gullet. Here expansion of expandable tokens takes place. The results of expansion reach the stomach for further processing.

The crucial point is:

TeX has subtle rules for creating tokens.

E.g., whenever TeX starts reading a line of input, some of the first things it does are:

  1. Converting the characters from the computer platform's character encoding scheme to the TeX engine's internal character representation scheme which with traditional TeX-engines is ASCII and with LuaTeX/XeTeX-based TeX-engines is Unicode.
  2. Removing all space characters at the end of the line. (Space-characters have codepoint-number 32 in the TeX engine's internal character representation scheme.)
  3. Appending a character at the end of the line whose code-point-number in the TeX engine's internal character representation scheme corresponds to the value of the integer-parameter \endlinechar. In case that value is outside the range of code-point-numbers that are possible in the TeX-engine's internal character representation scheme, no character will be appended.

Item 1 implies that writing to external file requires a routine for converting back to the computer-platform's character encoding scheme. With many TeX platforms this rouine can be configured via so-called TeX-character-translation-files/.tcx-files. On some TeX platforms/under some circumstances that routine might, e.g., be configured to deliver ^^-notation for some special characters although your computer-platform can handle these characters. This might lead to copied output differing from what was typed into the .tex-input-file.

Item 2 implies that TeX will never ever in whatsoever ways process space characters that were written at the end of a line in a .tex-input-file.
I.e., you can never ever use TeX for copying a sequence of space characters that was written at the end of a line in a .tex-input-file.

E.g., a character of category code 0(escape)—usually the backslash-character is of that category code—is taken for a signal to not gather a character-token (category code/character code-pair) from a single character but to gather the name of a control sequence token from subsequent characters. (I intend to convey a broad-brushed picture only, so I omit the exact rules for gathering names of control sequence tokens from .tex-input-files here.)

E.g., after creating a control-word-token, the reading-apparatus is set to state S(skipping blanks) which means that subsequent input-characters of category code 10(space) will be ignored/skipped/dropped instead of yielding tokens.

E.g., a character of category code 14(comment) in the .tex-input-file causes TeX to ignore/skip/drop the remaining input-characters of the .tex-input-file's current line instead of forming tokens from them, and to switch the reading-apparatus to state N(new line) and to start reading the next line of the .tex-input-file if present. In state N characters of category code 10(space) found in the .tex-input-file will be ignored/skipped/dropped instead of yielding tokens.

E.g., a character of category code 5(end of line) in the .tex-input-file causes TeX to ignore/skip/drop the remaining characters of the .tex-input-file's current line and to append  depending on the state of the reading-apparatus either (in State N, new line) the token \par, or (in State M, middle of line) a space-token, or (in State S, skipping blanks) no token at all to the token-stream, and to switch the reading-apparatus to state N(new line) and to start reading the .tex-input-file's next line if present.

Did you notice the distinction between characters in the .tex-input and character-tokens (with character code and category code) which come into being while the TeX-program is running and which get processed during compilation?

Be that as it may.

All these subtleties related to how TeX at the time of reading the .tex-input-file and creating tokens deals with input-characters depending on category codes and the state of the reading-apparatus lead to the conclusion that, if you wish to process an argument "as is", you need to switch to a category code régime where no characters of the .tex-input will be ignored/skipped/dropped during tokenization, but where each character will yield a (character-)token from which, at the time of writing the tokens to file, that very character will be regained, and where the reading-apparatus during tokenizing a line of .tex-input will always remain in state M (middle of line) and in any case will switch from state N to state M when encountering the very first character of a line of .tex-input, leading to the effect that space-characters at the beginnings of lines of .tex-input will not be ignored. (The above mentioned item 2 implies that even under such a category code régime you cannot prevent the ignoring of space-characters within lines of .tex-input that contain nothing else but space-characters.)

The command \verb|...| and the environment verbatim (temporarily) switch to a category code régime where this is the case. Henceforth this category code régime is called "verbatim-catcode-régime".

TeX has subtle rules for writing tokens to external text files.

E.g., the \write-directive will always expand expandable tokens (macros and some primitives), unless they either come from expanding a token-register via \the during the \write-operation or they come as the ⟨balanced text⟩ of an \unexpanded-directive during the \write-operation.

E.g., explicit character tokens of category code 6(parameter), i.e., things like hashes (#), will be doubled while writing.

E.g., TeX always appends a space-character when unexpanded-writing a control-word-token.

E.g., the \write-directve is intended to write a line of text. Therefore the ⟨line-break-characters of your platform⟩ will be appended to the characters which the \write-directive forms from its argument for writing.
\immediate\write\MyNiceFile{\noexpand\LaTeX\relax}
will deliver
\LaTeX⟨space character⟩\relax⟨space character⟩⟨line-break-characters of your platform⟩
Having things in the same line of a text file that come from different \write-directives is not possible.

Another interesting question with "using/writing argument as is" is the question how to handle code that in the .tex-input-file is indented.

I fiddled together an approach where xparse's +v-argument-type (+v denotes that the argument is to be read and tokenized from the .tex-input-file under verbatim-catcode-régime) is used for reading/tokenizing the argument, so that

  • linebreaks etc are preserved,
  • hashes are not doubled,
  • no space-characters (but those at the endings of lines) and the like are lost,
  • ^^-notation in the input will not be converted,

and where writing is done via a token-register whereby unwanted expansion of active characters is suppressed:

With the following example the macro
\MyWriteFile{⟨text-file's name⟩}{⟨text-file's content⟩}
(plus all its helper-macros) is defined.

That macro creates anew a text-file of name ⟨text-file's name⟩ and writes ⟨text-file's content⟩ to it.

In the following example that macro is used for creating the text-files MyFirstBizarreTemp.tex and MySecondBizarreTemp.tex.

The \verbatiminput*-command of the verbatim-package is used for displaying the contents of these text-files.

\documentclass{article}

\usepackage{xparse}
\usepackage{verbatim}% <- \verbatiminput* from that package is used for
                     %    displaying the content of text-files 
                     %    that got created during compilation.
                     %    The package is not used otherwise.

\makeatletter
%%=============================================================================
%% Check whether argument is empty:
%%=============================================================================
%% \UD@CheckWhetherNull{<Argument which is to be checked>}%
%%                     {<Tokens to be delivered in case that argument
%%                       which is to be checked is empty>}%
%%                     {<Tokens to be delivered in case that argument
%%                       which is to be checked is not empty>}%
\newcommand\UD@CheckWhetherNull[1]{%
  \romannumeral0\if\relax\detokenize{#1}\relax
  \expandafter\@firstoftwo\else\expandafter\@secondoftwo\fi
  {\@firstoftwo{\expandafter}{} \@firstoftwo}%
  {\@firstoftwo{\expandafter}{} \@secondoftwo}%
}%
%%=============================================================================
%% Exchange arguments
%%=============================================================================
\newcommand\UD@Exchange[2]{#2#1}%
%%/////////////////////////////////////////////////////////////////////////////
%% Code for  \ReadAndProcessVerbatimizedStuff
\begingroup
% Dummy-definition, will be overridden. Is used only to get ^^M of 
% category code 12(other) as #1 into subsequent definition-texts:
\NewDocumentCommand\UD@CheckWhetherLeadingEndl{+m}{%
  \endgroup
  %%===========================================================================
  %% Check whether_verbatimized_ argument starts with a endline-character
  %%===========================================================================
  %% \UD@CheckWhetherLeadingEndl{<Argument which is to be checked>}%
  %%                            {<Tokens to be delivered in case <argument
  %%                              which is to be checked>'s 1st token is a
  %%                              endline-charactern>}%
  %%                            {<Tokens to be delivered in case <argument
  %%                              which is to be checked>'s 1st token is not
  %%                              a endline-character>}%
  \newcommand\UD@CheckWhetherLeadingEndl[1]{%
    \UD@@CheckWhetherLeadingEndl\UD@SelDom##1\UD@SelDom#1\UD@@SelDom
  }%
  \@ifdefinable\UD@@CheckWhetherLeadingEndl{%
    \long\def\UD@@CheckWhetherLeadingEndl##1\UD@SelDom#1##2\UD@@SelDom{%
      \UD@CheckWhetherNull{##2}{\@secondoftwo}{\@firstoftwo}%
    }%
  }%
  %%===========================================================================
  %% Remove one leading endline-character from _verbatimized_ argument
  %%===========================================================================
  \@ifdefinable\UD@@TrimLeadingEndl{\long\def\UD@@TrimLeadingEndl#1{}}%
  %%===========================================================================
  %% Check whether _verbatimized_ argument ends with a endline-character
  %%===========================================================================
  %% \UD@CheckWhetherTrailingEndl{<Argument which is to be checked>}%
  %%                             {<Tokens to be delivered in case <argument
  %%                               which is to be checked>'s last token is a
  %%                               endline-charactern>}%
  %%                             {<Tokens to be delivered in case <argument
  %%                               which is to be checked>'s last token is not
  %%                               a endline-character>}%
  \newcommand\UD@CheckWhetherTrailingEndl[1]{%
    \UD@@CheckWhetherTrailingEndl##1\UD@SelDom#1\UD@SelDom\UD@@SelDom
  }%
  \@ifdefinable\UD@@CheckWhetherTrailingEndl{%
    \long\def\UD@@CheckWhetherTrailingEndl##1#1\UD@SelDom##2\UD@@SelDom{%
      \UD@CheckWhetherNull{##2}{\@secondoftwo}{\@firstoftwo}%
    }%
  }%
  %%===========================================================================
  %% Remove one trailing endline-character from _verbatimized_ argument
  %%===========================================================================
  \newcommand\UD@TrimTrailingEndl[1]{\UD@@TrimTrailingEndl##1\UD@SelDom}%
  \@ifdefinable\UD@@TrimTrailingEndl{%
    \long\def\UD@@TrimTrailingEndl##1#1\UD@SelDom{##1}%
  }%
  %%===========================================================================
  %% Remove one leading and one trailing endline-character from _verbatimized_ 
  %% argument if present. Due to \romannumeral0-expansion the result is
  %% delivered in 2 expansion-steps:
  %%===========================================================================
  \newcommand\UD@RemoveLeadingNTrailingEndl[1]{%
    \romannumeral0%
    \UD@CheckWhetherLeadingEndl{##1}{%
      \UD@CheckWhetherTrailingEndl{##1}{%
         \UD@Exchange{ }{%
           \expandafter\expandafter\expandafter\expandafter
           \expandafter\expandafter\expandafter
         }%
         \expandafter\UD@TrimTrailingEndl\expandafter{\UD@@TrimLeadingEndl##1}%
       }{%
         \UD@Exchange{ }{\expandafter}%
         \UD@@TrimLeadingEndl##1%
       }%
    }{%
      \UD@CheckWhetherTrailingEndl{##1}{%
         \UD@Exchange{ }{\expandafter\expandafter\expandafter}%
         \UD@TrimTrailingEndl{##1}%
      }{ ##1}%
    }%
  }%
  %%===========================================================================
  %% \ReadAndProcessVerbatimizedStuff{%
  %%   <Processor-macro><Processor-macro's arguments except Processor-macro's 
  %%   last argument>%
  %% }{<Argument to be read and tokenized under verbatim-catcode-regime>}
  %%
  %% yields:
  %%
  %% <Processor-macro><Processor-macro's arguments except Processor-macro's 
  %% last argument>{<Argument to be read and tokenized under verbatim-
  %%  catcode-regime>}
  %%
  %% I.e., 
  %% \ReadAndProcessVerbatimizedStuff reads and tokenizes under verbatim-
  %% catcode-régime from the .tex-input-file the last argument for
  %% <Processor-macro> .
  %% 
  %%===========================================================================
  \newcommand*\ReadAndProcessVerbatimizedStuff[1]{%
    % ##1 = file-name
    \begingroup
    % Make sure horizontal tabs won't cause problems when
    % \ReadAndProcessVerbatimizedStuff reads its +v-argument:
    \@makeother\^^I%
    \InnerReadAndProcessVerbatimizedStuff{##1}%
  }%
  \NewDocumentCommand{\InnerReadAndProcessVerbatimizedStuff}{m+v}{%
    % ##1 = file-name
    % ##2 = argument that got read and tokenized under verbatim-catcode-regime.
    \endgroup
    \expandafter\expandafter\expandafter\UD@Exchange
    \expandafter\expandafter\expandafter{%
    \expandafter\expandafter\expandafter{\UD@RemoveLeadingNTrailingEndl{##2}}}{##1}%
  }%
}%
%%-----------------------------------------------------------------------------
%% Now let's change the catcode of ^^M and then call the dummy-definition
%% of \UD@CheckWhetherLeadingEndl so that it can fetch the catcode-12-^^M,
%% close the group, override itself and define all those macros where that
%% catcode-12-^^M  is needed:
\catcode`\^^M=12 %
\UD@CheckWhetherLeadingEndl{^^M}%
%%End of code for  \ReadAndProcessVerbatimizedStuff
%%/////////////////////////////////////////////////////////////////////////////

%%=============================================================================
%% Let's define a processor-macro whose first argument is a file-name
%% and whose second argument is the verbatimized-argument:
%%   \MyWriteFileProcessor{<file>}{%
%%      <argument that already got  tokenized under 
%%      verbatim-catcode-regime>%
%%   }
%% writes the <argument that already got  tokenized under verbatim-
%% catcode-regime> to file if the <file> does
%% not exist. If the <file> does exist, you get a message on screen.
%%=============================================================================
\newcommand\MyWriteFileProcessor[2]{%
  \begingroup
  \openin\@inputcheck#1\relax
  \ifeof\@inputcheck
    \@latex@warning@no@line{Writing file `\@currdir #1'}%
    \chardef\reserved@c15\relax
    \ch@ck7 \reserved@c\write
    \immediate\openout\reserved@c#1\relax
    \toks@{#2}%
    \begingroup
    % Make sure endline-chars (carriage-return, ^^M in ^^-notation ) 
    % yield linebreaks:
    \newlinechar=`\^^M %
    \immediate\write\reserved@c{\the\toks@}%
    \endgroup
    \immediate\closeout\reserved@c#1\relax
  \else
    \closein\@inputcheck
    \@latex@warning@no@line{%
       File `#1' already exists on the system.\MessageBreak
       Not generating it from this source%
     }
  \fi 
  \endgroup
}%
%%=============================================================================
%% Let's define a macro which reads things verbatimized and writes them to file
%% \MyWriteFile{<file-name>}{<Argument to be read and tokenized 
%% under verbatim-catcode-regime and to be written to a new file 
%% with name <file-name> "as is">}
%%=============================================================================
\newcommand\MyWriteFile[1]{%
   % #1 = <file-name> ;
   % \ReadAndProcessVerbatimizedStuff will read and tokenize the
   % <Argument to be read and tokenized under verbatim-catcode-
   % regime> and append the result as another argument to
   % \MyWriteFileProcessor{<file-name>}
   \ReadAndProcessVerbatimizedStuff{\MyWriteFileProcessor{#1}}%
}%
\makeatother

\pagestyle{empty}
\begin{document}

\enlargethispage{4cm}
\vspace*{-2cm}

\noindent\textbf{The input}

\begin{verbatim*}
  \MyWriteFile{MyFirstBizarreTemp.tex}|
    [This time the content is within verbatim-delimiters.]
    {%
      \Huge
      another
      piece
      of
      text%
      äöü
    }\\
    {\scriptsize another piece of text}
  X|
\end{verbatim*}
  \MyWriteFile{MyFirstBizarreTemp.tex}|
    [This time the content is within verbatim-delimiters.]
    {%
      \Huge
      another
      piece
      of
      text%
      äöü
    }\\
    {\scriptsize another piece of text}
  X|

\noindent\textbf{yields \texttt{MyFirstBizarreTemp.tex} with following content:}

\verbatiminput*{MyFirstBizarreTemp.tex}

\noindent\hrulefill\null\vspace{\ht\strutbox}

\noindent\textbf{The input}

\begin{verbatim*}
  \MyWriteFile{MySecondBizarreTemp.tex}{
    [This time the content is within curly braces.]
    {%
      \Huge
      another
      piece
      of
      text%
      äöü
    }\\
    {\scriptsize another piece of text}
  X}
\end{verbatim*}
  \MyWriteFile{MySecondBizarreTemp.tex}{
    [This time the content is within curly braces.]
    {%
      \Huge
      another
      piece
      of
      text%
      äöü
    }\\
    {\scriptsize another piece of text}
  X}

\noindent\textbf{yields \texttt{MySecondBizarreTemp.tex} with following content:}

\verbatiminput*{MySecondBizarreTemp.tex}

\end{document}

enter image description here

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.