A way to “execute” only latex procedural code (\def, \let) in a sequence ignoring the rest?

I have an external input file which I am reading line by line. This file contains text as well as some tex/latex macros. My goal is to write the content of this file into another external output file, and I would like to be able to replace some of the latex commands by symbols (to be more specific: I would like to be able to translate simple latex commands into their corresponding markdown counterpart).

So far, it works well for simple commands, eg \textbf{bold} is translated into bold. I have achieved this by redefining \textbf before writing back to the output file (I am using \write for this):

\def\textbf#1{*#1*}


Now I would like to be able to do the same for itemize. I would like the following code:

\begin{itemize}
\item one
\item two
\item three
\end{itemize}


to be translated into:

*   one
*   two
*   three


A first idea would be to ignore \begin{itemize} and \end{itemize} and simply redefine \item so that is expands into the markdown itemize prefix.

However, I want to be able to handle nested itemize. I have been using the same method (redefining \begin and \end, which in turn will use a modified \itemize and \enditemize), keeping track of the nesting level and increasing the markdown prefix accordingly. As you can imagine, things went wrong: since there is a lot of \def, \if, \begingroup etc. involved in the process, what I get at the end in my output file is a sequence of latex code, mainly containing \def's, braces, and so on.

This brings me to my question: before I send a line containing both text and "procedural latex code" (eg \def, \if and so on) to \write, is there a way to "execute" the procedural code it contains, and then write the output to the external file?

I have been using all the tricks I know (\edef, \expandafter's, \romannumeral, ...) to force the expansion, with no success so far.

This is a very minimal example to illustrate what I am trying to achieve:

\documentclass{minimal}

\usepackage{filecontents}

\begin{filecontents*}{temp.tex}
This is \textbf{bold}

This is itemize:
\begin{itemize}
\item one
\item two
\item three
\end{itemize}
\end{filecontents*}

\newwrite\fileout

\def\markdownbf#1{**#1**}
\def\markdownbegin#1{\csname markdown#1\endcsname}
\def\markdownend#1{\csname markdownend#1\endcsname}
\def\markdownitemize{\edef\markdownprefix{*   }}
\def\markdownenditemize{}
\def\markdownitem{}
\def\markdownpar{}

\def\activatemarkdowncommands{%
\let\textbf=\markdownbf%
\let\begin=\markdownbegin%
\let\end=\markdownend%
\let\itemize=\markdownitemize%
\let\enditemize=\markdownenditemize%
\let\item=\markdownitem%
\let\par=\markdownpar%
}

\def\translatefileA{%
\begingroup%
\edef\markdownprefix{}%
\activatemarkdowncommands%
\immediate\openin\filein temp.tex%
\immediate\openout\fileout out.txt%
\loop\unless\ifeof\filein%
\immediate\write\fileout{\markdownprefix\fileline}%
\repeat%
\immediate\closeout\fileout%
\immediate\closein\filein%
\endgroup%
}

\def\premarkdownbf#1{}
\def\premarkdownbegin#1{\csname premarkdown#1\endcsname}
\def\premarkdownend#1{\csname premarkdownend#1\endcsname}
\def\premarkdownitemize{\edef\storemarkdownprefix{*   }}
\def\premarkdownenditemize{\edef\markdownprefix{}}
\def\premarkdownitem{\let\markdownprefix\storemarkdownprefix}
\def\premarkdownpar{}

\def\postmarkdownbf#1{**#1**}
\def\postmarkdownbegin#1{\csname postmarkdown#1\endcsname}
\def\postmarkdownend#1{\csname postmarkdownend#1\endcsname}
\def\postmarkdownitemize{}
\def\postmarkdownenditemize{}
\def\postmarkdownitem{}
\def\postmarkdownpar{}

\def\activatepremarkdowncommands{%
\let\textbf=\premarkdownbf%
\let\begin=\premarkdownbegin%
\let\end=\premarkdownend%
\let\itemize=\premarkdownitemize%
\let\enditemize=\premarkdownenditemize%
\let\item=\premarkdownitem%
\let\par=\premarkdownpar%
}

\def\activatepostmarkdowncommands{%
\let\textbf=\postmarkdownbf%
\let\begin=\postmarkdownbegin%
\let\end=\postmarkdownend%
\let\itemize=\postmarkdownitemize%
\let\enditemize=\postmarkdownenditemize%
\let\item=\postmarkdownitem%
\let\par=\postmarkdownpar%
}

\def\translatefileB{%
\begingroup%
\edef\markdownprefix{}%
\immediate\openin\filein temp.tex%
\immediate\openout\fileout out.txt%
\loop\unless\ifeof\filein%
\activatepremarkdowncommands%
{\nullfont\fileline}%
\activatepostmarkdowncommands%
\immediate\write\fileout{\markdownprefix\fileline}%
\repeat%
\immediate\closeout\fileout%
\immediate\closein\filein%
\endgroup%
}

\begin{document}
\translatefileB
\end{document}


If I use \translatefileA, then I get in out.txt:

This is **bold**

This is itemize:
\edef {* }
one
two
three


If I use \translatefileB, then I get:

This is **bold**

This is itemize:

* one
* two
* three


The first version is wrong because \markdownprefix is not set as it should, because \markdown\itemize is not expanded as intended I guess.

The second version is ok. Unwanted output in the PDF file is deleted using \nullfont. So far it seems the best solution I could come up with.

• Could you provide a MWE? Something that works for the simple \textbf, and what you want to work? Then we don't have to redo all of that :) – Andreas Storvik Strauman Mar 24 '18 at 13:51
• I suspect that you are asking the wrong question, but it is a bit hard to work out what workflow you are intending. You could look at (for example) ex4ht which is doing a similar job, but generating html rather than markdown – David Carlisle Mar 24 '18 at 13:54
• latex is a macro expansion language it doesn't really have procedural code, it isn't at all clear what you mean by executing code before you send it to a \write. If the code is \zzzz which has been defined by \newcommand\zzzz{hello} you can of course write hello (that will happen by default) but if it is \hbox{hello} what do you want to \write \hbox, { and } are all non expandable primitives – David Carlisle Mar 24 '18 at 13:57
• What is the purpose? If you want to translate latex to markdown, there probably exists software for this. Also LaTeX is not a text manipulation program for sure. Searching and replacing substrings is one of the most hazzleful things you can do. – Andreas Storvik Strauman Mar 24 '18 at 14:14
• @David Carlisle: I suspect my question may have a simple answer. I do know that a lot of tex primitives (like \hbox you mentionned) are not suitable for my purpose, and I dont want to be able to translate them into markdown. What I do want is to be able to process some of them, like \textbf and the likes. The problem for me is to deal with \itemize so far. – Vincent Forman Mar 24 '18 at 14:38

Check for a substring in the infile using \readline. \readline reads it as a raw verbatim input, and execute the line only if your commands are in there, if not the line is ignored. So there at two reads. Suffixed with ‘L’ for LaTeX to be run and with ‘V’ for verbatim (plain text)

Edit: Previous code not working. Now fixed.

\documentclass{article}
\usepackage{substr, pgf, tikz}
\begin{document}
\makeatletter
\openin\fileinL=inFile.dat
\openin\fileinV=inFile.dat

\gdef\executecommands{\string\xdef, \string\let}

\loop\unless\ifeof\fileinL%
\foreach \cmd in \executecommands{
\IfSubStringInString{\cmd}{\filelineV}{\filelineL}{}
}
\repeat%
\end{document}

• this doesn't really address the question, as the OP wants to execute the \def without typesetting the text that is on that line not in the def. – David Carlisle Mar 24 '18 at 16:11
• Yeah, doesn’t really cover the case with multiple commands on one line. – Andreas Storvik Strauman Mar 24 '18 at 16:14
• I may have found a solution using \nullfont (see above). I will be running some tests to decide if it completely solves my problem. – Vincent Forman Mar 24 '18 at 16:49
• Ah. I now got what you meant! Haha, okay. Then the answer I provided is pretty special casey! I was thinking you only wanted to run select commands, not that you wanted to run all commands, but suppress output. – Andreas Storvik Strauman Mar 24 '18 at 17:16

It seems \nullfont answers my question, also it needs a little more work to handle math mode properly (see \nullfont in mathmode).

Below is the modified code to handle math mode (ie: suppress all output, yet expanding macros and defining them). Macro \translatefile will read the content of temp.tex, expand the macros it contains (first stage) without generating output in PDF, and translate it into markdown (second stage), generating file out.txt. It is not a fully functional translator, it is rather a proof of concept: it will only handle \textbf and (non-nested) itemize.

Node: another (and probably better) solution to achieve the same effect would be to use \setbox0\vbox{....} rather than \nullfont as David Carlisle suggested. However I'm not familiar enough with this kind of command to use it!

\documentclass{minimal}

\usepackage{filecontents}

\begin{filecontents*}{temp.tex}
This is \textbf{bold}, this is math: $1+x^2$.

This is itemize:
\begin{itemize}
\item one,
\item two,
\item three.
\end{itemize}
\end{filecontents*}

\newwrite\fileout

\begingroup
\catcode\ =12%
\gdef\SPACE{ }%
\endgroup

\def\premarkdownbf#1{}
\def\premarkdownbegin#1{\csname premarkdown#1\endcsname}
\def\premarkdownend#1{\csname premarkdownend#1\endcsname}
\def\premarkdownitemize{\edef\storemarkdownprefix{*\SPACE\SPACE\SPACE}}
\def\premarkdownenditemize{\edef\markdownprefix{}}
\def\premarkdownitem{\let\markdownprefix\storemarkdownprefix}
\def\premarkdownpar{}

\def\postmarkdownbf#1{**#1**}
\def\postmarkdownbegin#1{\csname postmarkdown#1\endcsname}
\def\postmarkdownend#1{\csname postmarkdownend#1\endcsname}
\def\postmarkdownitemize{}
\def\postmarkdownenditemize{}
\def\postmarkdownitem{}
\def\postmarkdownpar{}

\def\activatepremarkdowncommands{%
\let\textbf=\premarkdownbf%
\let\begin=\premarkdownbegin%
\let\end=\premarkdownend%
\let\itemize=\premarkdownitemize%
\let\enditemize=\premarkdownenditemize%
\let\item=\premarkdownitem%
\let\par=\premarkdownpar%
}

\def\activatepostmarkdowncommands{%
\let\textbf=\postmarkdownbf%
\let\begin=\postmarkdownbegin%
\let\end=\postmarkdownend%
\let\itemize=\postmarkdownitemize%
\let\enditemize=\postmarkdownenditemize%
\let\item=\postmarkdownitem%
\let\par=\postmarkdownpar%
}

\newfont\dummyfont{dummy}

\def\translatefile{%
\begingroup%
\nullfont%
\everymath{%
\count0=0%
\loop%
\ifnum\count0<16%
\textfont\count0\dummyfont%
\scriptfont\count0\dummyfont%
\scriptscriptfont\count0\dummyfont%
\repeat%
}%
\edef\markdownprefix{}%
\immediate\openin\filein temp.tex%
\immediate\openout\fileout out.txt%
\loop\unless\ifeof\filein%
\activatepremarkdowncommands%
\fileline%
\activatepostmarkdowncommands%
\immediate\write\fileout{\markdownprefix\fileline}%
\repeat%
\immediate\closeout\fileout%
\immediate\closein\filein%
\endgroup%
}

\begin{document}
This should be visible.
\translatefile %This should not be visible, only processed to out.txt
This should be visible.
\end{document}
`