A way to "execute" only latex procedural code (\def, \let) in a sequence ignoring the rest?

Question

I have an external input file which I am reading line by line. This file contains text as well as some tex/latex macros. My goal is to write the content of this file into another external output file, and I would like to be able to replace some of the latex commands by symbols (to be more specific: I would like to be able to translate simple latex commands into their corresponding markdown counterpart).

So far, it works well for simple commands, eg \textbf{bold} is translated into bold. I have achieved this by redefining \textbf before writing back to the output file (I am using \write for this):

\def\textbf#1{*#1*} 

Now I would like to be able to do the same for itemize. I would like the following code:

\begin{itemize}
  \item one
  \item two
  \item three
\end{itemize}

to be translated into:

*   one
*   two
*   three

A first idea would be to ignore \begin{itemize} and \end{itemize} and simply redefine \item so that is expands into the markdown itemize prefix.

However, I want to be able to handle nested itemize. I have been using the same method (redefining \begin and \end, which in turn will use a modified \itemize and \enditemize), keeping track of the nesting level and increasing the markdown prefix accordingly. As you can imagine, things went wrong: since there is a lot of \def, \if, \begingroup etc. involved in the process, what I get at the end in my output file is a sequence of latex code, mainly containing \def's, braces, and so on.

This brings me to my question: before I send a line containing both text and "procedural latex code" (eg \def, \if and so on) to \write, is there a way to "execute" the procedural code it contains, and then write the output to the external file?

I have been using all the tricks I know (\edef, \expandafter's, \romannumeral, ...) to force the expansion, with no success so far.

This is a very minimal example to illustrate what I am trying to achieve:

\documentclass{minimal}

\usepackage{filecontents}

\begin{filecontents*}{temp.tex}
This is \textbf{bold}

This is itemize:
\begin{itemize}
  \item one
  \item two
  \item three
\end{itemize}
\end{filecontents*}

\newread\filein
\newwrite\fileout

\def\markdownbf#1{**#1**}
\def\markdownbegin#1{\csname markdown#1\endcsname}
\def\markdownend#1{\csname markdownend#1\endcsname}
\def\markdownitemize{\edef\markdownprefix{*   }}
\def\markdownenditemize{}
\def\markdownitem{}
\def\markdownpar{}

\def\activatemarkdowncommands{%
  \let\textbf=\markdownbf%
  \let\begin=\markdownbegin%
  \let\end=\markdownend%
  \let\itemize=\markdownitemize%
  \let\enditemize=\markdownenditemize%
  \let\item=\markdownitem%
  \let\par=\markdownpar%
}

\def\translatefileA{%
  \begingroup%
    \edef\markdownprefix{}%
    \activatemarkdowncommands%
    \immediate\openin\filein temp.tex%
    \immediate\openout\fileout out.txt%
    \loop\unless\ifeof\filein%
      \immediate\read\filein to\fileline%
      \immediate\write\fileout{\markdownprefix\fileline}%
    \repeat%
    \immediate\closeout\fileout%
    \immediate\closein\filein%
  \endgroup%
}

\def\premarkdownbf#1{}
\def\premarkdownbegin#1{\csname premarkdown#1\endcsname}
\def\premarkdownend#1{\csname premarkdownend#1\endcsname}
\def\premarkdownitemize{\edef\storemarkdownprefix{*   }}
\def\premarkdownenditemize{\edef\markdownprefix{}}
\def\premarkdownitem{\let\markdownprefix\storemarkdownprefix}
\def\premarkdownpar{}

\def\postmarkdownbf#1{**#1**}
\def\postmarkdownbegin#1{\csname postmarkdown#1\endcsname}
\def\postmarkdownend#1{\csname postmarkdownend#1\endcsname}
\def\postmarkdownitemize{}
\def\postmarkdownenditemize{}
\def\postmarkdownitem{}
\def\postmarkdownpar{}

\def\activatepremarkdowncommands{%
  \let\textbf=\premarkdownbf%
  \let\begin=\premarkdownbegin%
  \let\end=\premarkdownend%
  \let\itemize=\premarkdownitemize%
  \let\enditemize=\premarkdownenditemize%
  \let\item=\premarkdownitem%
  \let\par=\premarkdownpar%
}

\def\activatepostmarkdowncommands{%
  \let\textbf=\postmarkdownbf%
  \let\begin=\postmarkdownbegin%
  \let\end=\postmarkdownend%
  \let\itemize=\postmarkdownitemize%
  \let\enditemize=\postmarkdownenditemize%
  \let\item=\postmarkdownitem%
  \let\par=\postmarkdownpar%
}

\def\translatefileB{%
  \begingroup%
    \edef\markdownprefix{}%
    \immediate\openin\filein temp.tex%
    \immediate\openout\fileout out.txt%
    \loop\unless\ifeof\filein%
      \immediate\read\filein to\fileline%
      \activatepremarkdowncommands%
      {\nullfont\fileline}%
      \activatepostmarkdowncommands%
      \immediate\write\fileout{\markdownprefix\fileline}%
    \repeat%
    \immediate\closeout\fileout%
    \immediate\closein\filein%
  \endgroup%
}

\begin{document}
  \translatefileB
\end{document}

If I use \translatefileA, then I get in out.txt:

This is **bold** 

This is itemize: 
\edef {* } 
one 
two 
three 

If I use \translatefileB, then I get:

This is **bold** 

This is itemize: 

* one 
* two 
* three 

The first version is wrong because \markdownprefix is not set as it should, because \markdown\itemize is not expanded as intended I guess.

The second version is ok. Unwanted output in the PDF file is deleted using \nullfont. So far it seems the best solution I could come up with.

Answer 1

Check for a substring in the infile using \readline. \readline reads it as a raw verbatim input, and execute the line only if your commands are in there, if not the line is ignored. So there at two reads. Suffixed with ‘L’ for LaTeX to be run and with ‘V’ for verbatim (plain text)

Edit: Previous code not working. Now fixed.

\documentclass{article}
\usepackage{substr, pgf, tikz}
\begin{document}
\makeatletter
\newread\fileinL
\newread\fileinV
\openin\fileinL=inFile.dat
\openin\fileinV=inFile.dat

\gdef\executecommands{\string\xdef, \string\let}

\loop\unless\ifeof\fileinL%
  \readline\fileinV to\filelineV%
  \read\fileinL to\filelineL%
  \foreach \cmd in \executecommands{
    \IfSubStringInString{\cmd}{\filelineV}{\filelineL}{}
  }
\repeat%
\end{document}

Answer 2

It seems \nullfont answers my question, also it needs a little more work to handle math mode properly (see \nullfont in mathmode).

Below is the modified code to handle math mode (ie: suppress all output, yet expanding macros and defining them). Macro \translatefile will read the content of temp.tex, expand the macros it contains (first stage) without generating output in PDF, and translate it into markdown (second stage), generating file out.txt. It is not a fully functional translator, it is rather a proof of concept: it will only handle \textbf and (non-nested) itemize.

Node: another (and probably better) solution to achieve the same effect would be to use \setbox0\vbox{....} rather than \nullfont as David Carlisle suggested. However I'm not familiar enough with this kind of command to use it!

\documentclass{minimal}

\usepackage{filecontents}

\begin{filecontents*}{temp.tex}
This is \textbf{bold}, this is math: $1+x^2$.

This is itemize:
\begin{itemize}
  \item one,
  \item two,
  \item three.
\end{itemize}
\end{filecontents*}

\newread\filein
\newwrite\fileout

\begingroup
  \catcode`\ =12%
  \gdef\SPACE{ }%
\endgroup

\def\premarkdownbf#1{}
\def\premarkdownbegin#1{\csname premarkdown#1\endcsname}
\def\premarkdownend#1{\csname premarkdownend#1\endcsname}
\def\premarkdownitemize{\edef\storemarkdownprefix{*\SPACE\SPACE\SPACE}}
\def\premarkdownenditemize{\edef\markdownprefix{}}
\def\premarkdownitem{\let\markdownprefix\storemarkdownprefix}
\def\premarkdownpar{}

\def\postmarkdownbf#1{**#1**}
\def\postmarkdownbegin#1{\csname postmarkdown#1\endcsname}
\def\postmarkdownend#1{\csname postmarkdownend#1\endcsname}
\def\postmarkdownitemize{}
\def\postmarkdownenditemize{}
\def\postmarkdownitem{}
\def\postmarkdownpar{}

\def\activatepremarkdowncommands{%
  \let\textbf=\premarkdownbf%
  \let\begin=\premarkdownbegin%
  \let\end=\premarkdownend%
  \let\itemize=\premarkdownitemize%
  \let\enditemize=\premarkdownenditemize%
  \let\item=\premarkdownitem%
  \let\par=\premarkdownpar%
}

\def\activatepostmarkdowncommands{%
  \let\textbf=\postmarkdownbf%
  \let\begin=\postmarkdownbegin%
  \let\end=\postmarkdownend%
  \let\itemize=\postmarkdownitemize%
  \let\enditemize=\postmarkdownenditemize%
  \let\item=\postmarkdownitem%
  \let\par=\postmarkdownpar%
}

\newfont\dummyfont{dummy}

\def\translatefile{%
  \begingroup%
    \nullfont%
    \everymath{%
      \count0=0%
      \loop%
        \ifnum\count0<16%
        \textfont\count0\dummyfont%
        \scriptfont\count0\dummyfont%
        \scriptscriptfont\count0\dummyfont%
        \advance\count0 by 1%
      \repeat%
    }%
    \edef\markdownprefix{}%
    \immediate\openin\filein temp.tex%
    \immediate\openout\fileout out.txt%
    \loop\unless\ifeof\filein%
      \immediate\read\filein to\fileline%
      \activatepremarkdowncommands%
      \fileline%
      \activatepostmarkdowncommands%
      \immediate\write\fileout{\markdownprefix\fileline}%
    \repeat%
    \immediate\closeout\fileout%
    \immediate\closein\filein%
  \endgroup%
}

\begin{document}
  This should be visible.
  \translatefile %This should not be visible, only processed to out.txt
  This should be visible.
\end{document}