Hooks into mathmode

Question

Short version

Is there a way to pipe every symbol in a math environment unevaluated to an external program w/o manipulating the (La)TeX file directly (except for adding header files) or patching TeX itself?

Longer version

I wrote a program which performs certain transformations on (La)TeX code in mathmode. Until now I used to implement a very simplified TeX interpreter (doesn't follow \includes/\usepackages etc.), which filtered out code between the usual suspects ($…$ pairs, \begin…\end{equation} etc.). This approach fails when facing newly defined math environments or constructs which only partly work in mathmode.

So now I want to use the already existing TeX compiler to my advantage and want it to filter the crucial mathmode parts out for me (I'd then use \write18 or something to communicate with my external application). But I figured that redefining all the math environments or even the $…$ or $$…$$ pairs fails since this approach isn't safe against manipulation with symbol category codes (a user could just define | to be the new $).

So far the only thing I found which was related to automatically finding mathmode environments are the commands \everymath and \everydisplay. Unfortunately, I can't seem to figure out how to use them to accomplish my goals.

Answer 1

\everymath won't solve your problem, for a number of reasons:

1. It is only inserted at the beginning of maths, so you have to scan the maths to find the end;
2. Tex doesn't have a real concept of the extent of a mathematical formula, only commands that enter and leave math mode, so it is hard to reliably find the end of a formula;
3. There are several different ways in raw Tex to enter and leave math mode, and these can be and are multiplied by the freedom Tex gives you to redefine how its source is processed, so it is really hard to reliably find the end of a formula.

I do not think it is possible to run insert some Tex code in the middle of any Latex document to do what you want. However Tex does have a principled notion of math mode a little further down its digestive tract, when it starts figuring out how to put boxes together.

This can be exploited by software that work like Tex4ht, which runs Tex but changes the macros to pass extra information from the input to later digestive stages. Tex4ht uses this trick to produce HTML, but Ross Moore has been working on creating tagged PDF, where formulae in the output PDF are (nearly always) tagged with the Tex equations used to produce them. This could be adapted to outputting the formulae to separate files.

This probably falls foul of your request not to modify Tex, but is really the only technique that works with the mass of documents one finds. Alternatively, you can do as Andrey suggested, and only work with documents where some constraint is followed that makes it easy to recover formulae from the source. My preference would be to insist that only \( ... ]) and \[ ... \] are used, so that formulae are delimited by macros that you can redefine yourself.

Answer 2

The following works somewhat. That is, it does work with mathshift instead of just $, it has some problems though. For one, it requires a newline at some point. Which isn't a problem most of the time though. A more severe point is that it "leaks" a newline token at some point while that is assigned category code 12. I know when and why it happens, just not how to fix it yet. Furthermore it only works on regular inline math at this point, although an analogous version for dispaymath should not be incredibly hard. Clearly all it is right now is a possible direction to take with regards to this problem.

\documentclass{article}
\begin{document}
  \catcode`|=3
  \def\y{\bgroup\catcode`\^^M=12 (\x}
  {\catcode`\^^M=12 %
  \gdef\x#1^^M{\z#1\null^^M}%
  \gdef\z#1#2^^M{\ifcat#1|\egroup#1)\ifx#2\null\else#2^^M\fi\else#1\ifx#2\null\else\z#2^^M\fi\fi}%
  }
  \everymath={\y}
  \noindent
  $x^2$
  \\|x^2|
  \\$x^2$ text
  \\$x^2$ text $y^2$ text
\end{document}

Sorry for the non-descriptive macro names by the way. The problem with the ^^M is in the \ifx#2\null\else#2^^M\fi.