I know, for various reason, some users preprocess their latex file using Perl or sed, say.

I'm considering doing this to, so I would like to seek your guidance, to smooth my entry in this area.

My use case is a simple macro expansion preprocessing

If $\ep$ is small...   -->  If $\epsilon$ is small...


So that a compilation session becomes

doc.tex  --preprocessor-->  post.tex  --pdftex--> post.pdf


And the following issues arise

1. There is generally a mismatch between line numbers in doc.tex and post.tex. So that pdftex error messages points to a line in post.tex. But, really, one wants the line number in doc.tex.

2. The same for synctex. Backward and foward search are with respect to post.tex. But, one really wants them to be with respect to doc.tex.

3. Sometimes, one wants to go back and forth from doc.tex and post.tex. The preprocessor knows the corresponds between lines of the two documents. But then, how to put this knowledge in a workable way. I mean, text editors does not have out-of-the-box support for these backward-forward trips.

So, do you know some software, workflow or technics which will flatten (1)-(3)?

I look around, but I could only find example of Perl or sed preprocessing scripts.

I never saw the question of the workflow discuss somewhere.

EDIT

The line mismatch can originate from big expansion

A function is continuous if
\multiline_continuity_definition_I_often_use
Hence ...


where the definition is expanded to

\begin{description}

\item[Standard]
$\forall \epsilon ...$

\item[Nonstandard]
$x \approx y ...$

\end{description}


Edit 2

The point of this question is to discuss preprocessing technics.

My examples are very simplistic, but I hope they catch the main challenge of a preprocessing session.

-
What causes the line mismatch? Are you hard-wrapping your lines so that \ep --> \epsilon causes 'hard' changes to lineation? Soft-wrapping might be better in your case, unless it is something else causing the mismatch. – jon Mar 29 '13 at 20:38
Even if the editor is hard-wrapping, the described preprocessor shouldn't move the line breaks. – T. Verron Mar 29 '13 at 20:41
Given that TeX is mostly a macro processing engine with some typesetting capabilities added, My use case is a simple macro expansion preprocessing must be the strangest use case for wanting to invoke an external processor. Why on earth can't you use \def\eps{\epsilon} ???? – David Carlisle Mar 29 '13 at 21:10
@DavidCarlisle It was not easy to write this question. I tried to catch the main points of a preprocessing session. It's true that my epsilon example is not so good. – Nicolas Essis-Breton Mar 29 '13 at 21:16
@NicolasEssis-Breton, perhaps but it seems that all three of your issues go away if you use TeX to do your re-processing so I think you'd need to give an example of something that can't so easily be done in TeX (there are examples: pulling fragments from a database, translating between different languages, ....) – David Carlisle Mar 29 '13 at 21:20

There is no full solution for this problem, as far as I know, because it will vary depending on your needs.

But here are two hints which should rule out your three problems for most cases:

## Use LaTeX as much as possible

Your first example could easily be solved by

\newcommand{\ep}{\epsilon}


\newcommand{\multiline_continuity_definition_I_often_use}{% You might prefer a shorter name :)
\begin{description}

\item[Standard]
$\forall \epsilon ...$

\item[Nonstandard]
$x \approx y ...$

\end{description}
}


The key point is to avoid shadok logic, that is "Don't waste time developing a tool you don't really need". Even if the tool looks beautiful.

## Try to separate what is in the document and what isn't

This is for "real" examples where you need a preprocessor. Usually, that will be because they involve dynamic data (if you need to do always the exact same thing, latex should be enough) and difficult computations/processing.

What you can do then is separate this dynamic part from your document. For example, \input a file which will only contain the part which will need preprocessing. Then preprocess the document (following these \input, or simply filtering the auxiliary files based on their extension, for example), and compile.

Latex can output the errors in a file-line-error style, and then the line-numbering won't be messed up. If you are using a multifile-aware editor (like, for example, emacs + AUCTeX + refTeX), this can be made almost transparently.

About synctex, it shouldn't cause any issue either. And for the back-and-forth between the base document and the processed one... Well, since you only change small auxiliary files, there should be no problem.

A good example of this workflow is the sagetex package. It allows users to make some computations with the symbolic computation software sage, and typeset their results directly in the .pdf document.

It uses some post-processing to make it even more transparent to the user, but the idea is the same:

1. The user inputs the computations in the .tex file (using a few commands)
2. A first run of pdflatex extracts all these sage bits in a document.sage file.
3. A run of sage on that file creates a document.sageout containing the results.
4. A second run of pdflatex inserts these results in the final document.

It is a fully-developed package, seriously optimised for user-friendliness, but you don't need such a carefully-designed solution. For example, you can create the extra files by hand (with the help of your editor, it can be as easy as typing \input{filename} and hitting a few keys).

And once again, this isn't a full solution, you will probably need some work to find out what really suits your needs... Take it as hints towards your workflow, nothing more.

-
+1 yes sagetex is a good example (rewriting sage in tex would be inconvenient:-) – David Carlisle Mar 29 '13 at 22:08