Accelerating LaTeX packages using a make like functionality

I use gnuplottex package quite a lot to draw graphs, and my own bashful package to write include computer output in a document I write. Yet another package of this sort allows one to include metapost output within your document.

The problem is that running all these commands from within LaTeX makes the document processing very slow. I seek an enhanced version of (say) gnuplottex, which given

\begin{gnuplot}[terminal=pdf,]
gnuplot commands
\end{gnuplot}


would compute a check sum of gnuplot commands, compare it to a value from a previous run, and they differ, would run gnuplot again; otherwise, it will just use the previously generated pdf file. I imagine you would need to use an auxiliary file for that, but I am not sure I understand the tradeoffs if any: should once save in the auxiliary file a checsum, or the entire data? How does one compare it?

Presumably, the invocation syntax would be:

\IfChanged{commands}{\regenerate}{\reuse}


where \regenerate is a user macro taking a single argument, which would be passed commands as argument. Command \reuse would somehow reuse the previous run's output. I am not even sure this is the right syntax.

Yet another challenge is to design the command in such a way that the user would not have to add a label to make it distinct from other calls to it.

Clarification: One way of doing that is the following: let h be hash function which generates (say) a 32 bits number. Then you could have a command of the following sort:

\IfChanged{commands}{\regenerate}{\use}

Where:

1. \regenerate is a two arguments macro.
2. \use is a single argument macro.
3. 'IfChanged will compute the h(commands), and if the value has changed, it will invoke: \regenerate{commands}{h(commands)} and then \use{h(x}}
4. If h(commands) did not change, then it would just invoke \use{h(x}}
5. \regenerate{commands}{h(commands)} would do some computation, placing the output in a file named h(commands)
6. The command \use{h(x}} would include the file h(commands)

I hope this clarifies the idea a bit. I would have to think some more about the exact design.

More on context: I have at least three applications in mind - the first is using the bashful package to show computer output on the screen, second is the use of gnuplottex to plot functions (not data), and (c) use of metapost to draw uml diagrams and other stuff.

The command would not be useful as is for running an external "make" to generate experimental data. For this one would need a command that would do the LaTeX equivalent of the following pseudo code:

  If (updated(rawDataFile(s) ))
exec(Make).
Include(outputFile)


but this is less of a problem I think, since you could always run from your command line:

  % make all; pdflatex main.tex

-
You are basically looking for the functionality that the Make tool provides. It triggers the creation of derived files when their source files have changed. However, it only works when source code is not included in a LaTeX file but stays in separate files. This tradeoff will be difficult to overcome. –  Christian Lindig Apr 10 '11 at 11:34
@Christian, yes, Make examines file dates and uses these to determine whether they need to be regenerated. The gnuplotex, bashful, and the mpost, are all alike in that the input is in the LaTeX source file. The output is an external file, which gets included. So, you have at least one file date. The input date is a problem, indeed. I sketched an idea for solution above in editing my question. –  Yossi Gil Apr 10 '11 at 13:49
Computing the hash of the source will be not enough to decide whether a result needs to be recomputed if the result also depends on other files that might have changed, for example data from an experiment. (Dependencies are a can of worms, even with something like Make.) I know that the R/Sweave community is facing the same problem and does have some solution but I am not familiar with it. –  Christian Lindig Apr 10 '11 at 17:11
I have three applications in mind: (a) using gnuplot to plot a function, not data (b) using mpost to draw UML diagram, (c) using bashful to present computer output. Experimental data is a different issue, but I think it is much easier to solve. I will add more information to the question, should someone be interested. –  Yossi Gil Apr 10 '11 at 19:42
I think, this is a case for LuaTeX. Lua is a good language to implement dependency tracking a la Make and could also provide a hash function for your solution. Access to the file system and the shell (for command execution) is also much better in Lua than in TeX. –  Christian Lindig Apr 11 '11 at 17:09