# Document equivalence

What is the best way to determine, for two arbitrary input files, if they will result in the same compiled document (disregarding timestamp information and external files)?

## Context

I'm attempting to implement a nontrivial change to a LaTeX code generator. The code base is backed with an integration test: Input documents with expected outputs. Unfortunately, the change I'm about to implement is fiddling with whitespace (and hopefully only with those!) all over the place. I'm looking for a nice automatic way to prove (or disprove) that the whitespace changes are irrelevant.

## Example

### Initial version

\documentclass{scrartcl}\pagestyle{empty}
\begin{document}

Hello
World!

\begin{verbatim}
Whitespace  are
relevant

here!!!
\end{verbatim}

\end{document}


### No-op modifications

\documentclass{scrartcl}
\pagestyle{empty}

\begin{document}
Hello World!

\begin{verbatim}
Whitespace  are
relevant

here!!!
\end{verbatim}
\end{document}


### Meaningful modifications

\documentclass{scrartcl}
\pagestyle{empty}

\begin{document}
Hello World!

\begin{verbatim}
Whitespace are
relevant

here!!!
\end{verbatim}
\end{document}


### Comparison (aligned at bottom)

Click on the <> links between the pictures to show a textual diff of the files.

## Question

So: What's the best way to implement such a comparison? Of course, one possible solution is to render as PNG (as in http://tex.stackexchange.com/a/55323/8057), remove timestamp information with, e.g., pngcrush, and compare at the byte level. Can a PDF timestamp be removed the same way? Are there other solutions at the TeX level?

-
what we do in the latex2e regression test suite is (essentially) add \showoutput to the document and then compare the relevant part of the log files. –  David Carlisle Oct 30 '13 at 1:12
@DavidCarlisle: This is pretty much exactly what I'm looking for. When running on my example, the meaningless differences look much different from the meaningful differences. Care to convert this to an answer? Also, where can I find the regression test suite? Is there logic to determine the "meaningful" parts of the log? –  krlmlr Oct 30 '13 at 1:32

What we do in the latex2e regression test suite is (essentially) add \showoutput to the document and then compare the relevant part of the log files.

The 2e test suite is here

http://latex-project.org/svnroot/latex2e-public/testfiles/

The lvt files being the tex input and the tlg files being the log files slightly normalised with dates and file paths being removed so the results should be exactly the same on all systems.

the helper macros used in the tests are

http://latex-project.org/svnroot/latex2e-public/testfiles/test2e.tex

Frank and I had a tugboat article on the basic mechanism, or at least I thought we did, anyway here is a paper of Frank's:-)

http://www.tug.org/TUGboat/tb18-4/tb57mitt.pdf

-
Thanks a lot! Is it enough to input test2e.tex (and provide the .cfg) to activate the mechanism? –  krlmlr Oct 30 '13 at 23:27
@krlmlr the test2e.tex isn't really needed, it just provides some helper macros that do \typeout to the log file the mechanism is some external perl script for cleaning up the log. But there we need to trace macro expansion selectively, but for your use I think you just want to check consistent output so you just need \showoutput and do a file diff that ignores the date at the start of the log, and any absolute paths to loaded package files. –  David Carlisle Oct 30 '13 at 23:34
Unfortunately, in my use case, not even the LaTeX input is consistent: see this diff -- the output doesn't change, but the log diff shows a few false positives. –  krlmlr Oct 30 '13 at 23:40
@krlmlr That's exactly as expected. The first block is because your source is 2 lines different, so either arrange that doesn't happen (eg comment out lines rather than removing them when making the versions) or run the log files through sed to remove on input line [0-9]+ before running diff, similarly delete the stack positions line. Actually if you are only concerened with the \showoutput output you could first delete every line of the log that does not start with a . and just diff the remainder –  David Carlisle Oct 31 '13 at 9:52

If you're looking for some sort of "observational equivalence", that is, if you want to directly compare LaTeX's output, I suggest that you use Postscript instead of PDF. By using a script such as the following for compiling the LaTeX files that you wish to compare, you can force your Postscript output to contain the same file name information and the same creation dates:

#!/bin/bash

TMPDATE=date +"%a %b %d %X %Y"
TMPFILE=mktemp doceq.XXXXXX --suffix=".tex"

while [ "$1" != "" ]; do cp -f "$1" $TMPFILE latexmk --quiet -ps$TMPFILE
TARGET=${1/.tex/.ps} cat${TMPFILE/.tex/.ps}                                  \
| awk "BEGIN { found = 0; }                            \
/^%%CreationDate: / {                           \
if (!found) {                                 \
printf(\"%%%%CreationDate: $TMPDATE\\n\"); \ found = 1; \ next; \ } \ } \ { print; }" \ > "$TARGET"
latexmk --quiet -C $TMPFILE rm -f$TMPFILE
shift
done


For example, if your three examples are contained in doceq1.tex, doceq2.tex and doceq3.tex and ./doceq.sh is the script above, here's what you can get:

$./doceq.sh doceq1.tex doceq2.tex doceq3.tex Latexmk: Run number 1 of rule 'latex' (snip)$ diff doceq1.ps doceq2.ps
\$ diff doceq1.ps doceq3.ps
802c802
< 405 y Fa(Whitespace)93 b(are)335 518 y(relevant)335 857
---
> 405 y Fa(Whitespace)45 b(are)335 518 y(relevant)335 744


If you are tied to PDF for some reason, you can do the same there but it will be trickier, as PDF files contain binary parts that you will have to skip. The lines that you will have to change are those starting with /CreationDate and /ModDate.

-
Actually, this works even for my pdflatex chain if I postprocess the document with Poppler's pdftops (not pdf2ps). See meaningful changes. Big advantage: no timestamp is written, no postprocessing needed! –  krlmlr Oct 30 '13 at 22:44