Say you write a macro or a package, which goes through the usual course of software life-cycle. How would you define a set of tests, and have this set run automatically to make sure that changes to the code are in compatible, in some sense of the word?

  • 3
    For svn-multi I made several (~50) test bench files which I can run automatically using a Makefile. You can have a look at them with the SVN browser of my website. ATM the Makefile only tests for compile errors. Some of the test files compare the results and cause such an error on purpose if the result isn't as expected. I wanted to create a real test package which a lot of macros to test packages, but never found the time. Feb 25, 2011 at 17:47
  • It seems not easy to write a simple regression test for TeX, since the test result are often typesetting output. PDF format is not reproducible, DVI may be better but I never tried. Mostly testing results need to be examined by human being.
    – Leo Liu
    Feb 25, 2011 at 18:03
  • 2
    Generate dvi and then use dvitype + diff to validate the output.
    – lhf
    Feb 25, 2011 at 18:08
  • 3
    It depends what you want to test. It is easier if the package produces something which can be compared as string or using lengths, e.g. I test if any command generate unwanted spaces by placing them inside a box which should be 0x0x0). If it is graphical output you need to compare it using image comparison software. I did something like that for the tikz-timing package to check if the timing transients are still correct. Feb 25, 2011 at 18:18
  • 1
    In current news, l3build looks promising. Dec 10, 2014 at 5:24

4 Answers 4


I think there are three broad categories for unit testing in TeX based on:

  1. Log file output
  2. DVI/PDF generated output
  3. TeX-based programming checks

Log file output

Joseph has already mentioned the log-file-based testing used for LaTeX2e and for expl3, but it's worth noting that these tests can also be used for checking the contents of boxes and output, so they're quite general in their scope. For example:

\tracingoutput=1 \tracingonline=1
\showboxdepth=999 \showboxbreadth=999

\savebox\foo{abc $\cos(x)$}

Generated output

For unicode-math and fontspec I've got a different system that is slower and less reliable than log file tests, in which I use generated PDFs to compare before/after differences using ImageMagick's compare tool.

One of the main problems with this approach is the presence of false positives due to off-by-one pixel rounding errors in the rasterisation. (And, possibly, installing new fonts changing the outputs of the tests.)

On thing I like about comparing PDFs, though, is that I've then got a useable document based on the test suite for demonstration purposes. No reason this couldn't be done as a by-product of any of the other testing processes, however.

Using a DVI-based comparison as suggested by other commenters may be more reliable, but I haven't looked into the various options there.

Programming checks

Both of the above techniques require Makefile or other scripts to automate. An easier method, albeit less flexible, is to use the qstest package to write unit tests as part of a TeX document (often included in the dtx source file). We've used this successfully in the hardwrap package, but this technique does require that the test can somehow be checked from within TeX — and often this is sufficient.

As an example from hardwrap.dtx, this sort of test looks like

\begin{qstest}{deal with explicit newlines}{}
\HardWrap{\xdef\TMP}{50}{}{NEWLINE}{aaa bbb^^Jccc}%
\Expect *{\TMP} *{aaa bbbNEWLINEccc}
  • 1
    I also found a unittest framework following 2.: pdf generated output quite usable. I liked it to work with the compare tool of imagemagick; and I combined it with git such I can always checkout a reference revision of my code, generate the reference pdf, then I compare the actual against that reference. However, all these scripts require quite some effort until everythings works as expected. Apr 28, 2011 at 18:49

If you're adding new features, and you have a collection of LaTeX (or TeX) files that don't use the new features, you can check that none of those files are affected by the change by using dvii, from the dvii homepage. If you use the command dvii -p -M1 myfile.dvi > myfile.digest, you'll get a "message digest", i.e., a list of "hashes", one for each page. If you then recreate myfile.dvi using your changed version, you can compare the digests to quickly see if anything on any of the pages has changed.

For example, I have a large collection of LaTeX files that I use for testing a document class. I have a script makedigest as follows:

#! /bin/sh

# We assume that the argument is the name of a dvi file.
# We write the message digest to standard output.

dvii -p -M1 $*

and for each file myfile.tex I've run makedigest to create myfile.dvi.digest.old. When I make a change that's not supposed to affect any of those files, I run the script checknew, which consists of


for f in *tex;do
  for i in `seq 1 6`;do
    latex $f

for f in *.dvi;do
  makedigest $f > ${f}.digest.new


echo "Checking for errors:"
grep rror *.log
echo "Checking for warnings:"
grep Warning *.log

echo "Comparing new digests with old:"
for f in *.dvi;do
  echo "${f}:"
  diff ${f}.digest.old ${f}.digest.new

This quickly runs LaTeX enough times to get a stable version, checks for errors or warnings, and compares the new digests with the old. Thus, I quickly see which pages of which test documents have been changed unexpectedly.

  • 1
    Just a tip: for non-TeX source code the <pre> .. </pre> HTML tags are better to avoid TeX syntax highlighting. In your post this isn't a big deal, thought. Feb 25, 2011 at 18:43
  • @Martin Thanks, but: What sort of content is supposed to be enclosed by <pre>..</pre>? Even better: Where can I find a description of how to properly mark things up when posting here? (I never remember how to mark clickable links here.) Thanks Feb 25, 2011 at 18:51
  • 1
    Here the shell scripts should be enclosed in <pre> tags instead of indenting them with four spaces. See the editing or the help button just below the Add Comment button beside the comment text field, for help on the available syntax. There is also this post by Hendrik Vogt which talks a lot about code highlighting. Feb 25, 2011 at 18:56

For both the LaTeX2e kernel and the current LaTeX3 code, there are a long series of automated tests which use log data only. These test things at the programming end of the spectrum, where you can write the results to the log. A bit of post-processing results in a file containing only the things you need to test and not the variable stuff (for example, which \count is being used for a particular variable). The biggest problem is then not so much the testing itself as writing the tests in the first place. (The automated system for LaTeX3 is available from the public SVN: http://www.latex-project.org/svnroot/experimental/trunk/)


I've been wondering about writing unit tests for my LaTeX packages.

For now, what I'm doing is testing the produced PDFs. I just started today doing this using rspec and pdf-reader` in ruby.

The moderntimeline package now has unit tests based on the produced PDF. The unit tests are in the spec/pdf_spec.rb file, and so far it mostly checks a few things such as the number of pages, the creator, the media box coordinates, the number of fonts and some of the contents.

I've integrated the tests into Travis CI to achieve continuous integration of the package. See the .travis.yml file in the repository for how this is done.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.