Maybe the question is also of interest beyond biblatex but here's the thing: I recently published my biblatex style biblatex-fiwi and I'm always having trouble making sure that updates don't lead to unexpected behavior. For example, I just realized that my last update includes a stupid error where a colon is dropped at an important place. The colon was there in previous versions but since I wasn't careful enough it's gone at the moment.

What I would like to have is some kind of automated testing environment which would check a set of bibliographic entries against a defined outcome. At the moment I always have to check the final PDF myself which is error-prone.

What's to easiest way to do this?

  • Follow-up: is there a way to do just the same, but with my whole latex distribution ? I often see package or font updates en up breaking stuff in subtle ways (in particular, ways that pdftotext would not notice). but maybe that deserves a question on its own.
    – Gyom
    Dec 9, 2011 at 7:09
  • @Gyom: compare also with a diff the two log files.
    – user2478
    Dec 9, 2011 at 7:16
  • See also tex.stackexchange.com/q/36351/5763 Dec 9, 2011 at 16:47
  • @Herbert, MartinSchröder: thanks a lot for these suggestions. I'll definitely try these as soon as I find a little bit of time to work on that topic !
    – Gyom
    Dec 13, 2011 at 18:49

2 Answers 2

  • Save the old output,
  • create the new one,
  • convert both with pdftotext to true text,
  • compare the output with diff.

can all be done with a script

  • For the moment I will try the pdftotext variant, thanks a lot.
    – Simifilm
    Dec 9, 2011 at 13:31

I have the same issue with biblatex-apa. I opted for a perl script using the CAM::PDF module which can compare PDFs and tell you the differences. I save a "reference" PDF of the test document included with the style and then run this script to compare the this PDF with the one output from the same test document after changes. If all ok, this becomes the new reference document. The output of the script isn't really nice - it compares page by page and says "ok" if the page is the same and otherwise outputs a fairly useless error message but at least I know which page to check for differences at that point ... Script is below. You can see where my two PDFs are mentioned. I just run this in the same directory as the two PDFs.


use Test::More;
use Test::Deep;
use CAM::PDF;

cmp_pdf('biblatex-apa-test.pdf', 'reference.pdf');

sub cmp_pdf ($$) {
  my ($got, $expected) = @_;
  $got = CAM::PDF->new($got) or die "Could not create CAM::PDF instance with : $got";
  $expected = CAM::PDF->new($expected) or die "Could not create CAM::PDF instance with : $expected";

  foreach my $page_num (1 .. $expected->numPages()) {
    my $tree1 = $got->getPageContentTree($page_num, "verbose");
    my $tree2 = $expected->getPageContentTree($page_num, "verbose");
    cmp_deeply($tree1->{blocks}, $tree2->{blocks}, "Page $page_num");
  • Yes, please share the script.
    – N.N.
    Dec 9, 2011 at 8:39
  • I'm definitely interested in that script.
    – Simifilm
    Dec 9, 2011 at 8:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.