biblatex: How to build a testing environment

Question

Maybe the question is also of interest beyond biblatex but here's the thing: I recently published my biblatex style biblatex-fiwi and I'm always having trouble making sure that updates don't lead to unexpected behavior. For example, I just realized that my last update includes a stupid error where a colon is dropped at an important place. The colon was there in previous versions but since I wasn't careful enough it's gone at the moment.

What I would like to have is some kind of automated testing environment which would check a set of bibliographic entries against a defined outcome. At the moment I always have to check the final PDF myself which is error-prone.

What's to easiest way to do this?

Answer 1

• Save the old output,
• create the new one,
• convert both with pdftotext to true text,
• compare the output with diff.

can all be done with a script

Answer 2

I have the same issue with biblatex-apa. I opted for a perl script using the CAM::PDF module which can compare PDFs and tell you the differences. I save a "reference" PDF of the test document included with the style and then run this script to compare the this PDF with the one output from the same test document after changes. If all ok, this becomes the new reference document. The output of the script isn't really nice - it compares page by page and says "ok" if the page is the same and otherwise outputs a fairly useless error message but at least I know which page to check for differences at that point ... Script is below. You can see where my two PDFs are mentioned. I just run this in the same directory as the two PDFs.

#!/usr/bin/perl

use Test::More;
use Test::Deep;
use CAM::PDF;

cmp_pdf('biblatex-apa-test.pdf', 'reference.pdf');
done_testing;


sub cmp_pdf ($$) {
  my ($got, $expected) = @_;
  $got = CAM::PDF->new($got) or die "Could not create CAM::PDF instance with : $got";
  $expected = CAM::PDF->new($expected) or die "Could not create CAM::PDF instance with : $expected";

  foreach my $page_num (1 .. $expected->numPages()) {
    my $tree1 = $got->getPageContentTree($page_num, "verbose");
    my $tree2 = $expected->getPageContentTree($page_num, "verbose");
    cmp_deeply($tree1->{blocks}, $tree2->{blocks}, "Page $page_num");
  }
}