40

I have a simple document, about 20 pages long with some mathematics and a couple of tikz diagrams. I use biblatex and biber (0.9.9 MacTeX 2011) to compile my references, of which there are currently 2, with maybe 10 citations.

When using the bibtex backend the bibliography stage of my compilation takes under a second. Biber takes over 5 seconds to process exactly the same file.

I use TeXShop and with biber the console window appears but none of biber's command line output appears for at least 4 seconds.

Is there a problem with my setup or is biber slow by design?

EDIT: yes, it is slow every time. I've done some digging into the folders where biber unpacks its perl dependencies and I think TeXShop might be unpacking it every time. Perhaps something deletes the unpacked binary after each use.

The way I've made TeXShop use biber is to change the bibTeX engine field in the preferences to `biber'.

Improve this question
8
  • 5
    The first time you run Biber, it has to unpack a lot of Perl stuff. Second and subsequent runs should therefore be a lot faster than the first run. Do you see this issue repeatedly?
    – Joseph Wright ♦
    Apr 25, 2012 at 10:34
  • 3
    @JosephWright -- no question it speeds up, but it still runs slower than bibtex. It's still worth it, given all the extra functionality, but there is a noticeable difference. (Same with luatex; and the two combined gives you time to grab a coffee for book-length projects.)
    – jon
    Apr 25, 2012 at 15:18
  • 3
    On my system, Biber is certainly slower than BibTeX when run for example from the Terminal, with no graphical element and no unpacking (4.5 s versus 0.3 s for the test file biblatex-chem-acs). I'm not sure you can say it's 'deliberate', but BibTeX is a small program written in (I think) C whereas Biber is more complex and written in Perl. I'm not sure what you can do here unless you are volunteering to work on the Biber codebase.
    – Joseph Wright ♦
    Apr 25, 2012 at 15:33
  • 7
    I don't really see the point porting to a faster language - it's a batch program after all. You will never get something as fast as bibtex but with 80% more functionality ... Of course it could be done in C but it would be a hell of a job.
    – PLK
    Apr 25, 2012 at 15:50
  • 4
    @PLK Maybe even cross-compiling it to C might be worthwhile. The time difference yun described is harmless compared to my document where it's 0.3s for bibtex vs. 14s for biber. That's almost 50 times slower.
    – Christian
    Mar 16, 2013 at 16:44

1 Answer 1

Reset to default
46

It is slower than bibtex which is in C, even if you take into consideration the first run unpacking. Bear in mind that biber does a lot more than bibtex too. They are hardly comparable in functionality at all. Your tikz and maths should make no difference to biber. If your cache is getting deleted every time you run, this will make a huge difference. Easy to check this - delete the cache and run. Is the second biber run any faster?

The main overhead is sorting. It is a complex business, dealing with much more than bibtex - Unicode 7.0, direction per-field, case per field ... Next overhead is uniqueness processing. Again, complex. Bibtex probably does about 20% of what biber does. See the biber PDF manual to get a sense of its share of the biblatex work.

As of version 2.5 (currently in DEV), I have done some profiling with NYTProf. The majority of bibers time is spent inside the Unicode::Collate module (written in C), as one would expect as sorting is a main focus and it's expensive to do tailored UCA sorting (which bibtex doesn't even come close to doing). After some examining of the call stacks, I've done some loop tidying for sorting calls and now biber 2.5 is about four times as fast as 2.4 and probably all earlier versions.

As mentioned in the doc, for performance testing, I use a 2150 entry, 15,000 line .bib file which references a 630 entry macro file with a resulting 160 or so page bibliography. In biber 2.4 this takes about 2 minutes to process. In the current 2.5 development version it takes about 28 seconds. This is almost the same now as when using the --fastsort option which doesn't use Unicode collation (so I may drop --fastsort since it is functionally far less useful and if there is no performance benefit, there is no longer any point in it).

Improve this answer
13
  • 6
    I will look more into optimisation at some point
    – PLK
    Apr 25, 2012 at 15:58
  • 2
    Try the biber 1.0 beta -it has a --cache option to show the cache location now. You will need to be using biblatex 2.0 beta
    – PLK
    Apr 26, 2012 at 14:37
  • 3
    I'm using biber 2.11 and it's still extremly slow. Sorting algorithms have evolved quite a lot and shouldn't have any problem sorting 10000 lines in much less than a second.
    – skan
    Jun 14, 2018 at 20:12
  • 3
    The sorting is done using a modern algorithm with custom cacheing, it has nothing to do with sorting. The main speed penalty is Unicode handling and sorting key generation and this is inevitable in any system that does complex Unicode handling. I can assure you short of rewriting everything in C (pointless anyway since all critical subcomponents - sorting, unicode collation etc. are already in C), there is no speeding such complex processing up. You can use any perl you want to run biber if you are not using the pre-packaged binaries.
    – PLK
    Jun 16, 2018 at 17:41
  • 3
    I'm wondering whether sorting is really the issue here. I tried using some rfc.bib file containing about 8500 entries, of which I use exactly one (so no sorting is needed, right?). Just parsing it takes 12 seconds, how can that be? (I stopped the time between the LaTeX decoding and Found BibTeX data source output.)
    – Marian
    Mar 22, 2019 at 18:38

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .