Tell me more ×
TeX - LaTeX Stack Exchange is a question and answer site for users of TeX, LaTeX, ConTeXt, and related typesetting systems. It's 100% free, no registration required.
up vote 15 down vote favorite
1

The creation of biblatex has made bst files obsolete allowing the formatting to be done by LaTeX and not an add on program like BibTeX, but it didn't completely eliminate the need for an add on program and gave us Biber. What is it that add on programs like Biber and MakeIndex do that cannot be done in "pure" LaTeX? If a few new "simple" primitives were introduced, could these add on programs be implemented in LaTeX? By simple I mean not creating a \biber primitive. Could they be implemented directly and "easily" in LuaTeX?

share|improve this question
5  
You might want to take a look at amsrefs, which does completely avoid an external program for creating a bibliography. – Joseph Wright♦ Sep 27 '12 at 10:14
datagidx (part of datatool) creates indexes and glossaries without the use of an external application, but the sorting and collation is a lot slower (and it currently only sorts according to the English alphabet). Using, say, xindy, is far more efficient. – Nicola Talbot Apr 12 at 11:42

3 Answers

up vote 13 down vote accepted

Good multi-field sorting can't really be done using regexps. You have to be able to compare fields in a user definable order, with each field match locally allowing for case-sensitivity, ascending/descending, perhaps UTF-8 variable weight punctuation etc. While this is probably theoretically doable with a (very expanded, serious, like Perl's) regexp engine it would be an utter nightmare and slow. This is usually done with some tricky array acrobatics.

The uniqueness systems in biber (uniquename and especially uniquelist) require lots of interdependent looping constructs which are unbounded but with guaranteed eventual exits. I suspect TeX would eat memory like anything doing this.

You also need bibtex format parser which is fast (biber uses the btparse library in C via the Text::BibTeX module, modified for UTF-8 passthrough). Parsing and grammar in TeX? Shudder.

Lua is another matter. It's probably possible and would be great. But you'd be surprised how much biber does internally - porting it would be quite a job. You could probably build an interface to the btparse library which would be the first step.

share|improve this answer
up vote 6 down vote

While the interpreter is Turing complete as tohecz correctly pointed out the main reason is that the job of these external programs would be hard to implement in TeX.

For LuaTeX the answer is more or less historical. LuaTeX was never meant to replace biber but I am quite confident that there are very few technical issues to overcome to create a biber library instead of an executable and then call this library from LuaTeX. In the long term the functionality might even be implemented in LuaTeX but considering how well biber/biblatex work together the question is whether someone is willing to reimplement this in Lua.

share|improve this answer
It is the "hard to implement" that I am trying to get at. What are they doing that is so hard to implement in LaTeX and why is it hard? – Daniel E. Shub Sep 27 '12 at 12:52
2  
@DanielE.Shub Would you use Assembler for programming a Web browser? You'd probably use a language with many predefined functions and data structures, instead. The comparison is exaggerated, of course, but not that much. – egreg Sep 27 '12 at 12:57
1  
@DanielE.Shub: An idea about the "hard to implement" issue is nicely shown in the pgf manual. In section 67 Till Tantau explains an implementation of object oriented programming in TeX. As he had to invent that building up only on a relatively primitive TeX you see what might all be necessary to reimplement biber. I think egreg's comment is right on and additionally assembler syntax is often not so confusing as TeX syntax. – Alexander Sep 27 '12 at 13:40
up vote 9 down vote

You do not even need (in theory) to introduce a new primitve.

  1. eTeX (the behind-interpreter, not TeX or LaTeX) is Turing complete: Are there any disadvantages of TeX being Turing complete?

  2. It is capable of reading and writing files.

So in theory, you don't need it. On the other hand, in real, you have two problems:

  1. Making real large programs in eTeX is quite dificult (for, see the code of beamer or tikz).

  2. The Turing-completeness is a theory based on "if the (memory) capacity was infinite". But it is not infinite, and running biber/bibtex means sorting large data structures by some criteria, and even loading these structures might be a problem in TeX itself. (Remember that you can call bibtex on a .bib file with thousands of entries.)

I cannot speak of LuaTeX, because I don't know Lua really.

share|improve this answer
6  
At least for pdfTeX and XeTeX, there are some things that TeX cannot do. It can't for example access networks, so grabbing remote data sources is not possible (Biber for example can do that). – Joseph Wright♦ Sep 27 '12 at 10:15
4  
There are some fairly complex things involved in, for example, biber which would be really quite hard to do in TeX. Multi-field sorting with UTF-8 CLDR for example. Can't imagine doing that in TeX. I don't know Lua but so much of Perl's history is involved with fast UTF-8 support that I would imagine it would be quite a job to port. – PLK Sep 27 '12 at 10:31
@PLK this is exactly the type of thing I am interested in. I would love to see you expand it as an answer if you get time. It seems like the regexp support in LaTeX3 that I have read a little bit about might make sorting much easier. – Daniel E. Shub Sep 27 '12 at 12:59
@DanielE.Shub We do have regex support in LaTex3, but it's truly 'regular', so many of the things the PCRE can do are out. Performance is also an issue if you want to use it for large-scale work. – Joseph Wright♦ Sep 27 '12 at 17:07

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.