The creation of
biblatex has made
bst files obsolete allowing the formatting to be done by LaTeX and not an add on program like BibTeX, but it didn't completely eliminate the need for an add on program and gave us Biber. What is it that add on programs like Biber and MakeIndex do that cannot be done in "pure" LaTeX? If a few new "simple" primitives were introduced, could these add on programs be implemented in LaTeX? By simple I mean not creating a
\biber primitive. Could they be implemented directly and "easily" in LuaTeX?
The creation of
Good multi-field sorting can't really be done using regexps. You have to be able to compare fields in a user definable order, with each field match locally allowing for case-sensitivity, ascending/descending, perhaps UTF-8 variable weight punctuation etc. While this is probably theoretically doable with a (very expanded, serious, like Perl's) regexp engine it would be an utter nightmare and slow. This is usually done with some tricky array acrobatics.
The uniqueness systems in biber (uniquename and especially uniquelist) require lots of interdependent looping constructs which are unbounded but with guaranteed eventual exits. I suspect TeX would eat memory like anything doing this.
You also need bibtex format parser which is fast (biber uses the btparse library in C via the Text::BibTeX module, modified for UTF-8 passthrough). Parsing and grammar in TeX? Shudder.
Lua is another matter. It's probably possible and would be great. But you'd be surprised how much biber does internally - porting it would be quite a job. You could probably build an interface to the btparse library which would be the first step.
You do not even need (in theory) to introduce a new primitve.
eTeX (the behind-interpreter, not TeX or LaTeX) is Turing complete: Are there any disadvantages of TeX being Turing complete?
It is capable of reading and writing files.
So in theory, you don't need it. On the other hand, in real, you have two problems:
Making real large programs in eTeX is quite dificult (for, see the code of
The Turing-completeness is a theory based on "if the (memory) capacity was infinite". But it is not infinite, and running
bibtexmeans sorting large data structures by some criteria, and even loading these structures might be a problem in TeX itself. (Remember that you can call
.bibfile with thousands of entries.)
I cannot speak of LuaTeX, because I don't know Lua really.
While the interpreter is Turing complete as tohecz correctly pointed out the main reason is that the job of these external programs would be hard to implement in TeX.
For LuaTeX the answer is more or less historical. LuaTeX was never meant to replace biber but I am quite confident that there are very few technical issues to overcome to create a biber library instead of an executable and then call this library from LuaTeX. In the long term the functionality might even be implemented in LuaTeX but considering how well biber/biblatex work together the question is whether someone is willing to reimplement this in Lua.