How expressive is TeX in terms of being a compiler or a preprocessor of other languages? One splendid example is the listings package in which TeX is changed so as to kind of recognize the kind of tokens of almost all programming languages. So, yes, TeX can be made into a kind of preprocessor.

But can TeX be a parser? Can you implement something like YACC in TeX, so that you define a BNF, and then the rest of the input is parsed by this BNF.

Let's simplify the question a bit. is at least theoretically possible (can't cheat by emulating a Turing machine) to make such lex.tex file so that the input file

 \input lex.tex
   id -> { some($); }
   hello world

will actually behave such as a lexical analyzer with three sections: defintions, rules, and input. The first two sections define how the INPUT section is processed, producing

some(hello) some(world)

If this is possible and realistic in some way, then one may aspire for more, yacc.tex, and then perhaps, a universal processor/compiler all written in tex? Wouldn't that be exciting?

  • 4
    TeX is Turing-complete. While it is possible, it is not immediately realistic. Even if implemented, I imagine such a parser would be incredibly slow. – Sean Allred Feb 5 '17 at 20:05
  • 5
    I've implemented a more or less complete parser for XML in TeX, including DTD syntax and namespace resolution, for example so can tex be a parser sounds like it ought to have "yes" as an answer, but I didn't implement lex(which is part of a parser generator rather than a parser, doable but tedious) – David Carlisle Feb 5 '17 at 20:07
  • 4
    while it would be theoretically possible to do in tex, if doing this now it would make far more sense to do it in luatex and use lpeg – David Carlisle Feb 5 '17 at 23:36
  • 1
    Your question triggered my memory of this answer of mine, tex.stackexchange.com/questions/332012/…. I mention this because the listofitems package provides powerful parsing capability, and in the [updated] answer cited, is used to digest alternative syntax for equations and [tries to] convert it to LaTeX, by producing a long string of "spaghetti" or "DNA" code that compiles into the desired result. – Steven B. Segletes Feb 9 '17 at 19:48

Browse other questions tagged or ask your own question.