I've read here and there that ConTeXt can produce XML output. We also have, from time to time, questions about converting LaTeX to different formats. On the basis that "The only parser for TeX is tex", if latex could produce text output instead of PDF then it would be possible to write a style file to convert reasonable input to a different markup language.

Would this be possible?

Bit of background: I encounter this "can we convert from LaTeX?" question in the context of the nLab where the input format is Markdown+iTeX (iTeX not being anything to do with Knuth's proposal but a subset-of-LaTeX-to-MathML converter) but people often have snippets of LaTeX articles that they want to include. So converting all the way to XHTML+MathML via, say, tex4ht isn't the right option. I wrote a Perl script that reimplements much of TeX to do this, but after doing so realised that my style files would work in ordinary LaTeX and produce the "right" output, except that they would be embedded in a PDF. So if I could just persuade TeX to produce text, I'd be almost there. Of course, I could try to extract the text from the PDF but that "feels wrong" and I'd worry about extra stuff sneaking in by accident.

  • 2
    What's wrong with a simple pdftotext postprocessor. Extra stuff can sneak in by accident in whatever solution you try.
    – Aditya
    May 31, 2011 at 18:59
  • @Aditya: I know, but I feel that I have more understanding of what TeX can produce itself. If it is a genuine text file, then I'd know what to filter out afterwards. It's partially to do with my own lack of understanding of things like the PDF format. I guess what I want is for TeX to send all "printable" characters to a "write". May 31, 2011 at 20:23
  • @Aditya: How does ConTeXt do this? (I got the quip about parsing TeX from your blog, by the way.) May 31, 2011 at 20:23
  • I am not 100% sure how ConTeXt does it. The code is in back-exp.lua. IIUC, it builds a tree of the entire document in memory...each macro and environment defined using \define and \definestartstops hooks into that tree; all the core envuronments (itemize, enumerate, section, etc) hook into the tree. Then, at the end of the document, ConTeXt simply serializes the tree and writes it to a separate text file.
    – Aditya
    May 31, 2011 at 21:34
  • 1
    @Aditya: I've now implemented something that uses pdftotext as the last stage ... and it works! I was able to produce the source text to this page: ncatlab.org/nlab/show/equivariant+tubular+neighbourhoods by writing a LaTeX document with a special style file. So could you assemble your various comments in to an answer which I can accept? (If it's alright by you, after you've done that then I might add some details on exactly what I did, but I'd like to give you the credit for the solution.) Jun 21, 2011 at 21:45

4 Answers 4


The underlying solution is of course the same for ConTeXt and LaTeX: you need to have a way of changing what macros do such that they write the correct output rather than typesetting. This is also much the same as tex4ht does. The advantage ConTeXt has is that the macros are provided mainly by one focussed group, and they include the necessary 'back end' to make that conversion easy. To do the same for LaTeX, you need to handle all of the macros that might be present, which is a problem given the number and variety of LaTeX packages. So while in principal it's possible, the implementation is a severe challenge.

(With my 'LaTeX3 hat' on, this is an obvious area to bear in mind when defining an updated format. To do that, you need to have a much more 'regular' syntax and input than is often the case with LaTeX files at present. Again, I think ConTeXt shows how this can be done as it is already good on keeping the input within it's own structures.)

  • I guess the key point is it partly depends on what package coverage you want. Doing the LaTeX2e kernel, with no hacks in the source, would be relatively straight-forward. Throw in some low-level TeX, a few kernel hacks and a load of packages, and things are different.
    – Joseph Wright
    May 31, 2011 at 17:31
  • Changing the macros is fine. I've basically already done that as part of my Perl script, and I can add more on a case-by-case basis (I'm not looking for a global solution). It's the fundamental getting LaTeX to produce ASCII text rather than DVI or PDF or ... that I'm interested in here. As I understand it, tex4ht works by adding hooks in to the DVI or PDF and post-processing it. I'd prefer to simply have tex output the text directly. May 31, 2011 at 17:48
  • @Andrew: Ah, I see.
    – Joseph Wright
    May 31, 2011 at 17:59
  • 2
    A blasphemous suggestion: write a few TeX (rather ConTeXt) macros that can read LaTeX syntax and convert them to ConTeXt Apart from syntax for references, other things is LaTeX are easily convertible to ConTeXt syntax. There was an old project with Brookes Moses (I think) that tried to do this. If you can find that, then most of the ground work is already done.
    – Aditya
    May 31, 2011 at 21:40
  • 1
    @Aditya: Yes, that was my project. It wasn't that hard to do, just a bit tedious, and I didn't really get all that far with it -- but what I have is here: files.dpdx.net/tex/context/latex-compat. I hadn't thought about it in years; I'm amazed that someone remembers it. Jun 2, 2011 at 15:59

It is possible to achieve what you want, provided you do not want TeX to act as a parser. In my opinion, part of the success of TeX, is that it has managed to transform itself over the years to act as a language transformation tool. First it was TeX->Postscript and now it is TeX->pdf. Tralics has been fairly successful to produce TeX->XML.

But, I think one needs to look at the problem from a different angle. With todays available technologies one, needs to have a "Universal Mark-up Language". Markdown and Yaml are scaled down tools and can never be able to be full document description languages, so going that route will limit one's efforts.

Sometime back, I designed a CMS based on text files. All mark-up was in plain text and fragments from Wikipedia's markup language. I would load the text file via php and then filter the input and produce the HTML page.

{{feature-image: http://localhost/images/sample102.jpg }}
{{feature: A collection is like a puzzle...}}

The feature-image was a div and the feature-text the caption. I had commands for image-credits and the like.

Now this is not so difficult to produce with TeX. So my proposal is to actually use TeX to write an intermediate mark-up in a text file then parse with your language of choice to achieve what you wish.

Workflow depending on targets can be one of the following:

   TeX->Intermediate MarkUp->HTML
   TeX->plain text
   Intermediate MarkUp->Translator (javascript, perl, python, 
                        ruby, php, your language) ->TeX

In a nutshell, retain TeX and output into a new mark-up language. Markdown and other technologies can be a subset of this.

%% create file and open it to write





  \immediate\write\file{\hash@@ #1}


\Section{Test Section}

\image{http://tex.stackexchange.com/questions/15440/parsing-files-through-lua-tex}{This is the caption}


The minimal is just a proof of concept. Main idea here is not to redefine the LaTeX commands but rather add new ones with switches for other mark-up.

  • That sounds pretty similar to what I want, apart from that I don't understand your statement "provided you do not want TeX to act as a parser". I want TeX to properly expand its input, acting accordingly, but I want the output file format to be text. I can do this with my perl version of TeX, but after writing it I felt a better solution would be possible if only I could make TeX output actual text. May 31, 2011 at 18:46
  • @Andrew Stacey What I mean as long as one does not expect TeX to read a file with alien mark-up like html and try to convert. Everytime TeX writes to an auxiliary file or a log it writes text! PGF can export a table to html. To transform to mathml I would add the tags, and include the maths verbatim and write to the file straight. Going the LuaTeX way might make things easier with the file operations.
    – yannisl
    May 31, 2011 at 18:57
  • Yiannis: Right, that's fine. All I want is for TeX to process TeX, nothing more. I feel that I can handle the actual conversion bit. It's the sheer mechanics of outputting the main stream to a text file that I don't know how to do. May 31, 2011 at 20:24
  • @Andrew OK, I will post a minimal in the morning, getting too late here!
    – yannisl
    May 31, 2011 at 21:05
  • Yiannis: It looks promising, but how does the ordinary text get in to the new file? Jun 1, 2011 at 9:05

In the interests of completeness, I feel I should record my current solution (my gut instinct is that this is the best method, but the exact implementation could probably do with improvement). That is to take a leaf out of the ConTeXt book and use LuaTeX. LuaTeX provides me with some hooks to get at the processed output of TeX just before it is packed into boxes and shipped out.

Specifically, I used the hook pre_linebreak_filter to dig out the contents of each line. It's actually not a long way from the idea of StrondBad's answer, just without all the unnecessary stuff and with a bit more control over things like groupings.

My implementation can be found at my github repository for my project. I don't think it can be just cut-and-pasted into something else as it is integrated with some other parts of my project so someone wanting to use the idea would need to untangle it a bit. The crucial files are the Lua file textoutput.lua, mainly the function list_elements, and the TeX file internettext.code.tex, specifically the "true" branch of the conditional \@ifundefined{directlua} on line 53 (at time of writing).

Also, as I said at the outset, although I think this is the right strategy, it is probably not the best implementation.


The wordcount package sets up LaTeX such that every character, space, line break, etc is added to the log file. This means that

Hello World
ff and fi
\(y=\alpha x+\beta\)

results in a log file that includes:

...\3.08632 H
...\3.08632 e
...\3.08632 l
...\3.08632 l
...\3.08632 o
...\3.08632 W
...\3.08632 o
...\3.08632 r
...\3.08632 l
...\3.08632 d

...\3.08632 ^^[ (ligature ff)
...\3.08632 a
...\3.08632 n
...\3.08632 d
...\3.08632 ^^\ (ligature fi)

...\3.08632 y
...\3.08632 =
...\3.08632 x
...\3.08632 +
...\3.08632 ^^L

.......\3.08632 a
.......\3.08632 b
.......\3.08632 c

I am not sure if it is possible to get LaTeX to write cleaner output in a separate file or how to deal with the full set of unicode characters, but the whole idea of wordcount is that you can then parse the log file for characters and spaces.

  • Thanks for the answer. One issue with this approach is that the log file very quickly becomes excessively large. But it's a good idea for short documents. Aug 31, 2016 at 20:17
  • @LoopSpace for a book or a thesis, the file size might be a problem, but the log file is under 2 MB for an academic article of around 10 pages.
    – StrongBad
    Aug 31, 2016 at 21:16
  • It's been a while since I looked at this, but I did do some experiments with this and found that producing the log file made TeX run slower than I would have liked (particularly as I tend to recompile quite frequently). Aug 31, 2016 at 21:26

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .