Recommended workflow? LaTeX -> ePub with math viewable on iPad

Question

This started as a comment in an older thread, but I thought I repost it here as a question to the forum...

I'm looking for a workflow that will take me efficiently and reliably from existing LaTeX documents (e.g., books) with lots of (sometimes) complex embedded math to high-quality ePubs that will display nicely in iBooks under iOS6 and, ideally, on other readers as well (lower priority).

I've been looking at LateXML, TeX4ht, plasTeX, XhtmLaTeX, pandoc, ebook-convert, and the tbook DTD (which starts with XML rather than LaTeX and thus looks promising primarily for new documents). All of these tools seem to have a lot to offer, but none of them, as far as I can tell, gets me to the finish line without what appears to be significant manual intervention (implying a need for significant learning/debugging as well). I haven't been able to decide yet which pathway is worth investing the time and energy on. Looking forward to an up-to-date assessment.

I'll note that I already published an ePub textbook with equations rendered as SVG, and it displayed very nicely in iBooks under iOS5. Unfortunately, iOS6 broke it, and the eBook is now a virtual paperweight. I'm trying to recover and get the same book back into a usable ePub format that renders equations nicely.

I don't mind investing in a commercial product if it's the least painful means to solve the problem, but I'd prefer to build a script-based workflow based on open-source tools for Mac OS X.

Answer 1

The basic problem you're running into is the immaturity of the epub format.

It's very easy for a publisher to produce an epub 3 of a novel if they're already set up to produce epub 2, and the epub 3 version will typically work fine on readers designed for epub 2.

However, there seems to have been very slow progress to date on getting publishers and device manufacturers going on the new and fancy epub 3 features such as mathml. It's probably an economic issue. Publishers can't wave a magic wand over their catalogs and produce mathml for all the equations that appear in all their books; it would be an expensive case-by-case slog for them. Their profit cows are K-12 and college textbooks, and most of those were designed in a large format that is not suitable for handheld devices. Since the publishers have little economic motivation to start selling epub 3 with mathml, the hardware manufactures have little economic motivation to start supporting mathml in their devices. Apple seems to have partial, lousy support on some of their devices. Meanwhile, Amazon shows no interest at all in making math work on their format. I wish I could hold out some hope that this would get fixed sooner rather than later, but, frankly, the experience with mathml in the browser doesn't encourage such a hope. For example, Wikipedia still doesn't do mathml after all these years. Because of all these factors, there basically is not much progress yet in getting good open-source tools for producing epub 3+mathml.

Since epub 3+mathml isn't likely to become good and mature in the near future, it's worth considering holding off completely on putting a lot of work into converting a book into the format.

Having said that, I do have some experience experimenting with doing this. Basically epub is xhtml, so if you can get xhtml+mathml output from your latex, you're not that far from having a working epub 3+mathml book. There are already lots and lots of tools for converting latex to html. (You listed them in your question.)

There is an open-source program called calibre that will convert any valid XHTML 1.1 + CSS 2.1 document to valid epub 2. What I did was to generate xhtml output, translate it to epub 2 using calibre, and then patch the epub 2 to try to make it valid epub 3. (Calibre is not capable of outputting epub 3+mathml according to the spec, and unfortunately the developer seems to have zero enthusiasm for making it do so: http://www.mobileread.com/forums/showpost.php?p=1904668&postcount=7 . ) Both my book and the scripts I wrote for patching are open source, so anyone who wants to tinker with them is welcome to: https://github.com/bcrowell/calculus . From a brief look at Andrew Stacey's page, it looks like the approach he's used is fairly similar.

The best epub 3 output I was able to produce is here: http://www.lightandmatter.com/calc/ . I don't own an iAnything, but I got one of my students to show it to me on his device, and basically it seemed to have worked to the extent that Apple had implemented mathml correctly on the device. (Their implementation at that time, about a year ago, was pretty awful, though. E.g., integrals signs appeared as boxes.)

Please don't even think about trying to use mathjax. AFAIK most readers don't support javascript in epub 3 at all. Frankly, I wouldn't want the feature activated on an ebook reader I owned. (Think ads, animations, annoying idiosyncratic user interfaces like the unskippable stuff at the beginning of a DVD.) Let's keep in mind that mathjax is a beautifully executed kludge, whose sole purpose is to cover up for Microsoft's failure to impement mathml in IE. Even on a desktop computer, its performance can be bad, and on a handheld device it would probably be atrocious. The epub 3 standard provides a standard way to do mathml, so that's the right way to do mathml on those devices.

Testing is a problem. I use the open-source java program epubcheck to check whether my epub output is valid. However, just because epubcheck says it's valid, that doesn't mean it will render correctly on handheld devices. There will probably be a period of a decade or more during which some people's devices can handle epub 3+mathml and other people's won't. Calibre 0.8.66+ can display epub+mathml properly, but it uses mathjax, which is completely different from the implementation of mathml on handheld readers. Calibre does not currently output epub 3 at all, which is why I wrote the scripts to patch its output to make it valid epub 3.

Answer 2

(This is really a comment on Ben's answer but is a bit long.)

Broadly I agree with what Ben has said about the current state of play. The outlook is a bit gloomy. However, I think that there are a couple of glimmers of light that give me a little hope for some progress to occur.

1. As Ben says, ePub is quite like XHTML. So if you can produce a valid XHTML+MathML document, then you can produce ePub3 via calibre's conversion tool (when reading calibre's documentation one has to be careful to distinguish between the viewer and the rest, I believe that it is the viewer that doesn't support MathML - but see below). Support for MathML in browsers is on the up now that Webkit (whence Safari and Chrome) officially supports it (see http://caniuse.com/mathml for the current state of play). So if you regard producing an ebook as an offshoot of producing a webpage, in that you do all the hard work for the webpage and then tweak it a little to make the ebook, it doesn't seem like such a bad idea to produce an ebook version.

2. Building on that, Apple's iBooks is based on Webkit so whatever Webkit does, iBooks will also do (sort of). Not only does this mean that iBooks' maths support is better than one would expect if they had to do it themselves, but also that it does seem in the computer industry that once one team does something everyone else feels that they have to do it too. So hopefully the fact that it is possible with iBooks will spur others on to do the same.

3. Calibre's viewer does now support MathML using MathJaX: http://manual.calibre-ebook.com/typesetting_math.html That is, MathJaX is embedded in to the viewer you don't need to add it to your ebook. Note that this now means that it is pretty much pointless adding MathJaX to your ebook: most ebook readers don't support javascript and two of the main ones that do are Calibre and iBooks both of which now have their own way of handling MathML.

4. It's chicken-and-egg. No-one's going to be interested in producing mathematical ebooks without decent software to read them on, but no-one's going to write the software without the demand for it from the books. But if there are some examples of good mathematical ebooks, even if only running on a certain platform, which really show off the benefits of the media then more people are going to want to produce them. Once again, the fact that you can produce an ebook via an XHTML page means that the real effort in producing such a book is not wasted.

I've not yet looked at Ben's set-up (but I will). My code is available at http://www.math.ntnu.no/~stacey/code/LaTeXporter (either grab the files or point bzr at that location). You also need itex2MML (http://www.math.ntnu.no/~stacey/code/itexToMML - I've added a few bits to the official version) to do the final conversion. My code is extremely experimental. There's a file there, diffloop.tex, which is an example of using the code. To compile it, you need to do:

pdflatex diffloop.tex
pdftotext -enc ASCII7 -nopgbrk -layout diffloop.pdf
itex2MML.pl < diffloop.txt > diffloop.xhtml

where itex2MML.pl is the script:

#! perl -w

use ItexToMML;
use MathML::Entities;

my $itex = new ItexToMML;

my $src;

while (<>) {
    $src .= $_;
}

my $res = $itex->html_filter($src);

my $utf8 = name2numbered($res);

print $utf8;

(MathML::Entities is available on CPAN)

The point of converting the entities is that Webkit doesn't handle named entities correctly.

I then converted it to epub using Calibre. You can see the results at http://loopspace.mathforge.org/discussion/15/the-differential-topology-of-loop-spaces

Answer 3

You can use tex4ebook tool by Michal Hoftich (https://github.com/michal-h21/tex4ebook). This tool can convert not only to epub, but also to epub3 and mobi. There is interesting example at https://github.com/michal-h21/epub3sample (and converted math book with LaTeX source).

Converted book uses svg for math, and looks nice, but the only epub 3 reader I have is readium. The tool uses TeX4ht as a main enfgine and some lua scripts I am just testing tex4ebook (with TL 2012). It simply works… Probably one will need customize configuration file to meet ones needs.

It will be interesting to know how the results looks on ipad.

Answer 4

After a lot of experimenting with epub and coming up with the same limitations as listed above, I choose to stay with PDF as my format. In my custom class, I have an option to specify document size, letter for 8.5x11, and two ipad versions, one for vertical and one for horizontal. I mainly stick with either letter or ipad vertical.

This way all the content is in latex and I can use the geometry package to customize the layout for the ipad. ie small margins, making the paper size the screen size of the iPad.

I have a lot of equations and tables and I found PDF to be the best and most consistent way for the time being.

Answer 5

I've found a workaround which is sufficient for me :

• first I use htlatex (tex4ht), but the images are too small
• I edit tex4ht.env, setting Gvipng -T tight -x2800-D144, but the small equations are too large …
• I use the following script in python :

    #!/usr/bin/env python
    import os
    import Image as im
    cmd=os.popen("ls *png")
    res=cmd.read()
    l=res.split()
    cmd=os.popen("ls *lg")
    res=cmd.read()
    lg=res.split()[0]
    cmd=os.popen("ls *idv")
    res=cmd.read()
    idv=res.split()[0]
    filelg=open(lg)
    dic={}
    import re
    def iniat():
     for line in filelg:
      if line.find("- needs")!=-1:
        line2=line.split()
        a=line2[3]
        b=line2[5]
        a2=re.split('[\[\]]', a)
        a3=a2[1]
        dic[b]=a3
    def exec1():
     print dic
     for file in l:
      ima=im.open(file)
      w,h=ima.size
      if w<1400:
       a=dic[file]
       s="dvipng -T tight -x 2800 -D 72 -bg Transparent -pp {}:{} {} -o  {}".format(a, a, idv, file)
       os.system(s)       
    iniat()
    exec1()

• this script reduce the small equations. The I convert the index.html file with calibre
• but calibre resizes the images (there is certainly a parameter to avoid that). So I unzip the epub, change the images with the previous ones, and rezip the epub. The result works fine of my kindle.

Best regards.