# Recommended workflow? LaTeX -> ePub with math viewable on iPad

This started as a comment in an older thread, but I thought I repost it here as a question to the forum...

I'm looking for a workflow that will take me efficiently and reliably from existing LaTeX documents (e.g., books) with lots of (sometimes) complex embedded math to high-quality ePubs that will display nicely in iBooks under iOS6 and, ideally, on other readers as well (lower priority).

I've been looking at LateXML, TeX4ht, plasTeX, XhtmLaTeX, pandoc, ebook-convert, and the tbook DTD (which starts with XML rather than LaTeX and thus looks promising primarily for new documents). All of these tools seem to have a lot to offer, but none of them, as far as I can tell, gets me to the finish line without what appears to be significant manual intervention (implying a need for significant learning/debugging as well). I haven't been able to decide yet which pathway is worth investing the time and energy on. Looking forward to an up-to-date assessment.

I'll note that I already published an ePub textbook with equations rendered as SVG, and it displayed very nicely in iBooks under iOS5. Unfortunately, iOS6 broke it, and the eBook is now a virtual paperweight. I'm trying to recover and get the same book back into a usable ePub format that renders equations nicely.

I don't mind investing in a commercial product if it's the least painful means to solve the problem, but I'd prefer to build a script-based workflow based on open-source tools for Mac OS X.

-
you can configure tex4ht to use mathjax for displaying mathematics and then include mathjax in epub. see albany.edu/~hammond/demos/Html5/arXiv and boolesrings.org/krautzberger/2013/01/13/… –  michal.h21 Jan 27 '13 at 19:46
I have a LaTeX class that I've used for converting to ePub (see loopspace.mathforge.org/discussion/15/… for the result). There is never going to be a perfect converter, though, so every pathway will need some tweaking. My version enables you to do all the tweaking in the original LaTeX file. If you would like me to take a look to see if your book would work with my class, feel free to send it to me. –  Loop Space Jan 27 '13 at 20:09
@AndrewStacey could you share your LaTeX class please? The result really is quite beautiful and it would be great to have a rather decent converted ePub to begin with. –  user24276 Jan 28 '13 at 10:40
Yes, it would be helpful to see @AndrewStacey's class. –  michael okane Jan 29 '13 at 5:34
Starting with a bare bones valid ePub that I created using pandoc from XHTML with MathML, I tried the steps described in Krautzberger's blog to add MathJax to the ePub file. Unfortunately, when I view the result in iBooks, it still displays the garbled MathML; no sign that the MathJax had any effect, and I'm not equipped to discern the reasons. Life would be so much easier if Apple would add MathJax support to iBooks. –  Grant Petty Jan 30 '13 at 15:30

The basic problem you're running into is the immaturity of the epub format.

It's very easy for a publisher to produce an epub 3 of a novel if they're already set up to produce epub 2, and the epub 3 version will typically work fine on readers designed for epub 2.

However, there seems to have been very slow progress to date on getting publishers and device manufacturers going on the new and fancy epub 3 features such as mathml. It's probably an economic issue. Publishers can't wave a magic wand over their catalogs and produce mathml for all the equations that appear in all their books; it would be an expensive case-by-case slog for them. Their profit cows are K-12 and college textbooks, and most of those were designed in a large format that is not suitable for handheld devices. Since the publishers have little economic motivation to start selling epub 3 with mathml, the hardware manufactures have little economic motivation to start supporting mathml in their devices. Apple seems to have partial, lousy support on some of their devices. Meanwhile, Amazon shows no interest at all in making math work on their format. I wish I could hold out some hope that this would get fixed sooner rather than later, but, frankly, the experience with mathml in the browser doesn't encourage such a hope. For example, Wikipedia still doesn't do mathml after all these years. Because of all these factors, there basically is not much progress yet in getting good open-source tools for producing epub 3+mathml.

Since epub 3+mathml isn't likely to become good and mature in the near future, it's worth considering holding off completely on putting a lot of work into converting a book into the format.

Having said that, I do have some experience experimenting with doing this. Basically epub is xhtml, so if you can get xhtml+mathml output from your latex, you're not that far from having a working epub 3+mathml book. There are already lots and lots of tools for converting latex to html. (You listed them in your question.)

There is an open-source program called calibre that will convert any valid XHTML 1.1 + CSS 2.1 document to valid epub 2. What I did was to generate xhtml output, translate it to epub 2 using calibre, and then patch the epub 2 to try to make it valid epub 3. (Calibre is not capable of outputting epub 3+mathml according to the spec, and unfortunately the developer seems to have zero enthusiasm for making it do so: http://www.mobileread.com/forums/showpost.php?p=1904668&postcount=7 . ) Both my book and the scripts I wrote for patching are open source, so anyone who wants to tinker with them is welcome to: https://github.com/bcrowell/calculus . From a brief look at Andrew Stacey's page, it looks like the approach he's used is fairly similar.

The best epub 3 output I was able to produce is here: http://www.lightandmatter.com/calc/ . I don't own an iAnything, but I got one of my students to show it to me on his device, and basically it seemed to have worked to the extent that Apple had implemented mathml correctly on the device. (Their implementation at that time, about a year ago, was pretty awful, though. E.g., integrals signs appeared as boxes.)

Please don't even think about trying to use mathjax. AFAIK most readers don't support javascript in epub 3 at all. Frankly, I wouldn't want the feature activated on an ebook reader I owned. (Think ads, animations, annoying idiosyncratic user interfaces like the unskippable stuff at the beginning of a DVD.) Let's keep in mind that mathjax is a beautifully executed kludge, whose sole purpose is to cover up for Microsoft's failure to impement mathml in IE. Even on a desktop computer, its performance can be bad, and on a handheld device it would probably be atrocious. The epub 3 standard provides a standard way to do mathml, so that's the right way to do mathml on those devices.

Testing is a problem. I use the open-source java program epubcheck to check whether my epub output is valid. However, just because epubcheck says it's valid, that doesn't mean it will render correctly on handheld devices. There will probably be a period of a decade or more during which some people's devices can handle epub 3+mathml and other people's won't. Calibre 0.8.66+ can display epub+mathml properly, but it uses mathjax, which is completely different from the implementation of mathml on handheld readers. Calibre does not currently output epub 3 at all, which is why I wrote the scripts to patch its output to make it valid epub 3.

-

(This is really a comment on Ben's answer but is a bit long.)

Broadly I agree with what Ben has said about the current state of play. The outlook is a bit gloomy. However, I think that there are a couple of glimmers of light that give me a little hope for some progress to occur.

1. As Ben says, ePub is quite like XHTML. So if you can produce a valid XHTML+MathML document, then you can produce ePub3 via calibre's conversion tool (when reading calibre's documentation one has to be careful to distinguish between the viewer and the rest, I believe that it is the viewer that doesn't support MathML - but see below). Support for MathML in browsers is on the up now that Webkit (whence Safari and Chrome) officially supports it (see http://caniuse.com/mathml for the current state of play). So if you regard producing an ebook as an offshoot of producing a webpage, in that you do all the hard work for the webpage and then tweak it a little to make the ebook, it doesn't seem like such a bad idea to produce an ebook version.

2. Building on that, Apple's iBooks is based on Webkit so whatever Webkit does, iBooks will also do (sort of). Not only does this mean that iBooks' maths support is better than one would expect if they had to do it themselves, but also that it does seem in the computer industry that once one team does something everyone else feels that they have to do it too. So hopefully the fact that it is possible with iBooks will spur others on to do the same.

3. Calibre's viewer does now support MathML using MathJaX: http://manual.calibre-ebook.com/typesetting_math.html That is, MathJaX is embedded in to the viewer you don't need to add it to your ebook. Note that this now means that it is pretty much pointless adding MathJaX to your ebook: most ebook readers don't support javascript and two of the main ones that do are Calibre and iBooks both of which now have their own way of handling MathML.

4. It's chicken-and-egg. No-one's going to be interested in producing mathematical ebooks without decent software to read them on, but no-one's going to write the software without the demand for it from the books. But if there are some examples of good mathematical ebooks, even if only running on a certain platform, which really show off the benefits of the media then more people are going to want to produce them. Once again, the fact that you can produce an ebook via an XHTML page means that the real effort in producing such a book is not wasted.

I've not yet looked at Ben's set-up (but I will). My code is available at http://www.math.ntnu.no/~stacey/code/LaTeXporter (either grab the files or point bzr at that location). You also need itex2MML (http://www.math.ntnu.no/~stacey/code/itexToMML - I've added a few bits to the official version) to do the final conversion. My code is extremely experimental. There's a file there, diffloop.tex, which is an example of using the code. To compile it, you need to do:

pdflatex diffloop.tex
pdftotext -enc ASCII7 -nopgbrk -layout diffloop.pdf
itex2MML.pl < diffloop.txt > diffloop.xhtml


where itex2MML.pl is the script:

#! perl -w

use ItexToMML;
use MathML::Entities;

my $itex = new ItexToMML; my$src;

while (<>) {
$src .=$_;
}

my $res =$itex->html_filter($src); my$utf8 = name2numbered($res); print$utf8;


(MathML::Entities is available on CPAN)

The point of converting the entities is that Webkit doesn't handle named entities correctly.

I then converted it to epub using Calibre. You can see the results at http://loopspace.mathforge.org/discussion/15/the-differential-topology-of-loop-spaces

-
You've done a very nice job on the epub version of your book. It looks sweet in calibre. Good to know about converting to numbered entities; this solves some problems I'd been having before. I ran your book through epubcheck and got a large number of errors. Basically this is because calibre outputs epub 2, and mathml is illegal in epub 3. I don't know if this is an issue for mathml-capable handheld devices, but it could, e.g., cause online stores to reject an epub constructed in this way. That's why I wrote the scripts to patch calibre's epub 2 into valid epub 3. –  Ben Crowell Jan 31 '13 at 20:00
It's good to know that calibre can now handle mathml. However, this doesn't really tell us whether a given epub+mathml book will display properly on handheld devices. I added a paragraph to the end of my answer about testing. –  Ben Crowell Jan 31 '13 at 20:02
@BenCrowell Thanks for the heads-up about Calibre outputting epub2. I've just spent a little time making a true epub3 (and the version on my webpage is the latest one). It almost validates, just some weird issues with fragment identifiers not being defined (even though they are). –  Loop Space Feb 1 '13 at 13:27
What method did you use for converting from epub 2 to 3? –  Ben Crowell Feb 1 '13 at 14:53
@BenCrowell I didn't. I started again from the XHTML+MathML version and built an ePub3 around it. I found some tutorials on the IBM website with a simple MathML example. Ideally, when I've figured out how the pieces fit together, I'll add this into my LaTeX class, but as the validator shows there are still some parts that I don't really understand so I need to get them clear first. –  Loop Space Feb 1 '13 at 15:01
show 1 more comment

After a lot of experimenting with epub and coming up with the same limitations as listed above, I choose to stay with PDF as my format. In my custom class, I have an option to specify document size, letter for 8.5x11, and two ipad versions, one for vertical and one for horizontal. I mainly stick with either letter or ipad vertical.

This way all the content is in latex and I can use the geometry package to customize the layout for the ipad. ie small margins, making the paper size the screen size of the iPad.

I have a lot of equations and tables and I found PDF to be the best and most consistent way for the time being.

-
This is a reasonable option (though it means missing out on the benefits of ePub), and you may be interested in the iTeX App (itunes.apple.com/us/app/itex/id419708947?mt=8) if you haven't already come across it. –  Loop Space Jan 31 '13 at 15:05
This is a sensible approach given the poor support for epub+mathml in devices. However, you're having to create two pdf files for a single device (vertical and horizontal). Now multiply that by the number of sizes in which different models of that device exist. Now multiply by the number of different devices. Epub, at least in theory, takes care of this with a single file for all sizes of screens. –  Ben Crowell Jan 31 '13 at 16:36

You can use tex4ebook tool by Michal Hoftich (https://github.com/michal-h21/tex4ebook). This tool can convert not only to epub, but also to epub3 and mobi. There is interesting example at https://github.com/michal-h21/epub3sample (and converted math book with LaTeX source).

Converted book uses svg for math, and looks nice, but the only epub 3 reader I have is readium. The tool uses TeX4ht as a main enfgine and some lua scripts I am just testing tex4ebook (with TL 2012). It simply works… Probably one will need customize configuration file to meet ones needs.

It will be interesting to know how the results looks on ipad.

-
I am also curios how the epub3 sample looks in ipad, I have tested in only with readium and azardi. –  michal.h21 Jun 21 '13 at 20:34

I've found a workaround which is sufficient for me :

• first I use htlatex (tex4ht), but the images are too small
• I edit tex4ht.env, setting Gvipng -T tight -x2800-D144, but the small equations are too large …
• I use the following script in python :
    #!/usr/bin/env python
import os
import Image as im
cmd=os.popen("ls *png")
l=res.split()
cmd=os.popen("ls *lg")
lg=res.split()[0]
cmd=os.popen("ls *idv")
idv=res.split()[0]
filelg=open(lg)
dic={}
import re
def iniat():
for line in filelg:
if line.find("- needs")!=-1:
line2=line.split()
a=line2[3]
b=line2[5]
a2=re.split('[]', a)
a3=a2[1]
dic[b]=a3
def exec1():
print dic
for file in l:
ima=im.open(file)
w,h=ima.size
if w<1400:
a=dic[file]
s="dvipng -T tight -x 2800 -D 72 -bg Transparent -pp {}:{} {} -o  {}".format(a, a, idv, file)
os.system(s)
iniat()
exec1()

• this script reduce the small equations. The I convert the index.html file with calibre
• but calibre resizes the images (there is certainly a parameter to avoid that). So I unzip the epub, change the images with the previous ones, and rezip the epub. The result works fine of my kindle.

Best regards.

-
you can use tex4ebook (github.com/michal-h21/tex4ebook) to compile your document to epub or kindle without need of using calibre. it uses tex4ht in the background, so you can use your custom tex4ht.env file –  michal.h21 Jun 21 '13 at 19:34
@michal.h21 I prepare lessons for my students so that they can read on their iPad. I want to try real test but :fab@debian:~$texlua /home/fab/texmf/tex/latex/tex4ebook-master /tex4ebook.lua /home/fab/texmf/tex/latex/tex4ebook-master/tex4ebook.lua:113: invalid escape sequence near '\$' –  Fabrice Jul 30 '13 at 8:51