Tools for automating document compilation

Question

A lot of people write makefiles that say something like

paper.pdf: paper.tex
        pdflatex paper
        bibtex paper
        pdflatex paper
        pdflatex paper

To handle re-running TeX to get new/changed references and so forth. Is there a better way to do this?

Answer 1

Latexmk is one possibility, although I've never used it myself.

Answer 2

Try arara. It also has a great manual.

\expandafter\thecomment

Arara provides us with ways (rules) to compile the document that are specific to the document. The rules for compilation are put inside the document. Hence, compilation boils down to a simple arara yourtexfile.

First, install arara and make sure that the arara executable is included in system path. Arara also needs java runtime environment (jre) to be installed. To use arara we should add the rules inside the document itself as in the following code. Save this code as yourfile.tex and execute arara yourfile:

% arara: pdflatex: {synctex: yes}
% arara: makeindex: { style: yourfile }
% arara: biber
% arara: pdflatex: {synctex: yes}
% arara: pdflatex: {synctex: yes}
% arara: clean: {files: [yourfile.aux, yourfile.idx, yourfile.ilg, yourfile.ind, yourfile.log, yourfile.bbl, yourfile.bcf, yourfile.ist, yourfile.blg, yourfile.run.xml]}
%
\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{filecontents}
\usepackage{makeidx}
\begin{filecontents}{\jobname.ist}
 heading_prefix "{\\bfseries "
 heading_suffix "\\hfil}\\nopagebreak\n"
 headings_flag  1
 delim_0 "\\dotfill "
 delim_1 "\\dotfill "
 delim_2 "\\dotfill "
 delim_r "\\textendash"
 suffix_2p "\\nohyperpage{\\,f.}"
 suffix_3p "\\nohyperpage{\\,ff.}"
\end{filecontents}
\usepackage[backend=biber]{biblatex}
\addbibresource{biblatex-examples.bib}
\makeindex
\begin{document}
A citation\cite{companion} and an index entry\index{bla} and some arbitrary text.
\printbibliography
\printindex
\end{document}

To comment out an arara directive, just put ! like

% !arara: makeindex

For more details, consult the excellent arara guide.

Answer 3

I did a large amount of research for a very similar StackOverflow question. To summarize the problems of each tool:

• latexmk will overwrite your document in place, frequently causing your viewer to display an incomplete document (eg when there are errors). Its error support is bad.

• rubber will overwrite your document in place (see above). It also frequently doesn't update your document enough.

• rubber-info prints errors spectacularly, and is better than all other tools at this task. It can be used seperately from rubber, with any other tool.

• vim-latexsuite has an awful build system. Avoid.

• I couldn't make ltx work.

• MikTeX is windows only.

• latex-makefile doesn't allow you use pdflatex, only latex -> ps -> pdf. It also misses bibtex sometimes.

My final solution constantly rebuilds the document via pdflatex, only overwriting when the document has changed (for your viewing pleasure). It relies on rubber-info for errors. I set vim to save the file after every X keystrokes, so that my script constantly rebuilds.

Answer 4

There is a tool named Rubber. It is for instance used by Gedit LaTeX plugin. It should recognize which tools to run (BibTeX, makeindex...) and run (pdf)latex and them in right order and how many times it is needed to converge to the ready file. It behaves like make, i.e. it does not recreate files that it thinks won't change recompiled (this can be omitted by adding -f option).

Compiling to dvi:

rubber filename.tex

Compiling to pdf:

rubber --pdf filename.tex

Cleaning compilation debris:

rubber --clean filename.tex

Answer 5

Scons seems to be the ultimate LaTeX building tool.

Let me elaborate on Karsten W.'s answer on the configuration file.

A typical build script for building LaTeX can be the following. Put this in a file called SConstruct:

# make sure scons finds tex executables:
import os
env = Environment(ENV=os.environ)
# target and source:
pdf_output = env.PDF(target='main.pdf', source='main.tex')
# Add synctex
env.AppendUnique(PDFLATEXFLAGS='-synctex=1')
# make sure that the pdf is reloaded properly (e.g., in Skim)
env.Precious(pdf_output)

You can build the pdf simply by running

scons

Notice that, quite amazingly, scons will detect the changes in the files \included in the main.tex file and also the bibliography file!

Answer 6

For emacs users, AUCTeX provides the command TeX-command-master (bound to C-c C-c by default) that doesn't quite answer the request, but nevertheless helps a lot. The command runs (pdf)latex, bibtex, and makeindex on a .tex file as required. You do have to invoke it multiple times, but it does a pretty good job of sorting out what commands need to be run in what order. (I have had the occasional document that outsmarted it, mostly ones using the exam.cls document class.) Note that TeX files must be parsed for this to work.

Starting from version 11.89, there is also the possibility to automatically run all commands needed to completely build the document at once: TeX-command-run-all, bound to C-c C-a. It's like running C-c C-c multiple times, it will compile the document until it's ready and automatically start the viewer if there are no errors. This feature can be considered a real automatic compilation tool.

Answer 7

Generation of references is handled automatically by the ConTeXt toolchain, so you never have to worry about remembering how many runs you need. The example Makefile would look like this if you used ConTeXt:

paper.pdf: paper.tex
    context paper

Answer 8

I find GNU Make incredibly convenient for parsing. In vim I've mapped F2 to make, so when I've edited something I can just hit F2 and switch to evince to see it instantly.

Below is my Makefile, which also handles parsing graphviz files.

GRAPHS=overview_dot.pdf hello_dot.pdf
TARGET=something.pdf
TEXFILES=file1.tex file2.tex file3.tex
HELPFILES=

all: $(TARGET)

$(TARGET): out.pdf
    mv out.pdf $(TARGET)

out.pdf: $(GRAPHS) $(TEXFILES) $(HELPFILES) Makefile

clean:
    rm -rf *.aux *.log *.out *.toc *.eps *.data *~ $(GRAPHS) $(TARGET)

%.svg: %.dot
    dot $*.dot -Tsvg -o $*.svg

%.png: %.dot
    dot $*.dot -Tpng -o $*.png

%.ps: %.dot
    dot $*.dot -Tps -o $*.ps

%_dot.pdf: %.dot
    dot $*.dot -Tpdf -o $*_dot.pdf

%_neato.pdf: %.dot
    neato $*.dot -Tpdf -o $*_neato.pdf

%_circo.pdf: %.dot
    circo $*.dot -Tpdf -o $*_circo.pdf

%_dia.eps: %.dia
    dia $*.dia -e $*_dia.eps

%_eps.pdf: %.eps
    epstopdf $*.eps -o $*_eps.pdf

%.pdf: %.tex
    pdflatex $*.tex
    pdflatex $*.tex
    pdflatex $*.tex

Answer 9

In addition to all of the good answers above, users on Windows using MiKTeX can use the texify program, which has a number of options to adjust output mode (DVI or PDF), options for BibTeX and MakeIndex and so on.

Answer 10

I like LatexMake. It defines a big pile of pattern rules for make to automate the most common steps of compiling a LaTeX document. Using it is a simple matter of copying one makefile into your document's directory, or include LaTeX.mk (as named in Debian's latex-make package) in a makefile you already have, or symlink to it if your makefile wouldn't need to include anything else.

One really neat feature about it is that it detects dependencies on included files, and rebuilds as much of your document as needed to reflect those changes.

Another neat feature is that it knows about things like bibtex, and runs the appropriate commands.

Answer 11

With all solutions I tried back in the day (definitely not all), I had at least one of the following issues.

1. You have to manually specify which commands should be executed how often and in which order.
2. Temporary files crowd the document folder.
3. No automatic rebuild on file changes.
4. No support for external tool X and hard to integrate.
5. No help at all with myriads of log files.

So I set out to build my own tool which I named ltx2any¹ around the following core features.

1. Figure out exactly what tools have to be run at runtime and execute them in the right order.
2. Determine dynamically how often pdflatex (or other engine) has to be run.
3. Copy the document tree and compile somewhere else, keeping temporary files tucked away.
4. As an option, rebuild automatically when files change.
5. Offer an easy interface for adding new engines and external tools.
6. Aggregate shell outputs resp. log files of all executed commands.

Basically, most documents should compile just right by typing ltx2any file.tex, provided the necessary extensions exist.

Since I started, I added:

• Parallel execution of external tools if possible (e.g. TikZ externalisation).
• Logs are parsed for error messages, warnings and information, counted and presented in an easy to digest format.²
• Rudimentary interactive shell in auto-rebuild mode, including setting options and requesting rebuilds of specific TikZ images.

It's definitely not perfect and has some rough edges (in particular w.r.t. the use of evolving libraries) but it works well enough for me.

I should probably add that you'll need Ruby >= 2.0.x.

Pull requests for extensions are appreciated!

---

1. It totally does not deserve the name anymore. I'll have to come up with a new one for the first "stable" release.
2. Due to the awful situation that is arbitrary log formatting, this probably has holes like Swiss cheese. I appreciate any reports of not or mis-parsed messages.

Answer 12

It does more than just compile documents, being a fully featured editor, but Kile is very good at completing all the compilation steps.

You can also define your own build processes which use command line tools. By adding a ViewPDF option at the end of the compile your pdf viewer of choice will be launched (or updated) after a successful compilation.

Answer 13

I use scons, a python based make tool. Some documents I have contain R code, scons (with rpy2) works well in handling them. I had to program the rules for document creation by myself, but it was basically calling texify.

Answer 14

There is nothing wrong with Makefiles, the magic is to understand the dependency chain that exist so make don't rerun everything all the time and when he reruns he does this in the right order.

How about a simple example with table of context, that is stored in a .toc file. So just add .toc as a dependency in the Makefile, that way it will run twice the first time but only once the following times.

NAME=MyTestFile

pdf: $(NAME).toc
    pdflatex $(NAME).tex

$(NAME).toc: 
    pdflatex -draftmode $(NAME).tex 

And when it is time to to a final version make sure you do a "clean" so everything builds and links together in a correct way.

---

Update: After reading you question (and comment), I will change my answer to. There is nothing wrong with makefiles but it is not the most optimal solution for all people in all circumstances.

If you don't plan to add some command line magic in your makefile and never plan to script anything at all maybe the makefile method is unnecessary complicated.

But if you plan to use LaTeX in a little bit unorthodox way and with some command line magic, a Makefile will help you a lot.

Answer 15

For completeness, I will mention autolatex (project page). I have not used it. It's implemented with a combination of perl scripts and makefiles.

With autolatex you can execute on the command line:

$ autolatex -f mydoc.tex

This generates the makefiles in the current directory and makes the default targets.

Answer 16

Only for Windows users. Create a batch file as follows. Let's name it batch.bat.

echo off

rem %1 TeX input filename without extension

rem %2 The number of times to invoke pdflatex in draftmode

del "%~1.pdf"

for /l %%x in (1,1,%2) do pdflatex --shell-escape -draftmode -interaction=batchmode "%~1.tex"

pdflatex --shell-escape "%~1.tex"

for %%x in (aux log out toc nav snm) do (del "%~1.%%x")

It takes 2 mandatory arguments:

• %1 represents the TeX input file name without extension.
• %2 represents the number of times you invoke pdflatex in draftmode. There are %2 plus one pdflatex invocations in total.

Once the PDF file has been successfully generated, the auxiliary files are deleted to save more data storage, etc, etc.

You can invoke the batch as follows:

You can edit and insert other calls to the batch file above to meet your need.

Answer 17

Like @crazymaik, I’ve set up a script that runs continually in the background so the file is automatically compiled whenever I save. The idea for that originally comes from Paul Biggar.

Mine uses latexmk instead of rubber (however, I use rubber-info from inside Vim to parse the log file). This has the advantage of working out of the box with TeXLive.

The script is a bit longer – I think the comments sufficiently explain why this is. I’ve posted the source as a gist on GitHub.

#!/bin/bash

# Author: Konrad Rudolph
# Original idea: Paul Biggar
#
# Usage: texit [--latex] target
#
#  --latex: Use pdflatex instead of xelatex as the processor. Optional
#  target:  The name of the target (i.e. the source file without trailing `.tex`)

…

Answer 18

texi2dvi, from the texinfo package, is a tool similar to MiKTeX's texify (actually, IIRC, MiKTeX has an alias to texify called texi2dvi). It is a front-end to both LaTeX and PDFLaTeX, and through environment variables you can specify extra options to either processor (and even run XeLaTeX).

Answer 19

My objective was to be able to compile easily even if the system haven't latex installed or the package are not up-to-date compared to my own system (I used several computer without all the package or with a 2 years back version).

I have a git repository to centralized all my source (you can use a draft repository easily). A Makefile similar to Charlie.

In my computer crontab:

* * * * * "cd /path/to/repository && script.sh"

and in script.sh:

#!/bin/sh    
git pull > update.tmp
TEST=`cat update.tmp`
COMP=`echo "Already up-to-date."`
if [ "$TEST" != "$COMP" ] #if the repository is up-do-date their is no need to recompile
then
    date > update.tmp #to have a file with the start compilation date (not needed)
    make #I use --interaction=nonstopmode as LaTeX flags to have no deadlock
    git add main.pdf main.log update.tmp #could be usefull to have the result and the log in case of mistake
    git commit -m "automatic push from script.sh"
    git push
fi

Of course you need no password on your git repository or to have configure it with an ssh key (but it's another subject)

If you work in a colaborative project it's a way for everyone to have the same system easily.

Futher more, you can bind your editor to make an automatic push at each save.

Answer 20

I use a shell script to automatically compile the file with rubber every time i save the file and my viewer automatically reloads the pdf.

#!/bin/sh

if [ "$#" != 2 ]; then
    echo "USAGE: onsave <file> <action>"
    exit 1
fi

file=$1
action=$2
last_modified="0"

while [ 1 ]
do
    current=`stat -c "%Y" $file`
    if [ $current -gt $last_modified ]; then
        `$action`
        last_modified=$current
    fi
    sleep 2
done

Answer 21

Those makefile are always a bit difficult, in part because they have to interact with the engine to find out how many compilation runs are necessary, which additional tools need to be run etc. and these steps are usually harder than trivial with makefiles.

That being said, I'd suggest you to try latex-makefile, which is a make-only solution, which goes long ways to accomplish what's needed. It's not complete, however it's better than most makefile solutions out there, tries covering all generation steps including converting images to right formats and most importantly it supports PDF(La)TeX and Xe(La)TeX out of the box now!

Answer 22

I posted some sample makefiles for TeX/Sweave on my site: http://cgibbons.berkeley.edu/Research/writings.html

One makefile does a simple compilation, another compiles a Beamer file into presentation slides, handout slides, and a notes page. The third takes a tex file that contains questions and solutions to an exam or problem set and turns it into a plain version to pass out and a solutions set.

Using some tips from the web (don't remember where), the following code only runs bibtex or reruns pdflatex if a warning is found in the log file produced by the initial pdflatex run:

$(TEXFILE).pdf: $(TEXFILE).tex $(FIGURES) $(INPUTS)
     # Initial run
     pdflatex $(TEXFILE)

     # Run BibTeX if missing citations
         @if(grep "Citation" $(TEXFILE).log > /dev/null);\
          then \
              bibtex $(TEXFILE);\
              pdflatex $(TEXFILE); \
     fi

     # Recompile if instructed
     @if(grep "Rerun" $(TEXFILE).log > /dev/null);\
     then \
         pdflatex $(TEXFILE); \
     fi

Note that $(TEXFILE) is the name of the tex file. Some documentation explaining the code is also available at my site. The documentation also gives code to create an emacs keybinding to save your document and run the makefile.

Answer 23

I used a simple bash / shell script for Mac OS X / teTex to produce PDFs of various documents including my master's thesis back several years ago. It basically scans the output produced by pdflatex to see whether it needs to run the document through additional passes and also executes bibtex and makeindex.

The script was inspired by the texify.exe program bundled with the MiKTeX package for Windows.

I'm not sure how well it works for other UNIX-like systems but at the time of initial writing it ran flawlessly on my Mac and a Linux box I tested.

Personally, this is still the solution I use when working with LaTeX documents as it just works for me, no compilation and no cumbersome installation needed, just one executable file.

However, as I haven't developed it further and didn't have the resources to test it extensively with various systems (and have no plans to develop it further), I cannot guarantee for its proper function on other systems than those that I've had at hand.

I hereby release it to the public in hope that it might still be useful for someone out there. You can find it on my user's profile page here.

Answer 24

What problem are you trying to solve? Make utility (I use BSD Make) is a perfect tool for automatizing TeX-ing at least on Unix and Unix like systems. If the above Makefile works for you I see nothing wrong with it. I have a little bit more elaborate but fairly simple Makefile which works for most of my needs and can be quickly adapted in the case I need to TeX more complicated document. Tools like Latexmk (Perl), LatexMake and few others that you can find on the ctan make an attempt to anticipate all possible situations/environments for all possible TeX users. Therefore they tent to be very complicated and sometimes do not work well for some people. This is one case in which I would stick to my home cooked Makefile. The advantage of using Perl over Make utility is that Perl is little bit easier to use on Windows (although still feels very awkward) over Make which I think require full blown Cygwin. Maybe somebody can suggest some Python script. Python unlike Perl feels fairly naturally on Windows.

Answer 25

Here is an easy was to do the automation: ( I am using TeXWorks in Ubuntu 14.04 Linux Enviroment )

1- add a new typeset tool, (Edit > Preferences > Typesetting > Add Processing tools)

2- And use Browse to add your simple bash code:

#!/bin/bash                                                                 

echo "Compiling $1 ..."

file_basename=$2                                                                                                                              

pdflatex $file_basename                                                          
bibtex $file_basename                                                       
pdflatex $file_basename                                                          
pdflatex $file_basename 

Answer 26

The module UseLATEX for CMake will automatically generate a makefile for you, handling all of the dependencies itself. It works really well for projects that combine C/C++ and LaTeX.

Answer 27

latexrun is my preferred way. It is a simple download of a single file and easy to use.

Unless there are no warnings or errors, the output is very limited - and in case there are, it presents warnings and errors clearly.

It also by default collects intermediate files in a subdirectory, so much cleaner than other methods, and is easy to integrate with for example GNU Make.

./latexrun document.tex