How to properly 'make' a latex project?

Question

I've got a latex project that looks like this:

project/
   |-- main.tex
   |-- main.bib
   |-- preamble.tex
   |-- preamble.fmt
   |-- makefile

The preamble (preamble.tex) is being precompiled into preamble.fmt. main.bib is generated using the file contents environment in main.tex.

This is what is in my makefile

TEX = pdflatex -shell-escape -interaction=nonstopmode -file-line-error
PRE =  $(TEX) -ini -job-name="preamble" "&pdflatex preamble.tex\dump"
BIB = bibtex

.PHONY: all view

all : main.pdf

view :
    open main.pdf

main.pdf : main.tex preamble.fmt main.bbl main.blg
    $(TEX) main.tex

main.bbl main.blg : main.bib main.aux
    $(BIB) main

main.aux : main.tex
    $(TEX) main.tex

main.bib : main.tex
    $(TEX) main.tex

preamble.fmt : preamble.tex
    $(PRE) preamble.tex

The problem is here that bibtex relies on main.aux to be generated, and main.aux is regenerated every pdftex run. This leads to regeneration of the bibtex files every run, which causes makefile to run $(TEX) main.tex a second time (it thinks the .bbl and .blg files are changed, because it looks at their edit time).

So, basically every time I call make all latex is compiled twice, even if there are no changes to references made anywhere in the document (thus making this unnecessary).

Is there a way I can tell make that it only compiles twice if there is an actual change to the .aux and the .bib files. Perhaps by checking of md5 sums?

I'm kind of new to the whole makefile thing, so I thought I'd ask here. It could be off-topic, but I thought the latex gurus that reside on this site might have an answer.

Answer 1

latexmk is the answer you are looking for.

LaTeX is notoriously difficult to "get right" using a Makefile, because it might take multiple compiler passes - updating e.g. .aux files - to get the finished results. Getting this right in a general Makefile (as opposed to one tailored to a specific document) is very hard, which is why there are pre-made solutions. Of these, latexmk comes included with your average LaTeX distribution, which is why I consider it first choice.

The trick is to provide a Makefile rule for every custom step x-to-TeX (or x-to-PDF or whatever) you might have, and having latexmk figure out all the LaTeX-related stuff while relying on the Makefile for the rest (via -use-make).

# You want latexmk to *always* run, because make does not have all the info.
# Also, include non-file targets in .PHONY so they are run regardless of any
# file of the given name existing.
.PHONY: MyDoc.pdf all clean

# The first rule in a Makefile is the one executed by default ("make"). It
# should always be the "all" rule, so that "make" and "make all" are identical.
all: MyDoc.pdf

# CUSTOM BUILD RULES

# In case you didn't know, '$@' is a variable holding the name of the target,
# and '$<' is a variable holding the (first) dependency of a rule.
# "raw2tex" and "dat2tex" are just placeholders for whatever custom steps
# you might have.

%.tex: %.raw
        ./raw2tex $< > $@

%.tex: %.dat
        ./dat2tex $< > $@

# MAIN LATEXMK RULE

# -pdf tells latexmk to generate PDF directly (instead of DVI).
# -pdflatex="" tells latexmk to call a specific backend with specific options.
# -use-make tells latexmk to call make for generating missing files.

# -interaction=nonstopmode keeps the pdflatex backend from stopping at a
# missing file reference and interactively asking you for an alternative.

MyDoc.pdf: MyDoc.tex
        latexmk -pdf -pdflatex="pdflatex -interaction=nonstopmode" -use-make MyDoc.tex

clean:
        latexmk -CA

This setup works flawlessly for anything referenced via \include.

However, \include might not be appropriate in every case. For one, it is not nestable (i.e. an \included file may not \include another). It also adds an automatic \clearpage to your document, i.e. \included content starts a new page. It also has advantages, like resulting in shorter re-build times if contents are modified, but sometimes you need nesting, or the referenced file's contents should be embedded in a page.

You need \input for this.

Sadly, \input breaks the build. If pdflatex encounters a missing \input file, it generates an error (instead of a warning like with \include), and stops compiling. Yes, latexmk will generate the file and re-start pdflatex, but this is inefficient, and breaks completely if you have multiple such file references, because eventually the compile will end with a "too many re-runs" message.

John Collins' answer to a question by me regarding this problem provides a workaround:

\newcommand\inputfile[1]{%
    \InputIfFileExists{#1}{}{\typeout{No file #1.}}%
}

This macro generates a warning instead of the error of a straight \input, and allows latexmk to generate all missing files in the first pass.

Note: A rule with the generic target %.pdf: %.tex gives you trouble once you start using \includeonly in your document, for reasons internal and complex. That's why I used a specific rule instead of a generic one.

---

There is actually one alternative to latexmk that I can also recommend. In case you are looking at a more involved project setup, you might consider CMake, for which Kenneth Moreland has done the excellent UseLATEX.cmake module.

This, however, is a bit too involved to give a how-to in the scope of this answer.

Answer 2

Publicly Available Makefiles for TeX Projects

Here're some good examples of makefiles for TeX projects. Feel free to add more.

By Matthias Vallentin

License: CC BY-NC-SA 4.0

Link: https://github.com/mavam/stat-cookbook/blob/master/Makefile

Includes input directives, standalone figures, bibliography, latexmk, and reruns. Part of the stat-cookbook repository, which gives some context how things work together.

By Yannick Copin

License: Not mentioned

Link: https://web.archive.org/web/20061205102406/http://snovae.in2p3.fr/ycopin/soft/Makefile.latex

Batch processing of .tex files, including bibliography. Supports reruns without latexmk.

By Chris Rost

License: Not mentioned

Link: https://stuff.mit.edu/people/jcrost/latexmake.html

By Kyle Woerner et al.

License: Not mentioned

Link (page 5): https://oceanai.mit.edu/k_w/latex_lab/lab_latex_8.pdf

By Petter Kallstrom

License: CC BY-SA 3.0

Link: https://en.wikibooks.org/wiki/Make/Examples

Answer 3

There is an answer to a similar problem. Features:

• Shows how to use checksum files with controlled timestamps where necessary.
  • There, it's not .aux to .bbl but index files .nlo to .nls, but you get the idea.
• Alternatively recommends latexmk.
  • Shows how to use latexmk with automatic dependency tracking in a GNU Makefile.
  • Example configuration latexmkrc for custom makeindex files as used by the nomencl package.

Have fun.

Answer 4

One trick I use when calling (pdf)latex from a regular Makefile is to check in a shell while loop with grep the %.log file for the presence of the string Rerun to get, to ensure the required number of reruns of LaTeX to resolve all references.

My Makefile typically starts out like this:

# requires GNU make
SHELL=/bin/bash

.DELETE_ON_ERROR:

%.pdf %.aux %.idx: %.tex
    pdflatex $<
    while grep 'Rerun to get ' $*.log ; do pdflatex $< ; done
%.ind: %.idx
    makeindex $*
%.bbl: %.aux
    bibtex $*
%.pdftex %.pdftex_t: %.fig
    fig2dev -L pdftex_t -p $*.pdftex $< $*.pdftex_t
    fig2dev -L pdftex $< $*.pdftex

Where things get tricky are rules involving %.aux as a dependency, which can lead to circular dependencies.