How to properly 'make' a latex project?

Question

I've got a latex project that looks like this:

project/
   |-- main.tex
   |-- main.bib
   |-- preamble.tex
   |-- preamble.fmt
   |-- makefile

The preamble (preamble.tex) is being precompiled into preamble.fmt. main.bib is generated using the file contents environment in main.tex.

This is what is in my makefile

TEX = pdflatex -shell-escape -interaction=nonstopmode -file-line-error
PRE =  $(TEX) -ini -job-name="preamble" "&pdflatex preamble.tex\dump"
BIB = bibtex

.PHONY: all view

all : main.pdf

view :
    open main.pdf

main.pdf : main.tex preamble.fmt main.bbl main.blg
    $(TEX) main.tex

main.bbl main.blg : main.bib main.aux
    $(BIB) main

main.aux : main.tex
    $(TEX) main.tex

main.bib : main.tex
    $(TEX) main.tex

preamble.fmt : preamble.tex
    $(PRE) preamble.tex

The problem is here that bibtex relies on main.aux to be generated, and main.aux is regenerated every pdftex run. This leads to regeneration of the bibtex files every run, which causes makefile to run $(TEX) main.tex a second time (it thinks the .bbl and .blg files are changed, because it looks at their edit time).

So, basically every time I call make all latex is compiled twice, even if there are no changes to references made anywhere in the document (thus making this unnecessary).

Is there a way I can tell make that it only compiles twice if there is an actual change to the .aux and the .bib files. Perhaps by checking of md5 sums?

I'm kind of new to the whole makefile thing, so I thought I'd ask here. It could be off-topic, but I thought the latex gurus that reside on this site might have an answer.

Answer 1

latexmk is the answer you are looking for.

LaTeX is notoriously difficult to "get right" using a Makefile, because it might take multiple compiler passes - updating e.g. .aux files - to get the finished results. Getting this right in a general Makefile (as opposed to one tailored to a specific document) is very hard, which is why there are pre-made solutions. Of these, latexmk comes included with your average LaTeX distribution, which is why I consider it first choice.

The trick is to provide a Makefile rule for every custom step x-to-TeX (or x-to-PDF or whatever) you might have, and having latexmk figure out all the LaTeX-related stuff.

# You want latexmk to *always* run, because make does not have all the info.
.PHONY: MyDoc.pdf

# First rule should always be the default "all" rule, so both "make all" and
# "make" will invoke it.
all: MyDoc.pdf

# CUSTOM BUILD RULES

# In case you didn't know, '$@' is a variable holding the name of the target,
# and '$<' is a variable holding the (first) dependency of a rule.

%.tex: %.raw
        ./raw2tex $< > $@

%.tex: %.dat
        ./dat2tex $< > $@

# MAIN LATEXMK RULE

# -pdf tells latexmk to generate PDF directly (instead of DVI).
# -pdflatex="" tells latexmk to call a specific backend with specific options.
# -use-make tells latexmk to call make for generating missing files.

# -interactive=nonstopmode keeps the pdflatex backend from stopping at a
# missing file reference and interactively asking you for an alternative.

MyDoc.pdf: MyDoc.tex
        latexmk -pdf -pdflatex="pdflatex -interactive=nonstopmode" -use-make MyDoc.tex

clean:
        latexmk -CA

This setup works flawlessly for anything referenced via \include.

However, \include might not be appropriate in every case. For one, it is not nestable (i.e. an \included file may not \include another). It also adds an automatic \clearpage to your document, i.e. \included content starts a new page. It also has advantages, like resulting in shorter re-build times if contents are modified, but sometimes you need nesting, or the referenced file's contents should be embedded in a page.

You need \input for this.

Sadly, \input breaks the build. If pdflatex encounters a missing \input file, it generates an error (instead of a warning like with \include), and stops compiling. Yes, latexmk will generate the file and re-start pdflatex, but this is inefficient, and breaks completely if you have multiple such file references, because eventually the compile will end with a "too many re-runs" message.

John Collins' answer to my question regarding this provides a workaround for this.

\newcommand\inputfile[1]{%
    \InputIfFileExists{#1}{}{\typeout{No file #1.}}%
}

This macro generates a warning instead of the error of a straight \input, and allows latexmk to generate all missing files in the first pass.

Note: As I discovered yesterday, a rule with the generic target %.pdf: %.tex gives you trouble once you start using \includeonly in your document, for reasons internal and complex. That's why I replaced the generic rule with a specific one.