How to properly 'make' a latex project?

I've got a latex project that looks like this:

``````project/
|-- main.tex
|-- main.bib
|-- preamble.tex
|-- preamble.fmt
|-- makefile
``````

The preamble (`preamble.tex`) is being precompiled into `preamble.fmt`. `main.bib` is generated using the `file contents` environment in `main.tex`.

This is what is in my `makefile`

``````TEX = pdflatex -shell-escape -interaction=nonstopmode -file-line-error
PRE =  \$(TEX) -ini -job-name="preamble" "&pdflatex preamble.tex\dump"
BIB = bibtex

.PHONY: all view

all : main.pdf

view :
open main.pdf

main.pdf : main.tex preamble.fmt main.bbl main.blg
\$(TEX) main.tex

main.bbl main.blg : main.bib main.aux
\$(BIB) main

main.aux : main.tex
\$(TEX) main.tex

main.bib : main.tex
\$(TEX) main.tex

preamble.fmt : preamble.tex
\$(PRE) preamble.tex
``````

The problem is here that `bibtex` relies on `main.aux` to be generated, and `main.aux` is regenerated every `pdftex` run. This leads to regeneration of the bibtex files every run, which causes makefile to run `\$(TEX) main.tex` a second time (it thinks the `.bbl` and `.blg` files are changed, because it looks at their edit time).

So, basically every time I call `make all` latex is compiled twice, even if there are no changes to references made anywhere in the document (thus making this unnecessary).

Is there a way I can tell `make` that it only compiles twice if there is an actual change to the `.aux` and the `.bib` files. Perhaps by checking of md5 sums?

I'm kind of new to the whole makefile thing, so I thought I'd ask here. It could be off-topic, but I thought the latex gurus that reside on this site might have an answer.

-
Have you considered `rubber`, `latexmk` or similar tools? Also, why are you building a format? –  Joseph Wright Jan 11 '12 at 11:13
I only saw after posting the question that there is the `{makefile}` tag, which I'm now scanning for useful code. I'll leave this question up for now. –  romeovs Jan 11 '12 at 11:14
@JosephWright The format is nice because I only need to compile the preamble once, which save me a lot of compiling time. See for instance this answer I once posted. I'm looking into these alternatives now, but I'm integrating some other stuff too (like `c++` code that generates data which I input into the tex file) and I don't know if that can be easily done using `latexmk` or `rubber`. –  romeovs Jan 11 '12 at 11:17
Don't add `main.aux` to the dependencies of `\$(BIB)` (I don't know if its the right terminology). –  egreg Jan 11 '12 at 13:26
@egreg this won;t do, because `bibtex` does need the `main.aux` file to be present. Either way, I still would need to compile `main.tex` before `bibtex` is run to see if `main.bib` (which is generated in the `main.tex` run) has changed. The problem is in how `make` checks if a file has changed. Perhaps this question should be migrated to the Stack Overflow site? –  romeovs Jan 11 '12 at 14:24

`latexmk` is the answer you are looking for.

LaTeX is notoriously difficult to "get right" using a Makefile, because it might take multiple compiler passes - updating e.g. `.aux` files - to get the finished results. Getting this right in a general Makefile (as opposed to one tailored to a specific document) is very hard, which is why there are pre-made solutions. Of these, `latexmk` comes included with your average LaTeX distribution, which is why I consider it first choice.

The trick is to provide a Makefile rule for every custom step x-to-TeX (or x-to-PDF or whatever) you might have, and having `latexmk` figure out all the LaTeX-related stuff.

``````# You want latexmk to *always* run, because make does not have all the info.
# Also, include non-file targets in .PHONY so they are run regardless of any
# file of the given name existing.
.PHONY: MyDoc.pdf all clean

# The first rule in a Makefile is the one executed by default ("make"). It
# should always be the "all" rule, so that "make" and "make all" are identical.
all: MyDoc.pdf

# CUSTOM BUILD RULES

# In case you didn't know, '\$@' is a variable holding the name of the target,
# and '\$<' is a variable holding the (first) dependency of a rule.
# "raw2tex" and "dat2tex" are just placeholders for whatever custom steps
# you might have.

%.tex: %.raw
./raw2tex \$< > \$@

%.tex: %.dat
./dat2tex \$< > \$@

# MAIN LATEXMK RULE

# -pdf tells latexmk to generate PDF directly (instead of DVI).
# -pdflatex="" tells latexmk to call a specific backend with specific options.
# -use-make tells latexmk to call make for generating missing files.

# -interactive=nonstopmode keeps the pdflatex backend from stopping at a
# missing file reference and interactively asking you for an alternative.

MyDoc.pdf: MyDoc.tex
latexmk -pdf -pdflatex="pdflatex -interactive=nonstopmode" -use-make MyDoc.tex

clean:
latexmk -CA
``````

This setup works flawlessly for anything referenced via `\include`.

However, `\include` might not be appropriate in every case. For one, it is not nestable (i.e. an `\include`d file may not `\include` another). It also adds an automatic `\clearpage` to your document, i.e. `\include`d content starts a new page. It also has advantages, like resulting in shorter re-build times if contents are modified, but sometimes you need nesting, or the referenced file's contents should be embedded in a page.

You need `\input` for this.

Sadly, `\input` breaks the build. If `pdflatex` encounters a missing `\input` file, it generates an error (instead of a warning like with `\include`), and stops compiling. Yes, `latexmk` will generate the file and re-start `pdflatex`, but this is inefficient, and breaks completely if you have multiple such file references, because eventually the compile will end with a "too many re-runs" message.

John Collins' answer to my question regarding this provides a workaround for this.

``````\newcommand\inputfile[1]{%
\InputIfFileExists{#1}{}{\typeout{No file #1.}}%
}
``````

This macro generates a warning instead of the error of a straight `\input`, and allows `latexmk` to generate all missing files in the first pass.

Note: A rule with the generic target `%.pdf: %.tex` gives you trouble once you start using `\includeonly` in your document, for reasons internal and complex. That's why I used a specific rule instead of a generic one.

There is actually one alternative to `latexmk` that I can also recommend. In case you are looking at a more involved project setup, you might consider CMake, for which Kenneth Moreland has done the excellent UseLATEX.cmake module.

This, however, is a bit too involved to give a how-to in the scope of this answer.

-
haha, this actually works! sweet. I'm going to mess around with it a bit and probably accept! +1 –  romeovs Jan 11 '12 at 15:09
since `latexmk` apparently doesn't know about the pre formatted preamble. I needed to do `main.pdf: preamble.fmt`. Otherwise all is fine! –  romeovs Jan 11 '12 at 15:19
1. Actually, if you turn the recorder option on in `latexmk`, it does know about the format file. 2. If you are using a makefile, it's a good idea to have all the source files listed in the makefile. `Latexmk` can set this up automatically; see the section "USING `latexmk` WITH `make`" in the `latexmk` documentation. –  John Collins Jan 11 '12 at 20:59
@romeovs: Please note the complete overhaul of the answer. Thanks to John Collins, it now includes a fully functional example. –  DevSolar Jan 13 '12 at 8:17
@JohnCollins: I hope the link gets you lots of upvotes. Thank you for that macro solution! –  DevSolar Jan 13 '12 at 8:18