1

I have a project directory ~/Documents/test w/ the contents...

test
|- refs
|- paper
|-|- main.tex

The contents of main is...

\documentclass{article}
\usepackage[superscript,nomove]{cite}

\begin{document}

Hello world\cite{texbook}


\clearpage
\bibliographystyle{unsrt}
\bibliography{../refs}

\end{document}

the refs.bib file is...

@book{texbook,
  author = {Donald E. Knuth},
  year = {1986},
  title = {The {\TeX} Book},
  publisher = {Addison-Wesley Professional}
}

My .latexmkrc is...

$pdf_mode = 1;
$pdflatex = 'pdflatex --shell-escape -interaction=nonstopmode'; 
$emulate_aux = 1;
$out_dir = '.';
$aux_dir = '/tmp/latexmk';

When I try to compile, the citation appears as a question mark while the reference itself does not appear. The output is...

$ latexmk main.tex
Rc files read:
  /home/.latexmkrc
Latexmk: This is Latexmk, John Collins, 4 Apr. 2023. Version 4.80.
No existing .aux file, so I'll make a simple one, and require run of *latex.
Latexmk: applying rule 'pdflatex'...
Rule 'pdflatex':  Reasons for rerun
Category 'other':
  Rerun of 'pdflatex' forced or previously required

------------
Run number 1 of rule 'pdflatex'
------------
------------
Running 'pdflatex --shell-escape -interaction=nonstopmode  -recorder -output-directory="/tmp/latexmk"  "main.tex"'
------------
This is pdfTeX, Version 3.141592653-2.6-1.40.25 (TeX Live 2023) (preloaded format=pdflatex)
 \write18 enabled.
entering extended mode
(./main.tex
LaTeX2e <2023-06-01> patch level 1
L3 programming layer <2023-08-11>
(/usr/local/texlive/2023/texmf-dist/tex/latex/base/article.cls
Document Class: article 2023/05/17 v1.4n Standard LaTeX document class
(/usr/local/texlive/2023/texmf-dist/tex/latex/base/size10.clo))
(/usr/local/texlive/2023/texmf-dist/tex/latex/cite/cite.sty)
(/usr/local/texlive/2023/texmf-dist/tex/latex/l3backend/l3backend-pdftex.def)
(/tmp/latexmk/main.aux)

LaTeX Warning: Citation `texbook' on page 1 undefined on input line 6.

[1{/usr/local/texlive/2023/texmf-var/fonts/map/pdftex/updmap/pdftex.map}]
No file main.bbl.
(/tmp/latexmk/main.aux)

LaTeX Warning: There were undefined references.

 )</usr/local/texlive/2023/texmf-dist/fonts/type1/public/amsfonts/cm/cmbx7.pfb>
</usr/local/texlive/2023/texmf-dist/fonts/type1/public/amsfonts/cm/cmr10.pfb>
Output written on /tmp/latexmk/main.pdf (1 page, 20534 bytes).
Transcript written on /tmp/latexmk/main.log.
Latexmk: Moving '/tmp/latexmk/main.pdf' to 'main.pdf'
Latexmk: Getting log file '/tmp/latexmk/main.log'
Latexmk: Examining '/tmp/latexmk/main.fls'
Latexmk: Examining '/tmp/latexmk/main.log'
Latexmk: Missing bbl file '/tmp/latexmk/main.bbl' in following:
 No file main.bbl.
Latexmk: Log file says output to '/tmp/latexmk/main.pdf'
Latexmk: Found bibliography file(s):
  ../refs.bib
Latexmk: applying rule 'bibtex /tmp/latexmk/main'...
Rule 'bibtex /tmp/latexmk/main':  Reasons for rerun
Category 'other':
  Rerun of 'bibtex /tmp/latexmk/main' forced or previously required

------------
Run number 1 of rule 'bibtex /tmp/latexmk/main'
------------
Latexmk: Change directory to '/tmp/latexmk/'.
To assist finding of files in document directory, I set
  BIBINPUTS='/home/Documents/test/paper:/tmp/latexmk:'
  BSTINPUTS='/home/Documents/test/paper:'.
------------
Running 'bibtex  "main.aux"'
------------
This is BibTeX, Version 0.99d (TeX Live 2023)
The top-level auxiliary file: main.aux
The style file: unsrt.bst
I couldn't open database file ../refs.bib
---line 4 of file main.aux
 : \bibdata{../refs
 :                 }
I'm skipping whatever remains of this command
I found no database files---while reading file main.aux
Warning--I didn't find a database entry for "texbook"
(There were 2 error messages)
Latexmk: Change directory back to '/home/Documents/test/paper'
Latexmk: Summary of warnings from last run of *latex:
  Latex failed to resolve 1 citation(s)
Latexmk: ====List of undefined refs and citations:
  Citation `texbook' on page 1 undefined on input line 6
Latexmk: Errors, so I did not complete making targets
Collected error summary (may duplicate other messages):
  bibtex /tmp/latexmk/maint: Bibtex errors: See file '/tmp/latexmk/main.blg'

Latexmk: If appropriate, the -f option can be used to get latexmk
  to try to force complete processing.

This issue does not occur if I have the refs.bib file in the same directory as main.tex. How can I fix this?

2
  • fixed. thank you.
    – user41177
    Feb 21 at 4:57
  • Fixed that as well
    – user41177
    Feb 21 at 5:28

2 Answers 2

3

The culprit is the combination of -output-directory (in your case, setting $aux_dir) with a relative path for your bib file.

The following is an excerpt from the latexmk manual:

Choice of aux and output directories: Often the aux and output directories are given as subdirectories of the document directory, e.g., by -outdir=output. But it is possible to provide, for example, an absolute path or a path relative to a parent directory, e.g., "/tmp/foo" or "../output". Be aware that in general this can cause problems, notably with makeindex or bibtex. This is because modern versions of these programs, by default, will refuse to work when they find that they are asked to write to a file in a directory that appears not to be the current working directory or one of its subdirectories. This is part of security measures by the whole TeX system that try to prevent malicious or errant TeX documents from incorrectly messing with a user’s files.

By default, latexmk evades this issue: Before running bibtex and makeindex, latexmk changes working directory to the aux directory, with appropriate settings of search paths. The use or non-use of this trick is governed by the variables $bibtex_fudge and $makeindex_fudge. Unfortunately, the trick sometimes makes bibtex and makeindex unable to find files.

If necessary the trick can be turned off. But this is incompatible with an aux directory like, "/tmp/foo" of "../output"). If you really have to deal with this situation, and only if you have to deal with it, then you need to disable the security measures (and assume any risks). One way of doing this is to temporarily set an operating system environment variable openout_any to "a" (as in "all"), to override the default "paranoid" setting.

After your first run of latexmk, all the aux files are, per your configuration, placed in /tmp/latexmk. Next, latexmk attempts to run bibtex. To do so, it changes directory to /tmp/latexmk and sets BIBINPUTS to include your original project directory (where main.tex is found). However, it then encounters the specification that your file is ../refs.bib. This entirely negates the setting of BIBINPUTS since with a relative path specification, bibtex only searches the specified path, meaning /tmp/latexmk/../refs.bib, which of course does not exist.

This works with \bibliography{refs} because in this form, bibtex would search the directories listed in BIBINPUTS (in order) and take the first hit as the correct file to use.

Solutions

(Any one of the following would work: listed in order of my personal preference.)

  1. Don't use -output-directory. It often causes more trouble than it is worth.
  2. Put your refs.bib file in the same directory as main.tex as you discovered.
  3. Use \bibliography{refs.bib} and add ~/Documents/test/ to your BIBINPUTS (you can do so by adding a project-specific latexmkrc that sets @BIBINPUTS, if you don't want to pollute the search path globally)
  4. Use absolute paths instead of relative paths: so use \bibliography{/home/<your username>/Documents/test/refs}.

I would not suggest following the steps outlined in the quoted text above that opens a security hole on your machine.

3
  • As a general aside: I would also not suggest using /tmp/latexmk as the global $aux_dir. What if you have two different projects in test1/main.tex and test2/main.tex and you happen to be editing them both at the same time? This will create a collision of aux files all named main.*. Feb 21 at 5:35
  • Thank you so much. I really appreciate your help. With (1), I had set aux_dir to hide the auxiliary files related to the compilation (also I had thought this would create a new folder under /tmp/ for each new file to prevent such collisions, which I guess I was wrong about) while keeping the accelerated compilation of my document. Is there a better way of doing this, while allowing me to set relative paths in \bibliography{}?
    – user41177
    Feb 21 at 5:45
  • Is it an option to put the auxilliary files somewhere that is not in /tmp? A kludge is to make a project specific latexmkrc that puts the aux files in ~/Document/test/auxfiles, this way when bibtex is run inside the auxfiles directory, you will be able to still find refs.bib, now as ~/Document/test/auxfiles/../refs.bib // I didn't suggest this because this depends on the coincidence of ../refs working from two different paths, and is not terribly robust. But if you keep that it mind it should work. Feb 21 at 6:35
1

Here's an addition to the answer by @Willie Wong

You can also solve the problem by giving the -bibtexfudge- option to latexmk, or equivalently by putting $bibtex_fudge=0; in a latexmkrc file.

This works with latexmk in versions greater than 4.71 and at least with the bibtex that is in TeXLive 2019 or later. Unfortunately, the latexmk documentation on this point is out-of-date. There were some improvements in the handling of the output (and aux) directories in v. 4.71, to handle exactly the subject of this question, but I forgot the need to update the documentation.

In general, though, I much prefer to have the output directory (and the separate aux directory, if used) as subdirectories of the document directory. It's more convenient to have all the relevant files for a document near each other. (That's my usual way of working.)

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .