latexmk: build in another directory and move back file once it's compiled (avoid flickering pdf)

Question

I'm using emacs+latexmk to compile my document, and okular to view it. Okular automatically updates the pdf when it changes, but the issue is that during compilation Okular displays a black document since the pdf is currently being created. So during the compilation it means that I can't look at my file, which is a bit annoying since often I want to continue to read my document meanwhile.

What is the simpler way, in latexmk, to compile the document in another folder, and copy back the final pdf once it's fully compiled? That way, okular will let me read my pdf during the compilation process.

EDIT

Concerning the solution proposed by John, it seems that when I run this command:

$ latexmk main.tex 
Rc files read:
  /home/me/.latexmkrc
Latexmk: This is Latexmk, John Collins, 29 September 2020, version: 4.70b.
Set environment variable BIBINPUTS='build:'
Set environment variable TEXINPUTS='build:.:/home/me/.emacs.d/.local/straight/build-27.2/auctex/latex:'
Latexmk: applying rule 'pdflatex'...
Rule 'pdflatex': The following rules & subrules became out-of-date:
      'pdflatex'
------------
Run number 1 of rule 'pdflatex'
------------
------------
Running 'pdflatex  -recorder -output-directory="." -aux-directory="build"  "main.tex"'
------------
/nix/store/hr2wphv47rqnrckmcjavn1r5yq7a8ks0-texlive-combined-full-2021.20210408/bin/pdflatex: unrecognized option '-aux-directory=build'
This is pdfTeX, Version 3.141592653-2.6-1.40.22 (TeX Live 2021/NixOS.org) (preloaded format=pdflatex)
 restricted \write18 enabled.
entering extended mode
(./main.tex
LaTeX2e <2020-10-01> patch level 4
[...]

the file is not just copied at the end. Indeed, if I regularly check the size of the pdf during the compilation, I see that the size increases:

$ md5sum main.pdf; ls -alh main.pdf
58e69aead33520a0228f797773ff0030  main.pdf
-rw-r--r-- 1 me users 1,1M sept. 30 09:52 main.pdf

$ md5sum main.pdf; ls -alh main.pdf
42ad0da92fc621c7106101b21ad60a96  main.pdf
-rw-r--r-- 1 me users 18K sept. 30 09:52 main.pdf

$ md5sum main.pdf; ls -alh main.pdf
0cbf954ce5a1723304561673088bec65  main.pdf
-rw-r--r-- 1 me users 115K sept. 30 09:52 main.pdf

$ md5sum main.pdf; ls -alh main.pdf
59ee8e1ac1c2aa785b0f4b3444b5f0b3  main.pdf
-rw-r--r-- 1 me users 148K sept. 30 09:52 main.pdf

$ md5sum main.pdf; ls -alh main.pdf
1f15aff8cb79793a9a0b31495e5a7c21  main.pdf
-rw-r--r-- 1 me users 178K sept. 30 09:52 main.pdf

I tried to force the output to be in another folder and copy it back using $pdf_update_command like that:

$emulate_aux = 1;
$out_dir = 'out';
$aux_dir = 'build';
$pdf_previewer = '';
$pdf_update_method = 4;
$pdf_update_command = 'cp out/*.pdf .';
$pdf_mode = 1; # Default to pdf.

but the update command is not run, not sure why.

EDIT So as pointed out in comments, $pdf_update_command is ran only in -pvc mode. So I tried instead:

$out_dir = 'out';
$pdflatex = 'pdflatex %O %S && cp %D .';
$pdf_mode = 1; # Defaults to pdf.

and now the upgrade is great (I get a refresh at every iteration, without any flickering), but there is an issue with bibliography, it is not found anymore:

MWE:

\documentclass{article}

\usepackage{filecontents}
\usepackage[style=trad-alpha]{biblatex}

\begin{filecontents}{myrefs.bib}
@Book{Knuth:1990,
    author    = {Knuth, Donald E.},
    title     = {The {\TeX}book},
    year      = {1990},
    isbn      = {0-201-13447-0},
    publisher = {Addison\,\textendash\,Wesley},
}

@Book{Lamport:94,
    author    = {Lamport, Leslie},
    title     = {\LaTeX: A Document Preparation System},
    year      = {1994},
    isbn      = {0-021-52983-1},
    publisher = {Addison\,\textendash\,Wesley},
}
\end{filecontents}

\addbibresource{myrefs.bib}

\begin{document}

\section{Main Body}
{\LaTeX} is a Turing-complete (procedural) markup language and typesetting processor~\cite{Lamport:94}.

\printbibliography

\end{document}

Answer 1

If you don't mind having the aux, log, etc files in one directory, and the pdf file in another, recent versions (at least 4.73) of latexmk can do exactly what you want. Here's suitable code in a latexmkrc file:

$emulate_aux = 1;
$out_dir = 'output';
$aux_dir = 'aux';

The critical item is the first one. It tells latexmk that it must not supply the -aux-directory option to pdflatex (or whichever program you use), but must emulate its effect. The compilation itself produces the pdf file in the aux directory, and then latexmk moves it to the output directory, which is a rapid operation. (It also moves the fls file and any synctex file, as well.)

The reason that latexmk has this possibility is that it is only in the MiKTeX implementation that the tex programs support the -aux-directory option. TeXLive's programs don't. The official TeXLive position is that such things should be handled by a program such as latexmk. So that is exactly what's now implemented.

Answer 2

It looks like I've a too old latexmk for the solution of John Collins to work, and configuring out and build seems to break biblatex. So I found two solutions I don't really like (because I need to disable okular auto-update feature) but that work quite well:

Solution 1 (emacs reloads okular after the last compilation)

Using latexmk in emacs + disable auto-reload in Okular (Configuration > Configure Okular > Reload document when file is updated (tab general, approximate translation)), emacs automatically calls Okular at the end of the compilation.

To configure emacs, I'm actually using doom emacs (with options (latex +latexmk)) and my .config contains something like that (should work with emacs, but my actual configuration is a bit different so let me know if it fails):

(after! latex
  ;; you can get the list of builtin viewers using C-h v TeX-view-program-list-builtin RET
  ;; But for some reasons, the default Okular viewer is broken due to a "file:"...
  (add-to-list 'TeX-view-program-list '("Okular" ("okular --unique %o"
                                                  (mode-io-correlate "#src:%n%a"))))
  (setq TeX-view-program-selection '((output-pdf "Okular")))

Solution 2 (latexmk reloads okular at every compilation)

Disable auto-reload in Okular (Configuration > Configure Okular > Reload document when file is updated (tab general, approximate translation)), and add in ~/.latexmkrc:

$pdflatex = 'pdflatex %O %S && okular --unique %D';

With this solution, okular will be reloaded by latexmk directly (even if ran from command line) at every new compilation (nice to have an incremental preview).