I can't write on file 'foo.aux'

Question

I am getting an error when compiling this template with pdflatex -output-directory=./_out thesis.tex:

! I can't write on file `text/abbreviations.aux'.
\@include ...\immediate \openout \@partaux #1.aux 
                                                  \immediate \write \@partau...
l.187 \include{text/abbreviations}

(Press Enter to retry, or Control-D to exit; default file extension is `.tex')
Please type another output file name: ^D
! Emergency stop.
\@include ...\immediate \openout \@partaux #1.aux 
                                                  \immediate \write \@partau...
l.187 \include{text/abbreviations}

!  ==> Fatal error occurred, no output PDF file produced!
Transcript written on _out/thesis.log.

This is usually the way that I use the -output-directory option, and I have never had a problem with that usage before, but I must be doing something wrong because the compiling works fine without this option.

---

File Tree

Result of ls -R from the project root:

_README.txt     _out            figures         ociamthesis.cls references.bib  splitcolor.py   text            thesis.tex

./_out:
thesis.aux       thesis.bcf       thesis.lof       thesis.mtc       thesis.out
thesis.bbl       thesis.blg       thesis.log       thesis.mtc0      thesis.toc

./figures:
beltcrest.pdf sample

./figures/sample:
Gray498.png

./text:
abbreviations.tex    abstract.tex         acknowledgements.tex appendix-1.tex       ch1-intro.tex        ch2-litreview.tex

Version

I am on OSX and the manual for pdflatex doesn't show any option -aux_directory, my version is:

pdfTeX 3.14159265-2.6-1.40.16 (TeX Live 2015)
kpathsea version 6.2.1
Copyright 2015 Peter Breitenlohner (eTeX)/Han The Thanh (pdfTeX).
There is NO warranty.  Redistribution of this software is
covered by the terms of both the pdfTeX copyright and
the Lesser GNU General Public License.
For more information about these matters, see the file
named COPYING and the pdfTeX source.
Primary author of pdfTeX: Peter Breitenlohner (eTeX)/Han The Thanh (pdfTeX).
Compiled with libpng 1.6.17; using libpng 1.6.17
Compiled with zlib 1.2.8; using zlib 1.2.8
Compiled with xpdf version 3.04

Answer 1

@DavidCarlisle proposed a solution in comment of the OP. It seems to be sufficient to create a subdirectory _out/text for the compilation to succeed.

Answer 2

I had the same issue and solved it by writing a build script that creates subdirectories in my build folder that mirror the subdirectories in my source folder before compiling:

#!/bin/sh

for OUTPUT in $(find src -type d)
do
  mkdir dist/$OUTPUT
done

pdflatex -output-directory=./dist ./src/main.tex

This saves you having to manually create subdirectories every time you add one.

Answer 3

Inspired by this answer I wrote a proper shell script¹ to automatically create all needed output directories. It handles filenames containing whitespace correctly - it can't cope with filenames containing a newline character though².

---

Usage

Save the below code as create_output_dirs.sh (or whatever you like it to be named) and make it executable with chmod +x create_output_dirs.sh.

Make sure you are in the root directory of your Latex project and execute the script.

E.g. if the script is saved in your home directory:

~/.create_output_dirs.sh outdir

This will create a outdir directory and replicate the directory structure of the current directory inside it. If no argument is given it will use dst as destination directory.

It will not recurse the destination directory when looking for directories to create - you can run it multiple times without deleting the output directory.

---

Code

#!/bin/sh

# if no parameter is given, use 'dst' as destination directory
if [ -z "$1" ]; then
    dst=dst
else
    dst="$1"
fi

# find all directories which are not the destination dir or inside it
find . -type d -not -name "$dst" -not -path "./$dst/*" -printf '%P\n' \
    | while IFS= read -r dir
do
    mkdir -p "$dst/$dir"
done

---

¹ _{This means a POSIX shell. Also it needs find}
² _{You really shouldn't have those filenames!}

Answer 4

This is my solution, the same as Kyle Meenehan, but a little more elaborate using Makefile:

#!/usr/bin/make -f
# https://stackoverflow.com/questions/7123241/makefile-as-an-executable-script-with-shebang
ECHOCMD:=/bin/echo -e
SHELL := /bin/bash

# The main latex file
THESIS_MAIN_FILE := main

# This will be the pdf generated
THESIS_OUTPUT_NAME := thesis

# This is the directory where the temporary files are going to be
CACHE_DIRECTORY := cache
THESIS_MAIN_FILE_PATH := $(CACHE_DIRECTORY)/$(THESIS_MAIN_FILE).pdf

# Find all files ending with `main.tex`
LATEX_SOURCE_FILES := $(wildcard *main.tex)

# Create a new variable within all `LATEX_SOURCE_FILES` file names ending with `.pdf`
LATEX_PDF_FILES := $(LATEX_SOURCE_FILES:.tex=.pdf)

# https://stackoverflow.com/questions/55642491/how-to-check-whether-a-file-exists-outside-a-makefile-rule
FIND_EXEC := $(if $(wildcard /bin/find),,/usr)/bin/find

.PHONY: all help latex thesis verbose clean biber_hook biber_hook1 pdflatex_hook \
pdflatex_hook1 pdflatex_hook2 pdflatex_hook3 pdflatex_hook4

# http://stackoverflow.com/questions/1789594/how-do-i-write-the-cd-command-in-a-makefile
.ONESHELL:

# Default target
# https://stackoverflow.com/questions/46135614/how-to-call-makefile-recipe-rule-multiple-times
all: pdflatex_hook1 biber_hook1 pdflatex_hook2 pdflatex_hook3 pdflatex_hook4 biber


# https://tex.stackexchange.com/questions/91592/where-to-find-official-and-extended-documentation-for-tex-latexs-commandlin
# https://tex.stackexchange.com/questions/52988/avoid-linebreaks-in-latex-console-log-output-or-increase-columns-in-terminal
PDF_LATEX_COMMAND = pdflatex --synctex=1 -halt-on-error -file-line-error
PDF_LATEX_COMMAND += $(if $(shell pdflatex --help | grep time-statistics),--time-statistics,)
PDF_LATEX_COMMAND += $(if $(shell pdflatex --help | grep max-print-line),--max-print-line=10000,)

LATEX = $(PDF_LATEX_COMMAND) --interaction=batchmode
LATEX += $(if $(shell pdflatex --help | grep aux-directory),-aux-directory="$(CACHE_DIRECTORY)",)
LATEX += $(if $(shell pdflatex --help | grep output-directory),-output-directory="$(CACHE_DIRECTORY)",)

# Copies the PDF to the current directory
# https://stackoverflow.com/questions/55671541/how-define-a-makefile-condition-and-reuse-it-in-several-build-rules/
define copy_resulting_pdf=
    if [[ -f "${THESIS_MAIN_FILE_PATH}" ]]; \
    then \
        printf 'Coping PDF...\n'; \
        cp "${THESIS_MAIN_FILE_PATH}" "${current_dir}/${THESIS_OUTPUT_NAME}.pdf"; \
    else \
        printf "\\nError: The PDF "${THESIS_MAIN_FILE_PATH}" was not generated!\\n"; \
        exit 1; \
    fi
endef

# https://stackoverflow.com/questions/4210042/exclude-directory-from-find-command
DIRECTORIES_TO_CREATE := $(shell "${FIND_EXEC}" -not -path "./**.git**" \
-not -path "./pictures**" -type d \
-not -path "./${CACHE_DIRECTORY}**" -type d)

# https://stackoverflow.com/questions/11469989/how-can-i-strip-first-x-characters-from-string-using-sed
define setup_envinronment =
    $(eval current_dir := $(shell pwd)) echo $(current_dir) > /dev/null
    printf "\\n";

    readarray -td' ' DIRECTORIES_TO_CREATE_ARRAY <<<"$(DIRECTORIES_TO_CREATE) "; \
    unset 'DIRECTORIES_TO_CREATE_ARRAY[-1]'; \
    declare -p DIRECTORIES_TO_CREATE_ARRAY; \
    for directory_name in "$${DIRECTORIES_TO_CREATE_ARRAY[@]}"; \
    do \
        full_cache_directory="${CACHE_DIRECTORY}/$${directory_name:2}"; \
        printf "Creating %s\\n" "$${full_cache_directory}"; \
        mkdir -p "$${full_cache_directory}"; \
    done
    printf "\\n";
endef


# Run pdflatex, biber, pdflatex
biber: setup_things biber_hook pdflatex_hook
    $(copy_resulting_pdf)


setup_things:
    $(setup_envinronment)


# Call biber to process the bibliography and does not attempt to show the elapsed time
# https://www.mankier.com/1/biber --debug
biber_hook biber_hook1:
    $(setup_envinronment)

    echo "Running biber quietly..."
    biber --quiet --input-directory="$(CACHE_DIRECTORY)" --output-directory="$(CACHE_DIRECTORY)" $(THESIS_MAIN_FILE).bcf


# https://stackoverflow.com/questions/46135614/how-to-call-makefile-recipe-rule-multiple-times
pdflatex_hook pdflatex_hook1 pdflatex_hook2 pdflatex_hook3 pdflatex_hook4: setup_things
    @$(LATEX) $(LATEX_SOURCE_FILES)


# This rule will be called for every latex file and pdf associated
latex: $(LATEX_PDF_FILES)
    $(setup_envinronment)


# Dynamically generated recipes for all PDF and latex files
%.pdf: %.tex
    $(setup_envinronment)
    @$(LATEX) $<


clean:
    $(RM) -rv $(CACHE_DIRECTORY)
    $(RM) -v $(THESIS_OUTPUT_NAME).pdf

See the original and full Makefile here:

1. https://github.com/evandrocoan/ufscthesisx/blob/master/Makefile
2. https://github.com/evandrocoan/ufscthesisx-setup/blob/master/makefile.mk
3. https://github.com/evandrocoan/ufscthesisx-setup/blob/master/makerules.mk