Reference label with included file and relaxed clearpage

Question

I am running into a rather weird error that apparently originates from the combination of \include, \let\clearpage\relax and a referenced label after a page break.

In this particular example, the label mylabel defined in the listing cannot be referenced anymore when the reference and the label are close enough to a page break.

The MWE consists of two files, the main-file and an included file. I am using the \let\clearpage\relax command to avoid page breaks after every \include. The error does not occur if the file is \input instead of \included.

\documentclass[11pt,twoside]{book} % 
\usepackage[utf8]{inputenc} % Enoding
\usepackage[english]{babel} % Language
\usepackage{listings}
\usepackage{forloop}

\begin{document}
\begingroup
\let\clearpage\relax

\include{includedfile} % changing include for input would work

\endgroup
\end{document}

Included file:

\newcounter{ct}
\forloop{ct}{1}{\value{ct} < 40}%
{%
    .\\
}

Listing \ref{mylabel}.

\begin{lstlisting}[caption=somecaption, label=mylabel]
some code;
\end{lstlisting}

Running this example will have an output where the label cannot be referenced (even after multiple runs):

Now, one solution is to use \input as mentioned earlier but I am rather interested in why this happens. Does the \clearpage command fulfil an important function for references to work? And can the error be avoided without removing \let\clearpage\relax while using \include?

EDIT: I just found this question with the same issue (though it does not have a definite answer).

Answer 1

\include uses a separate .aux file. If the file is not included, the .aux file is still read and the references of the .aux file are available.

Entries into the .aux file can be written \immediate or at the time, when the page is shipped out. Therefore \include starts a new page. Thus the labels of the previous page goes into the previous .aux file. Then the .aux files are switched immediately (\immediate\openout). The labels go into the .aux file of the included file. After the included file is processed, \include calls \clearpage to output any stuff of the included file including labels, which goes into the .aux file of \include. Then again the .aux files are switched, the include .aux file is immediately closed and the next writes to an .aux file is going to the main .aux file.

With \let\clearpage\relax you destroy the capabilities of \include. In the case of the question, the last page of the included file contains the label. The shipped out page contains the command to write the label to file includedfile.aux. But the page is not yet shipped out before the end of \include and the file handle for includedfile.aux is closed. TeX then redirects the write request to a closed file handle at the next page shipout to the .log file, where you will find:

\newlabel{mylabel}{{1}{2}}
\@writefile{lol}{\contentsline {lstlisting}{\numberline {1}somecaption}{2}}

Also the entry into the list of listings will be missing.

Summary:

---

As addendum the code for the "flashy thing" as requested by kan:

\documentclass{article}
\pagestyle{empty}
\usepackage{xcolor}
\usepackage[T1]{fontenc}
\usepackage{lmodern}
\begin{document}
\newcommand*{\doit}[2][]{%
  \ifx\\#1\\%
    \color{#2}%
  \else
    \color[{#1}]{#2}%
  \fi
  \noindent
  \Large
  \sffamily
  \bfseries
  Never disable page breaks with %
  \texttt{\textbackslash include}!\\
  Use \texttt{\textbackslash input} instead.
  \newpage
}
\newcommand*{\x}[1]{%
  \doit[Hsb]{0,1,1}%
  \doit[Hsb]{#1,1,1}%
}
\x{30}
\x{60}
\x{90}
\x{120}
\x{150}
\x{180}
\x{210}
\x{240}
\x{270}
\x{300}
\x{330}
\end{document}

Then

pdflatex test
pdfcrop test.pdf
gs -sDEVICE=png256 -q -dQUIET -dSAFER -dBATCH -dNOPAUSE -dNOPROMPT -dMaxBitmap=500000000 -dAlignToPixels=0 -dGridFitTT=2 -dPDFSETTINGS=/prepress -dUseFastColor=true -r216x216 -sOutputFile=test%d.png -ftest-crop.pdf
convert -delay 30 -loop 0 test1.png test2.png test3.png test4.png test5.png test6.png test7.png test8.png test9.png test10.png test11.png test12.png test13.png test14.png test15.png test16.png test17.png test18.png test19.png test20.png test21.png test22.png test.gif