Different file opened by \openin (was: between 2017 and 2018)

Question

Short version:

It seems that some trivial behavior of file opening (\openin) changed between TeX Live 2017 and 2018 differs between environment. Only Overleaf one behaves differently. I did not see any notice about that, but why?

---

Consider the following code:

\openin1 \read1 to \X \closein1 \show\X

Normally \openin should be followed by a file, but here that is omitted. TeX/e-TeX/e-(u)pTeX/XeTeX/LuaTeX of Overleaf (TL2017) showed \par, but my local installation (TL2018) and Cloud LaTeX (TL2017), Japanese TeX on Web (TL2016; only e-pTeX available) shows nothing (empty).

Similar difference with e-TeX:

\openin1 \readline1 to\X\show\X\readline1 to\X\show\X\readline1 to\X\show\X\closein1 \bye

e-TeX of Overleaf returned only ^^M and stopped, but that of my local installation shows:

> \X=macro:
->%%^^M.
<*> \openin1 \readline1 to\X\show\X
                                   \readline1 to\X\show\X\readline1 to\X\sho...

?
> \X=macro:
->%% This is file `.tex',^^M.
<*> ...readline1 to\X\show\X\readline1 to\X\show\X
                                                  \readline1 to\X\show\X\clo...

?
> \X=macro:
->%% generated with the docstrip utility.^^M.
<*> ...readline1 to\X\show\X\readline1 to\X\show\X
                                                  \closein1 \bye
?
No pages of output.

The followings are my guess from these behavior:

• ??? was read in Overleaf,
• tools/.tex is being read in my local installation, Cloud LaTeX

I don't think this is a problem; just for curiosity.

Answer 1

A few observations:

• As of now, it appears that the atypical behaviour can be reproduced only on Overleaf, though an earlier version of the question mentioned the OP's observations on TL 2017 and on Japanese TeX on Web.

• On Overleaf, this behaviour (where \X gets set to \par) can be reproduced not only with \openin1 \read1 to \X (as in the question) but even with \openin1= \read1 to \X and \openin1="" \read1 to \X, but not with \openin1=".tex" \read1 to \X (in which case \X gets set to the empty macro, same as on other environments). In fact, with \verbatiminput{.tex} (compiling with LaTeX and \usepackage{verbatim}), one can see that even on Overleaf, the same .tex file is present, which we see loaded on other environments (texmf-dist/tex/latex/tools/.tex). (This is interesting, that reading from "" is not the same as reading from ".tex".) So with that explanation ruled out, let's stick to this \openin1="" \read1 to \X form as it's perhaps most clear.

• Sometimes \read tries to read from the terminal, so this is another potential explanation. But if we try \openin1="nonexistent.tex" \read1 to \X on Overleaf, we get a message from TeX about “cannot \read from terminal in nonstop modes”, suggesting that TeX is being run with \batchmode or \nonstopmode. Even if we try to force the issue by first doing \errorstopmode in our file, we get “End of file on the terminal!”. So this explanation is ruled out as well, which only deepens the mystery: if TeX is not reading from the file .tex and it's not reading from the terminal, where is it reading from?

Based on these observations, we can debug as follows: create a latexmkrc file containing

$latex = 'tex -recorder %O %S';

to record this, and prepare the following input file:

\openin 1=".tex" 
\read 1 to \X 
\message{... With full filename, \meaning\X ...}
\closein 1

\openin 2=""
\read 2 to \X
\message{... With blank filename, \meaning\X ...}
\closein 2

\openin 3=".tex" 
\read 3 to \X 
\message{... With full filename, \meaning\X ...}
\closein 3

\obeylines
\input \jobname.fls
\bye

(3 is the same as 1, just repeated to make the ordering clear). The log file contains:

This is TeX, Version 3.14159265 (TeX Live 2017) (preloaded format=tex 2017.7.12)  29 DEC 2018 10:20
**main.tex
(/compile/main.tex ... With full filename, macro:->... ... With blank filename,
 macro:->\par ... ... With full filename, macro:->... (/compile/output.fls) [1]
 )
Output written on /compile/output.dvi (1 page, 716 bytes).

which illustrates the oddity, but even more interesting is the typeset output:

PWD /compile
INPUT /usr/local/texlive/2017/texmf.cnf
INPUT /usr/local/texlive/2017/texmf-dist/web2c/texmf.cnf
INPUT /usr/local/texlive/2017/texmf-var/web2c/tex/tex.fmt
INPUT /compile/main.tex
OUTPUT /compile/output.log
INPUT /usr/local/texlive/2017/texmf-dist/tex/latex/tools/.tex
INPUT /compile/
INPUT /usr/local/texlive/2017/texmf-dist/tex/latex/tools/.tex
INPUT /compile/output.fls

-- note the INPUT /compile/ between the two occurrences of INPUTing .tex.

This seems to answer the mystery somewhat, while raising new ones: why does TeX (on Overleaf, and when given an empty filename) input (what appears to be) the directory containing the input file? (And what does it even mean to input a directory / why is the effect the same as reading an empty file?)

Unfortunately this is all very much in the system-dependent parts of TeX (i.e. in this case it's entirely internal to the kpathsea library), so to debug further without being able to reproduce locally seems hard. (Perhaps it has something do with chroot or some sort of sandboxing that Overleaf does.) I'll note though that there was a kpathsea commit on 2017-11-03 (so wasn't part of TL2017) that has something to do with empty filenames. Perhaps it's related.