How to extend filecontents to append and count lines?

Question

Here are two questions about filecontents environment:

1. is it possible to add content to a preexisting file or, at a minimum, to keep open the external file to append the content of several filecontents environments to the same external file?

2. is it possible to know how many lines have been written to the external file by a filecontents environment?

Answer 1

Counting the lines in filecontents is easy because the environment processes its contents line by line, reading, checking if the environment ended, and then writing. You just need to make a counter and add it every time a line is written.

Appending is trickier. First off, TeX (without Lua at least) can't open files with "append" permission, so that's ruled out. The idea of leaving the file open, and then resume writing later seems easier at first, but you'd need to save a bunch of variables across environments, and then you wouldn't be able to write to file a, then b, and then append to a.

In the code below the, \filecontents@preappend macro (when the append option is used) reads the file in the argument to a temporary macro, and then the \filecontents@append puts the read macro in the input stream before the actual contents of the environment, so it works just as if you typed everything in one go. The append option implies the overwrite and noheader options.

I was trying to write the code as a patch, for compactness, but apparently patching a command loses the ^^J characters, and then the header collapses to a single line, so this is a complete redefinition of \filec@ntents, but the changed lines are marked.

\documentclass{article}
\usepackage{expl3}
\ExplSyntaxOn
\cs_new_eq:NN \FileSize \file_size:n
\ExplSyntaxOff
\makeatletter
\begingroup%
\@tempcnta=1
\loop
  \catcode\@tempcnta=12  %
  \advance\@tempcnta\@ne %
\ifnum\@tempcnta<32      %
\repeat                  %
\catcode`\^^M\active
\catcode`\^^L\active\let^^L\relax
\catcode`\^^I\active
\gdef\filec@ntents#1{%
  \set@curr@file{\filec@ntents@checkdir#1}%
  \edef\q@curr@file{\expandafter\quote@name\expandafter{\@curr@file}}%
  \filecontents@preappend% <- ADDED for appending
  \gaborit@reset@counter% <- ADDED for counting lines
  \gaborit@reset@append% <- ADDED for counting lines
  \openin\@inputcheck\q@curr@file \space %
  \ifeof\@inputcheck%
    \@latex@warning@no@line%
        {Writing file `\@currdir\@curr@file'}%
    \chardef\reserved@c15 %
    \ch@ck7\reserved@c\write%
    \immediate\openout\reserved@c\q@curr@file\relax%
  \else%
    \if@filesw%
      \@latex@warning@no@line%
          {File `\@curr@file' already \filec@ntents@where.\MessageBreak%
             Not generating it from this source}%
      \let\gaborit@step@counter\@empty% <- ADDED for counting lines
      \let\gaborit@count@header\@empty% <- ADDED for counting lines
      \let\write\@gobbletwo%
      \let\closeout\@gobble%
    \else%
      \edef\reserved@a{#1}%
      \edef\reserved@a{\detokenize\expandafter{\reserved@a}}%
      \edef\reserved@b{\detokenize\expandafter{\jobname}}%
      \ifx\reserved@a\reserved@b%
        \@fileswtrue%
      \else%
        \edef\reserved@b{\reserved@b\detokenize{.tex}}%
        \ifx\reserved@a\reserved@b
          \@fileswtrue%
        \fi%
      \fi%
      \chardef\reserved@c15 %
      \ch@ck7\reserved@c\write%
      \if@filesw%  % Foul ... trying to overwrite \jobname!
      \@latex@error{Trying to overwrite `\jobname.tex'}{You can't %
        write to the file you a reading from!\MessageBreak%
        Data is written to screen instead.}%
      \else%
        \@latex@warning@no@line%
           {Writing or overwriting file `\@currdir\@curr@file'}%
        \immediate\openout\reserved@c#1\relax%
      \fi%
    \fi%
  \fi%
  \closein\@inputcheck%
  \if@tempswa%
    \gaborit@count@header% <- ADDED for counting lines
    \immediate\write\reserved@c{%
      \@percentchar\@percentchar\space%
          \expandafter\@gobble\string\LaTeX2e file `\@curr@file'^^J%
      \@percentchar\@percentchar\space  generated by the %
        `\@currenvir' \expandafter\@gobblefour\string\newenvironment^^J%
      \@percentchar\@percentchar\space from source `\jobname' on %
         \number\year/\two@digits\month/\two@digits\day.^^J%
      \@percentchar\@percentchar}%
  \fi%
  \let\do\@makeother\dospecials%
  \count@ 128\relax%
  \loop%
    \catcode\count@ 11\relax%
    \advance\count@ \@ne%
    \ifnum\count@<\@cclvi%
  \repeat%
  \edef\E{\@backslashchar end\string{\@currenvir\string}}%
  \edef\reserved@b{%
    \def\noexpand\reserved@b%
         ####1\E####2\E####3\relax}%
  \reserved@b{%
    \ifx\gaborit@reset@append##1\relax% <- ADDED for counting lines
      \gaborit@reset@append% <- ADDED for counting lines
    \else% <- ADDED for counting lines
    \ifx\relax##3\relax%
      \immediate\write\reserved@c{##1}%
      \gaborit@step@counter% <- ADDED for counting lines
    \else%
      \edef^^M{\noexpand\end{\@currenvir}}%
      \ifx\relax##1\relax%
      \else%
          \@latex@warning{Writing text `##1' before %
             \string\end{\@currenvir}\MessageBreak as last line of \@curr@file}%
        \immediate\write\reserved@c{##1}%
        \gaborit@step@counter% <- ADDED for counting lines
      \fi%
      \ifx\relax##2\relax%
      \else%
         \@latex@warning{%
           Ignoring text `##2' after \string\end{\@currenvir}}%
      \fi%
    \fi%
    \fi% <- ADDED for counting lines
    ^^M}%
  \catcode`\^^L\active%
  \let\L\@undefined%
  \def^^L{\expandafter\ifx\csname L\endcsname\relax\fi ^^J^^J}%
  \catcode`\^^I\active%
  \let\I\@undefined%
  \def^^I{\expandafter\ifx\csname I\endcsname\relax\fi\space}%
  \catcode`\^^M\active%
  \edef^^M##1^^M{%
    \noexpand\reserved@b##1\E\E\relax}%
  \filecontents@append}% <- ADDED for appending
%
% Code for append
\gdef\gaborit@pre@append{%
  \begingroup%
    \catcode`\^^L\active%
    \catcode`\^^I\active%
    \catcode`\^^M\active%
    \let\do\@makeother\dospecials%
    \count@ 128\relax%
    \loop%
      \catcode\count@ 11\relax%
      \advance\count@ \@ne%
      \ifnum\count@<\@cclvi%
    \repeat%
    \let^^M\relax%
    \edef\gaborit@tmpa{\FileSize{\q@curr@file}}%
    \ifnum\expandafter\@car\gaborit@tmpa\@nil=0\relax%
      \endgroup \def\gaborit@append{^^M\gaborit@reset@append}%
    \else%
      \everyeof{\noexpand}%
      \edef\gaborit@tmpa{\@@input\q@curr@file \space}%
      \edef\x{\endgroup%
        \edef\noexpand\gaborit@append{%
          \noexpand\gaborit@trim@EOF\gaborit@tmpa\noexpand\gaborit@EOF}}\x%
    \fi%
  \filec@ntents@overwrite%
  \filec@ntents@noheader}%
\gdef\gaborit@trim@EOF#1^^M\gaborit@EOF{^^M#1%
  ^^M\noexpand\gaborit@reset@append}%
\endgroup%
\def\filec@ntents@append{%
  \let\filecontents@preappend\gaborit@pre@append
  \def\filecontents@append{\gaborit@append}}
\let\filecontents@preappend\@empty
\let\filecontents@append\@empty
%
% For counting lines
\newcounter{FC@total@lines}
\newcounter{FC@lines}
\def\gaborit@reset@counter{\setcounter{FC@total@lines}{0}}
\def\gaborit@reset@append{\setcounter{FC@lines}{0}}
\def\gaborit@step@counter{%
  \stepcounter{FC@total@lines}%
  \stepcounter{FC@lines}}
\def\gaborit@count@header{%
  \addtocounter{FC@total@lines}{4}%
  \addtocounter{FC@lines}{4}} % Number of lines in the header
\makeatother

\begin{filecontents}[overwrite]{testfile.tex}
a
b
c
\end{filecontents}
\typeout{\arabic{FC@total@lines} lines written; \arabic{FC@lines} appended}

\begin{filecontents}[append]{testfile.tex}
d
e
f
g
\end{filecontents}
\typeout{\arabic{FC@total@lines} lines written; \arabic{FC@lines} appended}

\begin{document}
\end{document}

Running this code, TeX prints to the terminal:

LaTeX Warning: Writing or overwriting file `./testfile.tex'.

7 lines written; 7 appended
(./testfile.tex)

LaTeX Warning: Writing or overwriting file `./testfile.tex'.

11 lines written; 4 appended

(the FC@total@lines holds the total number of lines in the file, and the FC@lines counter holds only the number of appended lines) and the file testfile.tex contains:

%% LaTeX2e file `testfile.tex'
%% generated by the `filecontents' environment
%% from source `test' on 2019/12/20.
%%
a
b
c
d
e
f
g

---

As you noted, when you do:

\begin{filecontents*}{testfile.tex}
\end{filecontents*}
%
\begin{filecontents}[append]{testfile.tex}
a
\end{filecontents}

(that is, write nothing, then append a line with a) the file you get is:

a

That's because when TeX reads an empty file, it treats that file as if it were a file with a single empty line¹, thus you can't easily differentiate between those two cases and have to choose the lesser evil: either have a spurious empty line at the beginning of the file when appending to an empty file, or have an empty line removed when writing to a file which contains a single empty line.

However, with the file utilities introduced in pdfTeX (but also available in other engines nowadays) you can query the size (in bytes) of a file, so you can have a different behaviour for an empty file. In the code I used expl3's \file_size:n to avoid having to deal with nonexisting files.

---

^1. You can see this with this test file:

\catcode`\@=11
\newwrite\test
\def\testit#1{\immediate\openout\test \jobname.testfile\relax #1%
  {\catcode`\^^M=13 \everyeof{\noexpand}%
   \edef\tmpa{\ifdefined\@@input \@@input \else \input \fi \jobname.testfile }\show\tmpa}%
  \immediate\closeout\test}%
\testit{}% empty file
\testit{\immediate\write\test{}}% single empty line
\csname stop\endcsname\bye

This behaviour, I think, is due to how TeX adds an \endlinechar at the end of every line. When it reads an empty file, it still reads a line (even if empty), and then it inserts the \endlinechar (^^M) and this makes it look like as if the file had a single empty line.