3

I'm trying to follow along with this answer to automagically include files based on the filesystem hierarchy.

My file structure looks like:

article.tex
sections/
|-- 00-introductions.tex
|-- 10-data.tex
|-- 20-analysis.tex
|-- 90-conclusions.tex
|-- data/
|   |-- 00-xxx.tex 
|   |-- NN-xxx.tex
|
|-- analysis/
    |-- 00-xxx.tex
    |-- NN-xxx.tex

So I want to include everything inside sections in article.tex, everything in sections/data in 10-data.tex, and everything in sections/analysis in 20-analysis.tex. For the purposes of this question, let's just focus on article.tex.

In an attempt to follow the linked answer, I've created the following bash script to generate a list of the appropriate files:

#! /bin/bash
ls -mp sections/ | grep -v / | tr ' ' '\n' > sections.list

And I've added this to my .latexmkrc so that it runs on every build (this bit works). It outputs the following (the new line is there if that's relevant):

00-introduction.tex,
10-data.tex,
20-analysis.tex,
30-conclusions.tex,

I've also tried an alternative that didn't have the commas appended.

My articles.tex (sans cruft) looks like this:

\documentclass[twocolumn, a4paper]{article}

\usepackage{pgffor}

\newcommand*{\listofsections}{\input{sections.list}}

\begin{document}

\foreach \c in \listofsections {%
    \input{sections/\c}%
}%

\end{document}

And if I just output \listofsections it looks fine to me:

file list

It looks as though something's wrong in my definition of the loop as these errors are popping up in the logs:

/home/james/repos/discos-paper/article.tex:28: Missing endcsname inserted. let l.28 } % /home/james/repos/discos-paper/article.tex:28: Use of ??? doesn't match its definition. ???
! LaTeX3 Error: let invalid in file name. Lost: reserved@d =... l.28 } % /home/james/repos/discos-paper/article.tex:28: Missing endcsname inserted. let l.28 } % /home/james/repos/discos-paper/article.tex:28: Use of ??? doesn't match its definition. ???
! LaTeX3 Error: let invalid in file name. Lost: reserved@d =... l.28 } % /home/james/repos/discos-paper/article.tex:28: Missing endcsname inserted. let l.28 } % /home/james/repos/discos-paper/article.tex:28: Use of ??? doesn't match its definition. ???
! LaTeX3 Error: let invalid in file name. Lost: reserved@d =... l.28 } %

! LaTeX Error: File `sections/.tex' not found.

What have I missed here?


Edit 1:

For some reason, it seems as though my \listofsections is coming through as a single item and the iterator is only being called once...

I've also now tried editing the list file to be:

00-introduction.tex,10-data.tex,20-analysis.tex,30-conclusions.tex

Which still doesn't work, nor does it work if I place that in the \newcommand but curiously does if I copy the contents straight into the foreach loop... So I'm really not sure what's going wrong if I'm honest.

3 Answers 3

3

if sections.list has one filename per line (no comma)

00-introduction.tex
10-data.tex
20-analysis.tex
30-conclusions.tex

You can simply read one line at a time to get the filename, then input:

\documentclass{article}
\newread\sections
\openin\sections=sections.list
\begin{document}

\loop
{\endlinechar=-1 \global\read\sections to \next}
\ifx\next\empty\else
\input{\next}
\repeat

\end{document}
5
  • This does seem to work, but I'd appreciate it if you could explain the syntax and commands you've used a little more Commented Nov 21, 2022 at 0:17
  • 1
    \read is a tex primitive whch reads one line of a file then assigns it to a macro so it is like \newcommand\next{00-introduction.tex} on the first iteration. On the last it is empty so matches \empty and the loop stops @ScottishTapWater Commented Nov 21, 2022 at 0:42
  • @ScottishTapWater documentation in e.g. TeXbook or TeX by Topic. Although if the file names might contain "special characters" you might consider using \readline whose "documentation" (actually not documentation, more like "brief description") is in the ε-TeX manual
    – user202729
    Commented Nov 21, 2022 at 3:29
  • @DavidCarlisle - Quick follow up, I've tried to add a nested version of this to add files in sections/data to 10-data (which works) but then the outer-loop doesn't seem to continue... Would you have any idea why this might be or should I make a fresh question? Commented Nov 27, 2022 at 18:06
  • Actually, I've made a new question, it's here if you've got any ideas tex.stackexchange.com/questions/666649/… Commented Nov 27, 2022 at 18:15
2

I don't think that \foreach is the right tool, as it executes every item in the loop in a group.

You aren't defining \listofsections as the list, but as a command that inputs a file.

\begin{filecontents*}{\jobname-sections.list}
00-introduction.tex,
10-data.tex,
20-analysis.tex,
30-conclusions.tex,
\end{filecontents*}

\documentclass[twocolumn, a4paper]{article}

\usepackage{pgffor}
\usepackage{catchfile}

\CatchFileDef{\listofsections}{\jobname-sections.list}{}

\begin{document}

\foreach \c in \listofsections {%
  \if\relax\c\relax
  \else
    %\input{sections/\c}%
    I would input ``sections/\c''\endgraf
  \fi
}

\end{document}

I changed the name just to keep filenames distinguished in my “tryout” folder and used a “mock \input'' just to show what happens. The \if\relax\c\relax is needed to avoid a spurious final input.

enter image description here

My preferred way would be

\begin{filecontents*}{\jobname-sections.list}
00-introduction.tex,
10-data.tex,
20-analysis.tex,
30-conclusions.tex,
\end{filecontents*}

\documentclass[twocolumn, a4paper]{article}

\ExplSyntaxOn

\NewDocumentCommand{\inputfromlist}{m}
 {% #1 is the name of the file containing the list
  % store the contents of the file in \l_tmpa_tl
  \file_get:nnN { #1 } { } \l_tmpa_tl
  % transform the tl into a clist
  \clist_set:NV \l_tmpa_clist \l_tmpa_tl
  % cycle through the items i the list
  \clist_map_inline:Nn \l_tmpa_clist
   {
    %\input{##1}
    I~would~input~``sections/##1''\par
   }
 }

\ExplSyntaxOff

\begin{document}

\inputfromlist{\jobname-sections.list}

\end{document}

The function \file_get:nnN is the expl3 analog of \CatchFileDef. I then transform the contents into a clist (empty items are automatically removed) and map it.

What does \clist_set:NV \l_tmpa_clist \l_tmpa_tl do? It is the same as

\clist_set:Nn \l_tmpa_clist { <contents of \l_tmpa_tl> }

The \clist_set:Nn function does a normalization, so empty items (that is, consisting of zero or more spaces) are removed.

Finally, the \clist_map_inline:Nn function loops over the list's items. The current item is denoted by #1, but, since we're inside the definition of a command, we must use ##1.

2
  • Could you please elaborate on the syntax you're using? I'm not familiar with expl3 Commented Nov 21, 2022 at 0:23
  • 1
    @ScottishTapWater I added a few comments
    – egreg
    Commented Nov 21, 2022 at 0:36
1

If doing this without packages I might have TeX switch to verbatim-catcode-régime with \endlinechar denoting the number of the code-point of the carriage-return-character and input the file list via expandable \@@input and process it as a list of carriage-return-delimited arguments.

(The internal character encoding scheme with traditional TeX is ASCII. The internal character encoding scheme with LuaTeX and XeTeX is unicode whereof ASCII is a strict subset. The number of the code-point of the carriage-return-character is 13 both in ASCII and in unicode. In .tex-input-files the carriage-return-character can be denoted in TeX's ^^-notation via ^^M; M is the 13th letter in the latin alphabet. ^^-notation requires category-code 7(superscript) to be assigned to the character ^, which is the default.)

This way filenames are read under verbatim-catcode-régime. This might be a feature in case filenames contain sequences of space characters and/or characters like ^ etc that usually are/might be treated in special ways by TeX or—like { and }—need to be balanced when occurring within macro-arguments like the argument of the macro \input.

\futurelet is used only to ensure that TeX looks ahead and hereby removes the eof-token at the end of the file which otherwise might trigger a file ended while scanning the use of...-error when \inputfileloop grabs the last/appended \relax-dummy-line which marks the end of the loop. (Alternatively one could use \everyeof for appending the last dummy-line.)

\begin{filecontents*}{\jobname_inputfiles.txt}
00-introduction.tex
10-data.tex
20-analysis.tex
30-conclusions.tex
40-weird}%}  {^#^^61^^!.tex
\end{filecontents*}

\documentclass{article}

\begingroup
\makeatletter\endlinechar=-1\relax\catcode`\^^M=12 %
\@firstofone{%
  \endgroup
  \NewDocumentCommand\inputfiles{mv}{%
     \begingroup
     \let\do\@makeother\dospecials\do\^^M\do\^^I%
     %\endlinechar=`\^^M\relax %<- This is the default, thus this line is turned into a comment.
     \def\temp{\inputfileloop{}{#2}}%
     \expandafter\futurelet\expandafter\tempb\expandafter\temp\@@input "#1" \relax^^M%
  }%
  \@ifdefinable\inputfileloop{%
    % recursively in #1 accumulate a list of \input-commands
    %   #1 list of \input-commands gathered so far
    %   #2 directory
    %   #3 filename or \relax-token to process in this iteration.
    \long\def\inputfileloop#1#2#3^^M{%
      \ifx\relax#3\expandafter\@secondoftwo\else\expandafter\@firstoftwo\fi
      % Use this for inputting:
      %{\def\temp{\inputfileloop{#1\input{#2#3}}{#2}}\futurelet\tempb\temp}{\endgroup#1}%
      % Use this for message on terminal instead of inputting:
      {\def\temp{\inputfileloop{#1\message{I would \string\input{#2#3}^^J}}{#2}}\futurelet\tempb\temp}{\endgroup#1}%
    }%
  }%
}%

\begin{document}

\inputfiles{\jobname_inputfiles.txt}{section/}

\end{document}

Messages on terminal are:

I would \input{section/00-introduction.tex}
I would \input{section/10-data.tex}
I would \input{section/20-analysis.tex}
I would \input{section/30-conclusions.tex}
I would \input{section/40-weird}%}  {^#^^61^^!.tex}

With the last message the argument of the \input-command is:
section/section/40-weird}%}␣␣{^#^^61^^!.tex
( denotes a space character.)

The token-sequence forming that argument is tokenized under verbatim-category-code-régime, so

  • several spaces don't collapse into a single one.
  • ^^-notation is disabled, i.e., ^^61^^! is not tokenized as aa
  • { and } are not treated as braces for scoping/denoting arguments but are treaated as ordinary characters.
  • # is treated as an ordinary character which can be used without doubling within arguments of macros which define scratch-macros from their arguments.
  • % is treated as an ordinary character

It just occurred to me that you might like to crank out empty lines when the file holding the list of input-files is processed:

% Let's within the current directory/folder create the file holding
% the list of input-files:

\begin{filecontents*}{\jobname_inputfiles.txt}
00-introduction.tex
10-data.tex
20-analysis.tex
30-conclusions.tex

mysubsection/40-bla.tex
50-weird}%}  {^#^^61^^!.tex
\end{filecontents*}


\documentclass{article}

\begingroup
\makeatletter\endlinechar=-1\relax\catcode`\^^M=12 %
\@firstofone{%
  \endgroup
  \NewDocumentCommand\inputfiles{mv}{%
     \begingroup
     \let\do\@makeother\dospecials\do\^^M\do\^^I%
     %\endlinechar=`\^^M\relax %<- This is the default, thus this line is turned into a comment.
     \def\temp{\inputfileloop{}{#2}}%
     \expandafter\futurelet\expandafter\tempb\expandafter\temp\@@input "#1" \relax^^M%
  }%
  \@ifdefinable\inputfileloop{%
    % recursively in #1 accumulate a list of \input-commands
    %   #1 list of \input-commands gathered so far
    %   #2 directory;  slash etc needs to be included so you can leave it empty if you like.
    %   #3 filename or \relax-token to process in this iteration.
    \long\def\inputfileloop#1#2#3^^M{%
      \ifx\relax#3\expandafter\@secondoftwo\else\expandafter\@firstoftwo\fi
      {%
        \def\temp{%
          \ifcat$\detokenize{#3}$\expandafter\@gobble\else\expandafter\inputfileappendtolist\fi
          {#3}\inputfileloop{#1}{#2}%
        }%
        \futurelet\tempb\temp
      }{\endgroup#1}%
    }%
  }%
  \newcommand\inputfileappendtolist[4]{%
    #2{#3%
      % Use this for inputting:
      %\input{#4#1}%
      % Use this for message on terminal instead of inputting:
      \message{I would \string\input{#4#1}^^J}%
    }{#4}%
  }%
}

\begin{document}

\inputfiles{\jobname_inputfiles.txt}{mysection/}

\end{document}

Messages on terminal are:

I would \input{mysection/00-introduction.tex}
I would \input{mysection/10-data.tex}
I would \input{mysection/20-analysis.tex}
I would \input{mysection/30-conclusions.tex}
I would \input{mysection/mysubsection/40-bla.tex}
I would \input{mysection/50-weird}%}  {^#^^61^^!.tex}

With the last message the argument of the \input-command is:
mysection/50-weird}%}␣␣{^#^^61^^!.tex
( denotes a space character.)

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .