TeX capacity exceeded with several \input in a spreadtab

Question

I use several \input in a spreadtab using the method from this answer in my document until I got this error:

TeX capacity exceeded, sorry [text input levels=15].

I read in the Internet that I can manage the problem by changing the this parameter from max_in_open = 15 to max_in_open = 30.

The problem I don't know where I can find the data to make the changes. I use TeXStudio on my Mac.

I am looking for the settings to make my document work.

\documentclass{article}
\usepackage[a4paper,left=2.5cm,right=2.5cm,bottom=2.8cm,top=2.8cm]{geometry} 
\usepackage{longtable,tabu}
\usepackage[T1]{fontenc}
\usepackage[ngerman]{babel}
\usepackage{datetime}
\usepackage{siunitx}
\usepackage{multirow}
\usepackage[sc]{mathpazo}
\usepackage{datatool}
\usepackage{spreadtab,booktabs,xpatch}
\usepackage[]{numprint}
\usepackage[]{eurosym}
\usepackage{fp}
\usepackage{booktabs}
\usepackage{tabularx,siunitx}
\usepackage{numprint}
}%=========================



\makeatletter
\def\spreadtab@ii{\IfSubStr\ST@tab{\noexpand\input}{\expandafter\spreadtab@iii\ST@tab\@nil}\relax}
\def\spreadtab@iii#1\input#2#3\@nil{%
    \long\def\spreadtab@iv##1\spreadtab@iv{\endgroup\def\ST@tab{#1##1#3}\spreadtab@ii}%
    \begingroup
    \everyeof{\spreadtab@iv\noexpand}%
    \expandafter\spreadtab@iv\@@input#2
}
\xpretocmd\spreadtab@i\spreadtab@ii{}{}
\makeatother    

\subsection*{B}
    \begin{spreadtab}{{tabular}{@{\hskip1pt}r@{\hskip7pt}p{2.7cm}crrr}}
    \toprule
    &@ Ku& @ Re & @ Ne & @ Me & @ Br \\
        \midrule
        \input{Bu1.txt}
        \input{Bu2.txt}
        \input{Bu3.txt}
        \input{Bu4.txt}
        \input{Bu5.txt}
        \input{Bu6.txt}
        \input{Bu7.txt}
        \input{Bu8.txt}
        \input{Bu9.txt}
        \input{Bu10.txt}
        \input{Bu11.txt}
        \input{Bu12.txt}
        \input{Bu13.txt}
        \input{Bu14.txt}
        \input{Bu15.txt}
        \midrule
        &@ To & @\thefo &(sum(d2:[0,-1]))tag(BN) & sum(e2:[0,-1])tag(BM)& sum(f2:[0,-1])tag(BB)\\
        \bottomrule
    \end{spreadtab}

Sample content of Bu1.txt:

@\thefoo&@BU1& @513& 270.00& 551.30& 3231.30\\

The other *.txt look similar.

Answer 1

As David pointed out in the comment, the code \everyeof{\spreadtab@iv\noexpand} \expandafter\spreadtab@iv\@@input#2 is causing every \input to start yet another \input before finishing, which effectively nests inputs as far as TeX is concerned.

I don't really know spreadtab, so I don't have a good solution. Instead, here is a very, very, very, very, very, very, very, VERY bad hack to make your document work. The new spreadtabwithinput environment will replace every \input{<file>} command in its body by the contents of <file>, and then pass everything back to spreadtab. You can have arbitrarily many \input in there.

Proceed at your own risk.

% \newwrite\wrt
% \def\x#1;{\immediate\openout\wrt"Bu#1.txt"
%   \immediate\write\wrt{@\noexpand\thefoo
%     &@BU#1& @513& 270.00& 551.30& 3231.30\noexpand\\}%
%   \immediate\closeout\wrt
%   \ifnum#1<15 \expandafter\x\number\numexpr#1+1\expandafter;\fi}\x1;

\documentclass{article}
\usepackage{spreadtab,booktabs}

% HACK % HACK % HACK % HACK % HACK % HACK % HACK % HACK % HACK % HACK %
\usepackage{xparse}
\ExplSyntaxOn
\tl_new:N \l__ihpro_spreadtab_hack_tl
\tl_new:N \l__ihpro_tmpa_tl
\NewDocumentEnvironment { spreadtabwithinput } { +b }
  {
    \tl_clear:N \l__ihpro_spreadtab_hack_tl
    \__ihpro_spreadtab_loop:w \prg_do_nothing: #1
      \input \q_recursion_tail \q_recursion_stop
    \exp_last_unbraced:Nno
    \begin{spreadtab}
      \l__ihpro_spreadtab_hack_tl
    \end{spreadtab}
  } { }
\cs_new_protected:Npn \__ihpro_spreadtab_loop:w #1 \input #2
  {
    \tl_put_right:No \l__ihpro_spreadtab_hack_tl {#1}
    \quark_if_recursion_tail_stop:n {#2}
    \file_get:nnNF {#2} { } \l__ihpro_tmpa_tl
      {
        \msg_error:nnn { ihpro } { file-not-found } {#2}
        \tl_clear:N \l__ihpro_tmpa_tl
      }
    \exp_last_unbraced:NNo
    \__ihpro_spreadtab_loop:w \prg_do_nothing: \l__ihpro_tmpa_tl
  }
\msg_new:nnn { ihpro } { file-not-found } { File~'#1'~not~found. }
\ExplSyntaxOff
% HACK % HACK % HACK % HACK % HACK % HACK % HACK % HACK % HACK % HACK %

\newcounter{foo}

\begin{document}
\subsection*{B}
    \begin{spreadtabwithinput}{{tabular}{@{\hskip1pt}r@{\hskip7pt}p{2.7cm}crrr}}
    \toprule
    &@ Ku& @ Re & @ Ne & @ Me & @ Br \\
        \midrule
        \input{Bu1.txt}
        \input{Bu2.txt}
        \input{Bu3.txt}
        \input{Bu4.txt}
        \input{Bu5.txt}
        \input{Bu6.txt}
        \input{Bu7.txt}
        \input{Bu8.txt}
        \input{Bu9.txt}
        \input{Bu10.txt}
        \input{Bu11.txt}
        \input{Bu12.txt}
        \input{Bu13.txt}
        \input{Bu14.txt}
        \input{Bu15.txt}
        \midrule
        &@ To & @\thefoo &(sum(d2:[0,-1]))tag(BN) & sum(e2:[0,-1])tag(BM)& sum(f2:[0,-1])tag(BB)\\
        \bottomrule
    \end{spreadtabwithinput}
\end{document}

Answer 2

Off the cuff Heiko Oberdiek's package catchfile comes to my mind.

You can use macros of this package for defining macros from the content of a file.

I can offer a routine

\CatchFilesForScantokens{⟨comma-separated file list⟩}{⟨preamble code⟩}{⟨postamble code⟩}

which reads and collects the ⟨preamble code⟩ and the content of each file of the ⟨comma-separated file list⟩ and the ⟨postamble code⟩ in verbatim-catcode-regime and then passes collected/accumulated things to \scantokens so that normal catcode-régime is in effect when the verbatimized material gets re-tokenized.

You can use this routine as

\CatchFilesForScantokens{⟨comma-separated file list⟩}%
                        {⟨stuff for initiating spreadtab-environment}%
                        {⟨stuff for ending the spreadtab-environment⟩}

(Internally with each file of the ⟨comma-separated file list⟩ the routine uses the \CatchFileDef-macro of the package catchfile for (re)defining a scratch-macro A from the content of that file, tokenized in verbatim-catcode/category-12-régime, and then appends the expansion of that scratch-macro A to the definition of another scratch-macro B. ⟨preamble code⟩ and ⟨postamble code⟩ also go to that other scratch-macro B. Thus everything is accumulated in that other scratch-macro B whose expansion can be passed to \scantokens for re-tokenization under normal catcode-régime.

I am lazy, so I use expl3's function \clist_map_inline:nn for parsing the ⟨comma-separated file list⟩).

Here is the example:

% Create  15 MyBu<X>.txt-files
% ==========================
\begin{filecontents*}{MyBu1.txt}
[0,-1]+1&@BU1& @513& 270.00& 551.30& 3231.30\\
\end{filecontents*}
\begin{filecontents*}{MyBu2.txt}
[0,-1]+1&@BU2& @513& 270.00& 551.30& 3231.30\\
\end{filecontents*}
\begin{filecontents*}{MyBu3.txt}
[0,-1]+1&@BU3& @513& 270.00& 551.30& 3231.30\\
\end{filecontents*}
\begin{filecontents*}{MyBu4.txt}
[0,-1]+1&@BU4& @513& 270.00& 551.30& 3231.30\\
\end{filecontents*}
\begin{filecontents*}{MyBu5.txt}
[0,-1]+1&@BU5& @513& 270.00& 551.30& 3231.30\\
\end{filecontents*}
\begin{filecontents*}{MyBu6.txt}
[0,-1]+1&@BU6& @513& 270.00& 551.30& 3231.30\\
\end{filecontents*}
\begin{filecontents*}{MyBu7.txt}
[0,-1]+1&@BU7& @513& 270.00& 551.30& 3231.30\\
\end{filecontents*}
\begin{filecontents*}{MyBu8.txt}
[0,-1]+1&@BU8& @513& 270.00& 551.30& 3231.30\\
\end{filecontents*}
\begin{filecontents*}{MyBu9.txt}
[0,-1]+1&@BU9& @513& 270.00& 551.30& 3231.30\\
\end{filecontents*}
\begin{filecontents*}{MyBu10.txt}
[0,-1]+1&@BU10& @513& 270.00& 551.30& 3231.30\\
\end{filecontents*}
\begin{filecontents*}{MyBu11.txt}
[0,-1]+1&@BU11& @513& 270.00& 551.30& 3231.30\\
\end{filecontents*}
\begin{filecontents*}{MyBu12.txt}
[0,-1]+1&@BU12& @513& 270.00& 551.30& 3231.30\\
\end{filecontents*}
\begin{filecontents*}{MyBu13.txt}
[0,-1]+1&@BU13& @513& 270.00& 551.30& 3231.30\\
\end{filecontents*}
\begin{filecontents*}{MyBu14.txt}
[0,-1]+1&@BU14& @513& 270.00& 551.30& 3231.30\\
\end{filecontents*}
\begin{filecontents*}{MyBu15.txt}
[0,-1]+1&@BU15& @513& 270.00& 551.30& 3231.30\\
\end{filecontents*}


% ==========================================================================================
% Define macro 
%
%   \CatchFilesForScantokens{<comma-separated file list>}{<preamble code>}{<postamble code>}
%
% to read and collect <preamble-code> and each file of <comma-separated file list> and 
% <postamble code> in verbatim-catcode-regime and to pass collected things to \scantokens:
% ==========================================================================================
\RequirePackage{catchfile}
\newcommand\MyScratchmacroA{}%
\newcommand\MyScratchmacroB{}%

% Within a local scope temporarily change the catcode-régime:
\begingroup
%
\makeatletter
\ExplSyntaxOn
%
% We won't need \makeatother / \ExplSyntaxOff as -- like the other changes to the
% catcode-régime -- the effects of  \makeatletter and \ExplSyntaxOn will be gone
% when the local scope is closed by \@firstofone.
%
% Use the SOH(=Start Of Heading)-character, code-point-number 1 in ASCII, accessible
% as ^^A in TeX's ^^-notation, for commenting:
\catcode`\^^A=14 %
% Make the CR(=Carriage-Return)-character, code-point-number 13 in ASCII, accessible
% as ^^M in TeX's ^^-notation, an ordinary character --- every line must end by something
% that is taken for a comment-character by TeX as long as this setting is in effect :
\catcode`\^^M=12\relax%
% Make % an ordinary character:
\catcode`\%=12\relax^^A
^^A
\@firstofone{^^A
  ^^A the first thing \@firstofone shall do is close the local scope where the
  ^^A catcode-régime is changed:
  \endgroup^^A
  ^^A=============================================================================
  ^^A PARAPHERNALIA:
  ^^A \UD@firstoftwo, \UD@secondoftwo, \UD@stopromannumeral, \UD@CheckWhetherNull,
  ^^A=============================================================================
  \newcommand\UD@firstoftwo[2]{#1}^^A
  \newcommand\UD@secondoftwo[2]{#2}^^A
  \@ifdefinable\UD@stopromannumeral{\chardef\UD@stopromannumeral=`\^^00}^^A
  ^^A-----------------------------------------------------------------------------
  ^^A Check whether argument is empty:
  ^^A.............................................................................
  ^^A \UD@CheckWhetherNull{<Argument which is to be checked>}%
  ^^A                     {<Tokens to be delivered in case that argument
  ^^A                       which is to be checked is empty>}%
  ^^A                     {<Tokens to be delivered in case that argument
  ^^A                       which is to be checked is not empty>}%
  ^^A
  ^^A The gist of this macro comes from Robert R. Schneck's \ifempty-macro:
  ^^A <https://groups.google.com/forum/#!original/comp.text.tex/kuOEIQIrElc/lUg37FmhA74J>
  \newcommand\UD@CheckWhetherNull[1]{^^A
    \romannumeral\expandafter\UD@secondoftwo\string{\expandafter^^A
    \UD@secondoftwo\expandafter{\expandafter{\string#1}\expandafter^^A
    \UD@secondoftwo\string}\expandafter\UD@firstoftwo\expandafter{\expandafter^^A
    \UD@secondoftwo\string}\expandafter\UD@stopromannumeral\UD@secondoftwo}{^^A
    \expandafter\UD@stopromannumeral\UD@firstoftwo}^^A
  }^^A
  ^^A-----------------------------------------------------------------------------
  ^^A Check whether_verbatimized_ argument has a trailing explicit 
  ^^A <carriage-return>-character-token of catcode 12(other):
  ^^A-----------------------------------------------------------------------------
  ^^A \UD@CheckWhetherTrailingCarriageReturn{<Argument which is to be checked>}%
  ^^A                                      {<Tokens to be delivered in case
  ^^A                                       <argument which is to be checked>'s
  ^^A                                       last token is an explicit <carriage-
  ^^A                                       return>-character-token of 
  ^^A                                       catcode 12(other)>}%
  ^^A                                      {<Tokens to be delivered in case
  ^^A                                       <argument which is to be checked>'s
  ^^A                                       last token is not an explicit
  ^^A                                       <carriage-return>-character-token of 
  ^^A                                       catcode 12(other)>}%
  \newcommand\UD@CheckWhetherTrailingCarriageReturn[1]{^^A
    \UD@@CheckWhetherTrailingCarriageReturn#1\UD@SelDom^^M\UD@SelDom\UD@@SelDom^^A
  }^^A
  \@ifdefinable\UD@@CheckWhetherTrailingCarriageReturn{^^A
    \long\def\UD@@CheckWhetherTrailingCarriageReturn#1^^M\UD@SelDom#2\UD@@SelDom{^^A
      \UD@CheckWhetherNull{#2}{\UD@secondoftwo}{\UD@firstoftwo}^^A
    }^^A
  }^^A
  ^^A=============================================================================
  \NewDocumentCommand\CatchFilesForScantokens{m}{^^A
    \begingroup^^A
    \let\do\@makeother^^A
    \do\^^I^^A
    \CatchFilesForScantokensInner{#1}^^A
  }^^A
  \newcommand\verbatimregime{^^A
    \let\do\@makeother^^A
    \dospecials^^A
    \do\^^I^^A
    \do\^^M^^A
    \endlinechar=`\^^M\relax^^A
    \newlinechar=\endlinechar^^A
  }^^A
  \NewDocumentCommand\CatchFilesForScantokensInner{m+v+v}{^^A
    \endgroup^^A
    \begingroup^^A
    \edef\MyScratchmacroB{^^A
      \expandafter\UD@CheckWhetherTrailingCarriageReturn\expandafter{\detokenize{#2}}^^A
        {\unexpanded\expandafter{\detokenize{#2}}}^^A
        {\unexpanded\expandafter{\detokenize{#2^^M}}}^^A
    }^^A
    \clist_map_inline:nn{#1}{^^A
      ^^A\message{##1^^J}^^A
      \CatchFileDef{\MyScratchmacroA}{##1}{\verbatimregime}^^A
      \edef\MyScratchmacroB{^^A
        \unexpanded\expandafter\expandafter\expandafter{\expandafter\MyScratchmacroB\detokenize\expandafter{\MyScratchmacroA}}^^A
      }^^A
    }^^A
    \edef\MyScratchmacroB{^^A
      \unexpanded\expandafter\expandafter\expandafter{\expandafter\MyScratchmacroB\detokenize{#3}%}^^A
    }^^A
    \newlinechar=\endlinechar^^A
    ^^A=======================================================================
    ^^A If you want to see on connsole what \scantokens gets as spreadtab,
    ^^A then enable the next but one line by removing the leading ^^A
    ^^A=======================================================================
    ^^A\show\MyScratchmacroB^^A
    \scantokens\expandafter{\expandafter\endgroup\MyScratchmacroB}^^A
  }^^A
}%


% Create  document:
% =================
\documentclass{article}
\usepackage[a4paper,left=2.5cm,right=2.5cm,bottom=2.8cm,top=2.8cm]{geometry} 
\usepackage{longtable,tabu}
\usepackage[T1]{fontenc}
\usepackage[ngerman]{babel}
\usepackage{datetime}
\usepackage{siunitx}
\usepackage{multirow}
\usepackage[sc]{mathpazo}
\usepackage{datatool}
\usepackage{spreadtab,booktabs,xpatch}
\usepackage[]{numprint}
\usepackage[]{eurosym}
\usepackage{fp}
\usepackage{booktabs}
\usepackage{tabularx,siunitx}
\usepackage{numprint}

\begin{document}

\CatchFilesForScantokens{%
  MyBu1.txt, 
  MyBu2.txt,
  MyBu3.txt,
  MyBu4.txt, 
  MyBu5.txt,
  MyBu6.txt,
  MyBu7.txt, 
  MyBu8.txt, 
  MyBu9.txt, 
  MyBu10.txt,
  MyBu11.txt,
  MyBu12.txt, 
  MyBu13.txt, 
  MyBu14.txt, 
  MyBu15.txt
}%
{%
  \begin{spreadtab}{{tabular}{@{\hskip1pt }r@{\hskip7pt }p{2.7cm }crrr}}
  \toprule
  &@ Ku& @ Re & @ Ne & @ Me & @ Br \\ % <- This is row 1.
  \midrule
  \SThiderow 0&&&&&\\ % <- This is row 2 and it is invisible and used for initial values.
}%
{%
  \midrule
  &@To & [-2,-1] &(sum(d3:[0,-1]))tag(BN) & sum(e3:[0,-1])tag(BM)& sum(f3:[0,-1])tag(BB)\\
  \bottomrule
  \end{spreadtab}
}%

\end{document}

!!! This approach is based on reading/tokenizing files in verbatim-category-code-régime. Hereby \input-commands occurring within some of the files of the ⟨comma-separated file list⟩ will not be carried out. I.e., \input-nesting is not possible. All you can do is provide a list of files. !!!

!!! Also it is relied on reading and tokenizing the ⟨preamble code⟩ and the ⟨postamble code⟩ in verbatim-category-code-régime. This in turn implies that \CatchFilesForScantokens

• cannot be hidden inside the definition of whatsoever macro and
• cannot be used inside macro-arguments, i.e.,
• must be used in ways where it is ensured that the arguments will be obtained by reading and tokenizing things from the .tex-input-file while the changes of the category-code-régime temporarily introduced by \CatchFilesForScantokens are in effect.

!!!

Answer 3

On many TeX distributions there exists a file called texmf.cnf. It is located typically in texmf/web2c and contains the mentioned parameter. Usually it is not recommended to mess with this kind of configuration files. However, if your TeX supports this option it can be set on the command line with the switch --cnf-line='max_in_open=30', e.g.:

lualatex --cnf-line='max_in_open=30' some-file.tex

Many editors support special command lines for projects or single TeX files. E.g. GNU Emacs supports to add a file local variable named TeX-command-extra-options: "--cnf-line='max_in_open=30'" so that the end of your TeX file could look like:

% mode: latex
% TeX-master: t
% TeX-command-extra-options: "--cnf-line='max_in_open=30'"
% End: