# Errors produces when using \readline

This question, while related to this earlier question of mine, is less about removing content from the beginning of each line and more about how to preprocess lines of a file as it is read in.

Here is the file I want to preprocess:

  The character of the First Caesar has perhaps never been worse
appreciated than by him who in one sense described it best; that is,

\textbf{This is the first line.}\par
*  \textbf{This is the second line.}
*
*
*  Great as Caesar was by the benefit of his original nature, there can be
no doubt that he, like others, owed something to circumstances; and


The basic idea is that somewhere I have a boolean set which determines how to process the lines that begin with *. If the boolean is false, these lines are ignored: as if they had been commented out. If the boolean is true, then these lines are processed as though they did not start with a star. In this MWE I don't have any booleans. This MWE is supposed to represent how things will be processed had the boolean been set true.

MWE:

\documentclass{article}

\makeatletter

\def\@gobblethree#1#2#3{}
\newcommand\myaccumulatedfile{}
\long\def\aegobbleinput#1{%
\IfFileExists{#1}{%
\openin\file #1
\begingroup
\endlinechar\newlinechar
\loop\unless\ifeof\file
%<a>%     \edef\myaccumulatedfile{\myaccumulatedfile\expandafter\@aegobble\foo\@nil}%%
%<b>%     \aftergroup\edef\myaccumulatedfile{\myaccumulatedfile\expandafter\@aegobble\foo\@nil}%%
\repeat
\closein\file
\expandafter\endgroup
\scantokens\expandafter{\myaccumulatedfile}
}{%
\errmessage{File #1' doesn't exist!}%
}%
}

\def\@aegobble{%%'
\@ifnextchar*{\@@aegobble}{\@@nogobble}%%'
}

\def\@@aegobble*#1\@nil{Star}
\def\@@nogobble#1\@nil{#1Major}

\makeatother

\begin{document}

Try:

\aegobbleinput{test_file/star_test}

\end{document}


As written, this MWE will compile, but I've commented out the crucial lines.

If the line starting with %<a>% is uncommented, then I get the fatal error

! TeX capacity exceeded, sorry [input stack size=5000].
\foo ->
The character of the First Caesar has perhaps never been worse
l.7 \aegobbleinput{test_file/star_test}
^^M
!  ==> Fatal error occurred, no output PDF file produced!


If the line starting with %<b>% is uncommented, then I get the non-fatal error

Runaway definition?
\edef \edef \edef \edef \edef \edef \edef \edef
! File ended while scanning definition of \edef.
<inserted text>
}
l.7 \aegobbleinput{test_file/star_test}

?


This last approach seems somewhat correct. At least the produced pdf is approaching something like what I want.

What am I doing wrong here?

UPDATE

Per @jfbu 's suggestion, the following seems to work:

\makeatletter
\newif\ifaeshowall
\aeshowallfalse
\long\gdef\aegobbleinput#1{%%'
\begingroup
\ifaeshowall\catcode\*=9\relax\else\catcode\*=14\relax\fi
\input{#1}\endgroup}
\makeatother

• It the * is reserved for use as the first character of lines to be possibly omitted, why don't you do \catcode\*=14\relax (resp. \catcode\*=9\relax) to let it act as a comment character (resp. be ignored), just before inputting the file via \input (or variants)? – user4686 Jul 30 '13 at 15:55
• @jfbu That's part of my problem.... finding a character that I can reserve for the first character of a line. Though your suggestion of changing the catcode for this character is not one I had thought of. – A.Ellett Jul 30 '13 at 15:58
• @jfbu That works beautifully!!! If you post that as an answer I would accept. – A.Ellett Jul 30 '13 at 16:05
• @jfbu So, probably a better character than * would be " since for the documents I'm working with " is extremely unlikely to show up. – A.Ellett Jul 30 '13 at 16:07

As mentioned in comments, and switching to using " rather than *, or any other character strictly reserved to use in the input files as a label of lines possibly to be omitted, the suggestion is to do something like this:

\def\doublequotecommentsout {\catcode\"=14\relax}
\def\doublequoteisignored   {\catcode\"=9\relax }
\def\makedoublequoteother   {\catcode\"=12\relax }


and then wrap up the \input command, or \InputIfFileExists inside the desired commands.

In some languages, Babel makes " active, and the suitable Babel commands (\shorthandoff,\shorthandon) should be used rather than the above, or rather one should choose another character not playing a rôle in that language as shorthand character.

• Is it possible to assign catcodes to tokens? For example, would it be possible to do something like \catcode\myspecialtoken=14\relax etc.? – A.Ellett Jul 30 '13 at 16:28
• @A.Ellett No, control sequences like \myspecialtoken already have a catcode of sorts (internally for TeX, not one you can alter). You can only set the catcode for a character. – Joseph Wright Jul 30 '13 at 16:32

The problem is tricky to spot as the error isn't very helpful in the first case and gives little away in the second! You have two underlying issues. First, \@ifnextchar is not expandable (cannot be used inside an \edef or similar). There is an assignment (\futurelet) inside \@ifnextchar, and this simply doesn't happen inside \edef. You can define an expandable variant of \@ifnextchar with some limitations (you have to grab one token, so it can go wrong if there is for example 'nothing to see'). The second problem is that \aftergroup inserts exactly one token after the group, not lots of tokens. Thus \aftergroup\edef\myaccumulatedfile inserts one \edef after the group for each line of the file, and then just plonks \myaccumulatedfile into the input 'now'. This approach is not even worth fixing as \foo will be reset after the group anyway, so you will never get out what you are after.