# comment package and macro .sty file causing problems, why?

Here's main.tex:

\documentclass{article}
\usepackage[utf8]{inputenc}

\usepackage{comment}

\usepackage{tcolorbox}
\tcbuselibrary{theorems}

\newtcbtheorem[]{myprob}{Problem}%
{colframe=blue!35!black,enlarge top by=0.15cm,fonttitle=\slshape,breakable}{prob}

\includecomment{analysis}
\includecomment{solution}

\usepackage{ana}

\title{Testing Doc}
\author{}
\date{}

\begin{document}

\maketitle

\begin{analysis}
\section{Analysis}
\subsection{Basic Integration}
\csname ANAsp2015p1\endcsname
\end{analysis}

\end{document}


and ana.sty:

\ProvidesPackage{ana}[2021/08/11 Analysis Problems]

\expandafter\newcommand\csname ANAsp2015p1\endcsname{
\begin{myprob}{Spring 2015 Problem 1}{}
blah
\end{myprob}

\begin{solution}
Proof:
\end{solution}
}


Overleaf tells me "! File ended while scanning use of \next", and "I suspect you have forgotten a '}', causing me to read past where you wanted me to stop". If I delete the \begin{solution}...\end{solution} stuff from ana.sty:

\ProvidesPackage{ana}[2021/08/11 Analysis Problems]

\expandafter\newcommand\csname ANAsp2015p1\endcsname{
\begin{myprob}{Spring 2015 Problem 1}{}
blah
\end{myprob}
}


the code runs fine. Originally I thought it had something to do with me nesting the comment environment \begin{solution}...\end{solution} inside another comment environment \begin{analysis}...\end{analysis}, but the error remained after I got rid of \begin{analysis} and \end{analysis}. Does anyone know what's going on?

I have looked at comment-package and macro definitions but no one explained why this error happens.

• I believe it's a \catcode issue. By wrapping the solution environment in a macro, \catcodes are frozen, while comment relies on \catcode changes to work. Aug 12, 2021 at 7:16

As campa pointed out in a comment, it is a catcode-issue:

Iirc the environment solution expects to detect its end by finding tokens coming from tokenizing the phrase \end{solution} under verbatim-catcode-régime. In your scenario, however, that phrase is not tokenized under verbatim-catcode-régime but it got tokenized under usual catcode-régime already at the time of tokenizing the definition of \ANAsp2015p1.

You can circumvent that issue by having the tokens that shall form the definition of \ANAsp2015p1 read and tokenized under verbatim-catcode-régime and defining \ANAsp2015p1 to pass these tokens to \scantokens for retokenization.

The following example provides a command \DefineProbSolCmd which does these things. Like \verb or verbatim-environment etc, the command \DefineProbSolCmd can only be used in situations where it obtains its argument directly by reading and tokenizing from a .tex-input-file. The command \DefineProbSolCmd cannot be used as component of a macro-argument or of a definition-text of a macro or of the content of a token-register, etc.

\ProvidesPackage{ana}[2021/08/12 Analysis Problems]
\RequirePackage{xparse}

\NewDocumentCommand\DefineProbSolCmd{}{%
\begingroup
\catcode\^^I=12\relax % Give "horizontal tab" catcode 12. See comment below for the reason.
\InnerDefineProbSolCmd
}%
\begingroup
\catcode\^^A=14
\catcode\%=12
\@firstofone{^^A
\endgroup
\NewDocumentCommand\InnerDefineProbSolCmd{m+v}{^^A
\endgroup
\expandafter\newcommand\csname#1\endcsname{\begingroup\newlinechar=\endlinechar\scantokens{\endgroup#2%}}^^A
}^^A
}%
% Reason for changing the catcode of horizontal tab:
% xparse's processor of v/+v-arguments leaves the catcode of
% "horizontal tab" at 10(space) so that tokenizing horizontal tabs
% yields space-tokens, which is undesired at this stage of
% processing.
%
\DefineProbSolCmd{ANAsp2015p1}{%
\begin{myprob}{Spring 2015 Problem 1}{}
blah
\end{myprob}

\begin{solution}
Proof:
\end{solution}
}%

\documentclass{article}
\usepackage[utf8]{inputenc}

\usepackage{comment}

\usepackage{tcolorbox}
\tcbuselibrary{theorems}
\tcbuselibrary{breakable}

\newtcbtheorem[]{myprob}{Problem}%
{colframe=blue!35!black,enlarge top by=0.15cm,fonttitle=\slshape,breakable}{prob}

\includecomment{analysis}
\includecomment{solution}

\usepackage{ana}

\title{Testing Doc}
\author{}
\date{}

\begin{document}

\maketitle

\begin{analysis}
\section{Analysis}
\subsection{Basic Integration}
\csname ANAsp2015p1\endcsname
\end{analysis}

\end{document}


An environment "solution" is in use. If I had to give assignments in a class, I would never make a pretty sheet with the solutions. To minimize the chance of it getting into the hands of course participants and circulating. When discussing solutions, I prefer blackboard writing, if only because writing on the board regulates the speed at which information is conveyed so that it is better absorbed.

• +1 Help my memory, what's ^^A again? Aug 12, 2021 at 17:17
• @campa ^^A is ^^-notation for the character whose code-point in TeX's internal character-representation-scheme has the number <code-point-number of A> - 64 = 1. Such number is also called "character code". TeX's internal character-representation-scheme in turn with traditional TeX-engines is ASCII and with LuaTeX and XeTeX is unicode whereof ASCII is a strict subset. So ^^A denotes the character whose code-point-number in ASCII has the value "1", i.e., the SOH-character=start of heading character. It is not a printable character. In my example it is temporarily used as comment-character. Aug 12, 2021 at 17:26
• @campa I use it temporarily as comment-char so that the usual comment-character % can be assigned catcode 12. That in turn is necessary to get it at the end of the <balanced text> of the \scantokens-directive. Otherwise the endline-character inserted by \scantokens` when doing the reading-back-part and encountering the end of the "line" might get processed, yielding a space-token and thus undesired horizontal glue in some situations. Aug 12, 2021 at 17:34