# How are tokenized arguments of a macros?

In the following pdftex example we make verbatim environment inside "...", i.e., all special characters get catcode 12. The problem is with {:

\def\setverb{\def\do##1{\catcode##1=12}\dospecials}

\catcode\"=\active
\def"#1"{%
\leavevmode\hbox{%
\setverb
\tt
\endlinechar=-1 \scantokens{#1}%
}%
}

hello "{" world
\bye


Why it gives this error?

(./test.tex
Runaway argument?
{" world
! Forbidden control sequence found while scanning use of ".
<inserted text>
\par
<to be read again>
\bye
l.13 \bye

?


Should not it tokenize arguments until it encounters the second "?

How to make \def"#1"{...} work in all cases?

• Well braces can still group arguments, e.g. \def"#1#2"{arg1: #1 arg2: #2} and "{abc}d" will give arg1: abc arg2: d. So braces must be balanced. Check the definition of \verb in latex.ltx for a solution. – Ulrike Fischer Aug 10 '16 at 7:48

## 2 Answers

When TeX finds ", it looks for the next " at the same group level, which it doesn't find, because { still has its usual category code 1. Since " is not \long, the first \par token triggers the error.

You have to split processing in two parts: first set the category codes, then absorb the argument.

The parameter text for a macro can define both delimited or undelimited arguments, but, in any case, TeX keeps track of <left brace> and <right brace> (meaning explicit character tokens of category code 1 and 2 respectively).

So, for instance,

\def\foo (#1,#2){#1-#2}


defines a macro that must be followed by (12, with the first argument delimited by ,12 and the second argument delimited by )12.

However, the call

\foo({(a,b)},c)


would absorb #1 as {(a,b)} (and the braces would be removed), and #2 as c.

In your case, only balanced braces would take you home.

I'd not use \scantokens and not even make " scan for an argument at all but set the appropriate catcodes in a group and let " to \endgroup (also works with Knuth TeX).

\def\setverb{\def\do##1{\catcode##1=12}\dospecials}

\catcode\"=\active
\def"{\begingroup\setverb\tt\let"=\endgroup}

hello "{}%~&^_$#\" world \bye  If you want to have it in a box, use some two-stage method. \def\setverb{\def\do##1{\catcode##1=12}\dospecials} \catcode\"=\active \def"{\begingroup\setverb\tt\doverb} \def\doverb#1"{\hbox{#1}\endgroup} hello "{}%~&^_$#\" world
\bye


Knuth uses something similar in manmac.tex.

\chardef\other=12
\def\ttverbatim{\begingroup
\catcode\\=\other
\catcode\{=\other
\catcode\}=\other
\catcode\\$=\other
\catcode\&=\other
\catcode\#=\other
\catcode\%=\other
\catcode\~=\other
\catcode\_=\other
\catcode\^=\other
\obeyspaces \obeylines \tt}

\catcode\|=\active
{\obeylines \gdef|{\ttverbatim \spaceskip\ttglue \let^^M=\  \let|=\endgroup}}

• Which is a simplified version of what \verb does in LaTeX. – egreg Aug 10 '16 at 8:46
• The reason that I use \scantokens is to be able to use active quotes in arguments to a macro. See this question tex.stackexchange.com/questions/317879/how-is-catcode-handled Your solution will not work in such case. – Igor Liferenko Aug 10 '16 at 8:49
• @IgorLiferenko Okay, but then you can't use it to typeset verbatim content in the argument of a macro. I think that's just not possible. The unmatched brace is kind of harmless, but try forwarding %`. – Henri Menke Aug 10 '16 at 8:55
• @IgorLiferenko You could typeset the verbatim material at global scope and save it in a box. Then you can pass the box to a macro without having to worry. – Henri Menke Aug 10 '16 at 8:56