\newtoks token seems to have local scope

Question

I want to collect the first line of a quotation to use later. I have used \newtoks for a similar purpose so tried it here, this way:

\documentclass[12pt] {article} 

\newtoks{\qfirstlineinternal}
\newcommand{\firstline}[1]{%
  \qfirstlineinternal=\expandafter{#1}
 #1
}

\begin{document}

\qfirstlineinternal=\expandafter{\the\qfirstlineinternal initial value}

debug: before quotation -\the\qfirstlineinternal-

\begin{quotation}
debug: in quotation before: -\the\qfirstlineinternal-

\firstline{This is the first line of a quotation,}

debug: in quotation after: -\the\qfirstlineinternal-

which continues on more lines.
\end{quotation}

debug: after quotation: -\the\qfirstlineinternal-
\end{document}

Here's the output document, showing that the value of the token set inside the quotation environment is lost when that environment ends:

The quotation environment seems to have created a new local instance of my token. How can I prevent that?

Answer 1

Assignments in TeX are by default local to the group in which they are performed. Setting the value of a register is an assignment, as are macro definitions and other assorted actions.

All LaTeX environment (well, almost all, but the exceptions are irrelevant here) form a group, so if one wants that an assignment to a register survives when the group ends, the assignment has to be marked as \global.

An assignment to the token register \qfirstlineinternal takes the form

\qfirstlineinternal={<something>}

and in order to make it global it's sufficient to say

\global\qfirstlineinternal={<something>}

The \expandafter that is frequently seen in this context is used to add to the contents of the token register:

\global\qfirstlineinternal=\expandafter{\the\qfirstlineinternal<something>}

will append <something> to the previous value of the token register, because the assignment is performed after \expandafter has caused the expansion of \the, which in turn delivers the contents of \qfirstlineinternal. So the \expandafter is by no means necessary: its presence depends on what the register is supposed to contain.

The same considerations hold for registers allocated with \newcount, \newdimen or \newskip. Notice that the allocations themselves are global, but this has no influence on value assignments to the registers.

To the contrary, when a LaTeX counter is defined with \newcounter{foo}, value assignments to foo performed with \stepcounter{foo}, \refstepcounter{foo}, \setcounter{foo}{<number>} or \addtocounter{foo}{<number>} are always global (it would otherwise be quite difficult to implement assigning the number to a caption, which might be inside a minipage nested in a figure environment).

Answer 2

Modifications to a token inside a group is specific (or local) to that group (your group being the quotation environment). Prepending this assignment with \global breaks the barrier:

\documentclass[12pt]{article}
\newtoks{\qfirstlineinternal}
\newcommand{\firstline}[1]{%
  \global\qfirstlineinternal={#1}%
  #1%
}
\begin{document}
%...
\end{document}

Note the use of % to avoid spurious spaces in your output.