Verbatim text in token registers

Question

How can I add verbatim text in a token register (I want to accumulate pieces of code in a toks register and print them in an appendix). In the MWE below if you substitute verbatim text it fails.

\documentclass{article}
\begin{document}
  \newtoks\test
  \test={one }
  \test=\expandafter{\the\test test}
  \the\test
\end{document}

Answer 1

EDIT: as we added a verbatim functionality to the xparse package, manipulating verbatim text becomes easier. For this to work, you will need a version of xparse later than 15/08/2011. I used token lists instead of toks, but that's mostly irrelevant.

\documentclass{article}
\usepackage{xparse}
\ExplSyntaxOn
\NewDocumentCommand{\addverb}{m+v}
  { \tl_put_right:Nn #1 { #2 } }
\NewDocumentCommand{\useverb}{m}
  {
    \exp_args:Nx \scantokens
      { \token_to_str:N \verb + \tl_to_str:N #1 + }
  }
\tl_new:N \test
\ExplSyntaxOff
\begin{document}
  \addverb\test+%&#a  b c\+
  \addverb\test?+-*?
  \useverb\test
\end{document}

Answer 2

The problem is that \verb operates by making the first character active and then using that to end the group that restores all of the font changes and category code changes it made.

\documentclass{article}
\begin{document}
  \newtoks\test
  \test={\verb~one~ }
  \test=\expandafter{\the\test test}
  \the\test
\end{document}

This works since ~ is already active. However, this isn't very useful because the main use of \verb is to avoid having to escape things like % or &. That isn't going to work here since these get tokenized when you put them in a token register. Instead, you should just use \texttt.

Answer 3

I would write the code to an external file instead and reread it again when you need it. It avoids this issues and is much faster if you add a lot of material multiple times. Something based on filecontents or VerbatimOut of fancyvrb, but which allows to have multiple environments which all write to the same file:

---

Code:

% codewrite.sty
\RequirePackage{fancyvrb}

\newcommand*\CodeOut{\FV@Environment{}{CodeOut}}

\newwrite\@codewrite
% Or to save an output handle:
%\let\@codewrite\FV@OutFile

\newcommand*\CodeOpenFile[1]{%
    \immediate\openout\@codewrite #1\relax
    \def\CodeOpen@File{#1}%
}

\def\FVB@CodeOut{%
    \@bsphack
    \begingroup
    \FV@UseKeyValues
    \FV@DefineWhiteSpace
    \def\FV@Space{\space}%
    \FV@DefineTabOut
    \def\FV@ProcessLine{\immediate\write\@codewrite}%
    \let\FV@FontScanPrep\relax
    %% DG/SR modification begin - May. 18, 1998 (to avoid problems with ligatures)
    \let\@noligs\relax
    %% DG/SR modification end
    \FV@Scan
}
\def\FVE@CodeOut{\endgroup\@esphack}

\newcommand*\ReadCode{\immediate\closeout\@codewrite\input{\CodeOpen@File}}


\DefineVerbatimEnvironment{CodeOut}{CodeOut}{}

---

Usage Example:

\documentclass{article}

\usepackage{codewrite}

\CodeOpenFile{\jobname.cod}

\begin{CodeOut}
   Test
\end{CodeOut}

\begin{document}

\begin{CodeOut}
\begin{verbatim}
    test%^&$%\relax
\end{verbatim}
\end{CodeOut}

\ReadCode

\end{document}

There might be a package which does this already. Note that you can catch the content of a file to a macro using the catchfile package.