# Modify contents of environment using l3regex and then write to file

I want to define an environment (in a class file, as it happens) the contents of which are both displayed on-screen as they are written, but they are also modified (e.g. using the l3regex package) and then written to a file.

My specific requirement is that I want a solution environment (in which the answers to a question are given), but the solution has a mark scheme given by a series of \marks{...} commands. The version that is written to file needs to have these \marks{...} parts removed.

The following mwe code, based on this post, correctly carries out the regex replacement that I need in the context required:

\documentclass{article}

\usepackage{l3regex}
\usepackage{environ}
\usepackage{marginnote}

\renewcommand\marks[1]{\marginnote{[#1]}}

\ExplSyntaxOn
\NewEnviron{solution}
{
\strip_marks:V \BODY
}

\tl_new:N \solution_text_tl

\cs_new_protected:Nn \strip_marks:n
{
\tl_set:Nn \solution_text_tl { #1 }
\solution_text_tl  % Inserts original text in the right place
\regex_replace_all:nnN { \c{marks}\cB\{ [0-9]* \cE\} } {   } \solution_text_tl
\regex_replace_all:nnN { \c{marks} [0-9] } {   } \solution_text_tl
%\solution_text_tl <-- ***Regex done, how to write to file?***
}
\cs_generate_variant:Nn \strip_marks:n { V }
\ExplSyntaxOff

\begin{document}

\begin{solution}
This line has a \emph{mark} at the end of it.\marks{10}
\end{solution}

\end{document}


As written, this code returns an error (related to marginnote, but really it's because in writing to file, it has expanded the \emph{...}). If I remove the \emph{...} inside the solution environment, then the code compiles as expected. However, I really need to preserve the control sequences inside the solution environment when writing to file. I've tried, e.g., the answers package (or some other solution like this, all relying on verbatim to some extent), but then how do I do the regex replacement that I want?

• Please provide a complete example which can be compiled to reproduce the situation or problem you are asking about.
– cfr
Apr 17, 2016 at 22:10
• Probably generate \iow_now:NV and then use that ? Hard to tell without code to play with. (Well, one of the experts might not need the code, of course.) But I take it that you don't want it completely unexpanded. Or do you really want literally \solution_text_tl written to the file?
– cfr
Apr 17, 2016 at 22:19
• Hi, I've expanded my example into a MWE. I don't want literally to write \solution_text_tl (I can do that!), but rather I want to write the \BODY of the environment unexpanded, after carrying out some regex replacement. Apr 17, 2016 at 22:30
• Thanks. I'm certainly no expert and need code to play with!
– cfr
Apr 17, 2016 at 22:56
• Don't redefine \marks, which is a TeX primitive (to be precise, e-TeX). Apr 17, 2016 at 23:01

Is this what you want?

Output to file (I've used \jobname-soln.tex):

This line has a \emph {mark} at the end of it.


Output to PDF:

Note that if your command sequence takes arguments, the signature needs to be specified appropriately and the parameters given:

\cs_new_protected:Npn \strip_marks:n #1


Otherwise, you cannot pass an argument to \strip_marks:n. With just\cs_new_protected:Nn, you could define \strip_marks: but that can't take an argument and you can't then use #1 in the definition.

Also, token lists etc. should begin either \l_ or \g_ depending on whether local or global in order to conform to the L3 syntax specifications. I've therefore changed \solution_text_tl to \l_solution_text_tl. Streams are supposed to be declared only as global according to the documentation, so I've used \g_ when naming a stream.

% !TEX TS-program = pdflatex
% !TEX encoding = UTF-8 Unicode
% arara: pdflatex
\pdfminorversion=7
% \listfiles
\documentclass{article}

\usepackage{l3regex}
\usepackage{environ}
\usepackage{marginnote}

\newcommand*\mymarks[1]{\marginnote{[#1]}}

\ExplSyntaxOn
\NewEnviron{solution}
{
\strip_marks:V \BODY
}

\tl_new:N \l_solution_text_tl
\iow_new:N \g_solution_stream

\cs_new_protected:Npn \strip_marks:n #1
{
\tl_set:Nn \l_solution_text_tl { #1 }
\l_solution_text_tl  % Inserts original text in the right place
\regex_replace_all:nnN { \c{mymarks}\cB\{ [0-9]* \cE\} } {   } \l_solution_text_tl
\regex_replace_all:nnN { \c{mymarks} [0-9] } {   } \l_solution_text_tl
%   \l_solution_text_tl <-- ***Regex done, how to write to file?***
\iow_now:NV \g_solution_stream \l_solution_text_tl
}
\AtBeginDocument{
\iow_open:Nn \g_solution_stream { \jobname-soln.tex }
}
\AtEndDocument{
\iow_close:N \g_solution_stream
}
\cs_generate_variant:Nn \strip_marks:n { V }
\cs_generate_variant:Nn \iow_now:Nn { NV }
\ExplSyntaxOff

\begin{document}
\begin{solution}
This line has a \emph{mark} at the end of it.\mymarks{10}
\end{solution}
\end{document}


# EDIT

You can see that \BODY includes spaces for the newlines and \par for the blank lines by added \show \BODY to the definition of \mymarks:

\NewEnviron{solution}
{
\show \BODY
\strip_marks:V \BODY
}


displays the contents of \BODY on the console before passing it to \strip_marks:V:

> \BODY=macro:
->This line has a \emph {mark} at the end of it.\mymarks {10} \par This line fo
llows a paragraph break.\mymarks {90} That was a line break..
\env@solution@process ->\show \BODY
\strip_marks:V \BODY \env@ignore
l.49 \end{solution}


where the contents of solution was

\begin{solution}
This line has a \emph{mark} at the end of it.\mymarks{10}

This line follows a paragraph break.\mymarks{90}
That was a line break.
\end{solution}

• The iow_open:Nn instruction should be \AtBeginDocument and \iow_close:N should be \AtEndDocument Apr 17, 2016 at 23:18
• @egreg Oops. Thank you. I always forget that TeX does (the equivalent of) > rather than >> as I'd naïvely expect.
– cfr
Apr 17, 2016 at 23:24
• That's looking a lot better than I've managed so far. Is there a way to preserve line breaks in the tex source? Obviously a double-line-break gets replaced by a \par, but I'd like the output to \jobname-soln.tex to be as readable as possible. Apr 18, 2016 at 9:54
• As a follow-up, if (after inserting some carriage returns inside the solution environment) I add the l3tl-analysis package, and then insert \tl_show_analysis:N { \l_solution_text_tl } before and/or after the regex replacements, is see that the line breaks have already disappeared from the token list: have they already got removed/parsed by \BODY? Apr 18, 2016 at 15:03
• @RobertBrignall When TeX tokenizes the input, end-of-lines become spaces, but if two end-of-lines are found only separated by blank spaces, a \par is generated. Characters of category code 5 never get in the mouth of TeX, so they'll never appear in the replacement text for \BODY. Apr 18, 2016 at 22:30