# Extract from a token containing a list with math content

I have a token which contains a list consisting of:

1. a date in YYYY-MM-DD format,
2. followed by a colon, and
3. some text which may contain math. The text will consist of a sentence or two (i.e., no paragraphs).

I want to process this list and extract the date and content separately.

Sadly, my current MWE works perfectly fine and I am not able to reproduce the intermittent error that I am getting in my actual use case. The error occurs only where there is math content in the list (but other list elements with math are processed just fine). The MWE yields:

The problems I am seeing in my actual use case are:

1. For the case where there is a curly brace group (i.e., text containing a comma) I am getting an empty string as the date with \StrBefore{\ListMember}{:}[\Date].

2. For the case where there is $\deg(e^x)$ in the text I am getting:

TeX capacity exceeded, sorry [input stack size=5000].

\reserved@a ->\def \reserved@a

*{\let \@xs@assign \@xs@expand@and@detokenize...

==> Fatal error occurred, no output PDF file produced!

So, my question is, is there inherent problems in the way I am processing this list? Is there a better way to do the list processing and extraction (but still using a token register).

I will continue to attempt to reproduce a proper MWE, but was hoping that someone would have an "Aha!" moment and see the potential flaw in how I am doing things.

## Notes:

• In hindsight, for the case where the text contains a comma, I think it would have been better to place the brace group around the text only.

## Code:

\documentclass{article}
\usepackage{xstring}
\usepackage{tikz}
\usepackage{xcolor}

\newtoks\MyTodoList

% Test cases:
%   1. only text
%   2. ends with math
%   3. Math, but not at the end.
%   4. Brace group required in case there is a comma in the content
%   5. Contains a comma and math.
\MyTodoList={
2014-03-10: Go to class.,
2014-03-11: Solve $\sin x + \cos x = 0.5$.,
2014-03-12: Figure out $\deg(e^x)$ again.,
{2014-03-13: Study for Math, Physics and Geometry.},
{2014-03-14: Prepare for $\pi$ day, and eat pie.},
}

\newcommand*{\Date}{}%
\newcommand*{\TodoItem}{}%
\newcommand*{\MyTodoListExpanded}{}%

\begin{document}

\edef\MyTodoListExpanded{\the\MyTodoList}%
\foreach \ListMember in \MyTodoListExpanded {%
\IfStrEq{\ListMember}{}{}{% Filter out any empty rows (case of trailing comma on last item.
\StrBefore{\ListMember}{:}[\Date]%
\StrBehind{\ListMember}{:}[\TodoItem]%
\par\textcolor{red}{\Date}: \textcolor{blue}{\TodoItem}%
}%
}

\end{document}

-

Package xstring uses \fullexpandarg as default. That means, the strings are processed by \edef. This is fatal, if the string contains fragile stuff. Also \edef does not even respect LaTeX's \protect mechanisms.

Because the strings for the use cases of the xstring macros are given in macros, e.g.: \StrBefore{\ListMember}, \expandarg is the solution, because it only expands the first token once.

\foreach ... {%
\expandarg
\IfStrEq{\ListMember}{...


See also the documentation, section "3.1.1 The commands \fullexpandarg, \expandarg and \noexpandarg"

-
Still haven't figured out how to get my actual use case to work, but why is \expandarg not required in the MWE I provided -- seems to not have any effect if I have \expandarg or if I don't. – Peter Grill Apr 30 '14 at 6:23
@PeterGrill \fullexpandarg is on by default, which does more than \expandarg – egreg Apr 30 '14 at 9:57
@PeterGrill: Add a fragile command or something like \def\foobar{xy} to the text, then \fullexpandarg puts this in \edef that will complain, that \foobar is not defined. – Heiko Oberdiek Apr 30 '14 at 11:02