# Understanding token expansion

First of all, note that this question is to be regarded as a learning question as I am not looking for a package (I know very well the xparse package).

I am attempting to write a macro which has the following usage:

\dsee*{text}[optarg]


My code is thus the following:

\def\md@err{*}
\def\@hmno[#1]{%
\textsubscript{#1}}
\newcommand{\dsee}{\@ifstar\@@dsee\@dsee}
\def\@@dsee#1{%
\md@err#1\@ifnextchar[\@hmno\relax}
\def\@dsee#1{%
#1\@ifnextchar[\@hmno\relax}


So far it works as intended: \dsee{test1}, \dsee{test2}[opt] becomes

I then add some formatting to the macros.

\def\md@styl@foreign{%
\color[HTML]{0000AA}%
\rmfamily\mdseries\itshape}
% The new defs for \@@dsee and \@dsee
\def\@@dsee#1{%
\begingroup%
\md@styl@foreign\md@err#1\@ifnextchar[\@hmno\relax%]
\endgroup}
\def\@dsee#1{%
\begingroup%
\md@styl@foreign#1\@ifnextchar[\@hmno\relax%]
\endgroup}


Then my output ends as: . As I understand it, \@ifnextchar macro checks whether the following token is equivalent to its first argument, in my case [. However, since changing the definition of the macros, \@ifnextchar does not match with [ as, apparently, it is no longer the next token. I suspect that I need some \expandafter before my \endgroup.

My question is thus:

Is there a way to determine which token is next in the stream, or is there a way to determine where expansion has to be delayed?

Maybe using the \tracing... options?

Any pointers would be greatly appreciated.

• You're comparing [ with \endgroup; \@ifnextchar<token>{<true>}{<false>} must go at the very end of the replacement text. – egreg Jun 30 '14 at 8:41
• Tracing won't show what is next in the input per se: it will show what TeX grabs/executes from the input. If you need to work out what tokens are 'coming up, you'll need to make some notes yourself: write down the input then 'expand' your macros one at a time (perhaps using \tracingall to check you and TeX agree!). If you are careful, you'll be able to see what TeX is seeing. – Joseph Wright Jun 30 '14 at 8:50

Please always post complete documents showing all required packages.

You don't need tracing to see the next token, it is explicit in the source \endgroup so you just need to move it into the two branches.

\documentclass{article}
\usepackage{fixltx2e,color}
\begin{document}

\makeatletter
\def\md@err{*}
\def\@hmno[#1]{%
\endgroup
\textsubscript{#1}}
\newcommand{\dsee}{\@ifstar\@@dsee\@dsee}

\def\md@styl@foreign{%
\color{red}%
\rmfamily\mdseries\itshape}
% The new defs for \@@dsee and \@dsee
\def\@@dsee#1{%
\begingroup%
\md@styl@foreign\md@err#1\@ifnextchar[\@hmno\endgroup%]
}
\def\@dsee#1{%
\begingroup%
\md@styl@foreign#1\@ifnextchar[\@hmno\endgroup%]
}

So far it works as intended: \dsee{test1}, \dsee{test2}[opt] becomes First test

I then add some formatting to the macros.

\end{document}


Note that this isn't really about expansion \@ifnextchar does not work by expansion and includes non-expandable operations such as assignments in its definition. It takes the two arguments specifying the true and false case and then skips over white space and tests the following token, so it is almost always only used as the last thing in a macro definition otherwise no test is needed as the following token is explicit and so it will always take the same branch, as for example in your original code where the first non-white space token after the two arguments was \endgroup so it would always take the false branch.

• Thank you, I didn't include the preamble, since it didn't include more than \documentclass and the two packages you listed. – Minustar Jun 30 '14 at 16:30
• @Minustar yes it did, (I changed your color setting as I couldn't be bothered to guess your definitions) and even if it didn't, why make every person offering to help guess those (and not everyone would know that \textsubscript is defined in fixltx2e. It took me at least 4 iterations to make a working example (longer than it took to actually fix the code) – David Carlisle Jun 30 '14 at 16:43