Understanding token expansion

Question

First of all, note that this question is to be regarded as a learning question as I am not looking for a package (I know very well the xparse package).

I am attempting to write a macro which has the following usage:

\dsee*{text}[optarg]

My code is thus the following:

\def\md@err{*}
\def\@hmno[#1]{%
  \textsubscript{#1}}
\newcommand{\dsee}{\@ifstar\@@dsee\@dsee}
\def\@@dsee#1{%
  \md@err#1\@ifnextchar[\@hmno\relax}
\def\@dsee#1{%
  #1\@ifnextchar[\@hmno\relax}

So far it works as intended: \dsee{test1}, \dsee{test2}[opt] becomes

I then add some formatting to the macros.

\def\md@styl@foreign{%
   \color[HTML]{0000AA}%
   \rmfamily\mdseries\itshape}
% The new defs for \@@dsee and \@dsee
\def\@@dsee#1{%
   \begingroup%
     \md@styl@foreign\md@err#1\@ifnextchar[\@hmno\relax%]
   \endgroup}
\def\@dsee#1{%
   \begingroup%
     \md@styl@foreign#1\@ifnextchar[\@hmno\relax%]
   \endgroup}

Then my output ends as: . As I understand it, \@ifnextchar macro checks whether the following token is equivalent to its first argument, in my case [. However, since changing the definition of the macros, \@ifnextchar does not match with [ as, apparently, it is no longer the next token. I suspect that I need some \expandafter before my \endgroup.

My question is thus:

Is there a way to determine which token is next in the stream, or is there a way to determine where expansion has to be delayed?

Maybe using the \tracing... options?

Any pointers would be greatly appreciated.

Answer 1

Please always post complete documents showing all required packages.

You don't need tracing to see the next token, it is explicit in the source \endgroup so you just need to move it into the two branches.

\documentclass{article}
\usepackage{fixltx2e,color}
\begin{document}

\makeatletter
\def\md@err{*}
\def\@hmno[#1]{%
  \endgroup
  \textsubscript{#1}}
\newcommand{\dsee}{\@ifstar\@@dsee\@dsee}



\def\md@styl@foreign{%
   \color{red}%
   \rmfamily\mdseries\itshape}
% The new defs for \@@dsee and \@dsee
\def\@@dsee#1{%
   \begingroup%
     \md@styl@foreign\md@err#1\@ifnextchar[\@hmno\endgroup%]
  }
\def\@dsee#1{%
   \begingroup%
     \md@styl@foreign#1\@ifnextchar[\@hmno\endgroup%]
   }

So far it works as intended: \dsee{test1}, \dsee{test2}[opt] becomes First test

I then add some formatting to the macros.

\end{document}

---

Note that this isn't really about expansion \@ifnextchar does not work by expansion and includes non-expandable operations such as assignments in its definition. It takes the two arguments specifying the true and false case and then skips over white space and tests the following token, so it is almost always only used as the last thing in a macro definition otherwise no test is needed as the following token is explicit and so it will always take the same branch, as for example in your original code where the first non-white space token after the two arguments was \endgroup so it would always take the false branch.