“Missing \endcsname inserted” error after \expandafter chain

Question

Solved

Thanks to @PhelypeOleinik for the solution! And thanks to everyone else who responded so quickly to provide assistance. For anyone who finds this post in the future, the error really had nothing to do with ConTeXt, so the explanation below will be useful even if you use LaTeX or plain TeX.

Original Post

I am creating a module in ConTeXt that is primarily accessed through a command \setup<name>[] (for some generic structure named <name>). Apart from this and a few other commands intended for the end-user, I have been adding a namespace prefix to all my internal commands, but due to how I want this command to operate, I chose not to use ConTeXt's built-in \installcommandhandler.

Nonetheless, I have almost finished the entire thing except for one incredibly irritating namespace-related bug that I haven't been able to work out. At one point, one of my commands calls \processcommacommand[<list>]<command> where both <list> and <command> are in \csname style, due to namespace prefixing, and both need to be expanded with \expandafter chains. I've isolated the problematic section and changed command names below:

\unprotect

\installnamespace{nms}
%\def\????nms{@@@@nms} % For testing only
% A simpler namespace creator could just run e.g. \def\????nms{@@@@nms}, but since in
% practice the actual namespace installer won't determine the prefix string until runtime,
% I have to define everything as generally as possible below, in terms of
% \csname\????nms<other>\endcsname.

% ... other module code ...

% If #1 consists of <arg1A>,<arg1B>,... separated by commas, and #2 consists of
% <arg2Akey>=<arg2Aval>, <arg2Bkey>=<arg2Bval>,... separated by commas, the
% \<namespace>_processor command below attempts to create a series of commands like so:
% % \def\<namespace>@<tag>@<arg1A>@<arg2Akey>{arg2Aval}
% % \def\<namespace>@<tag>@<arg1B>@<arg2bkey>{arg2Bval}
% % etc.
% ...for some externally defined string \tag.

\expandafter\unexpanded\expandafter\def\csname\????nms _processor\endcsname[#1][#2]{%
    \expandafter\def\csname\????nms _params\endcsname##1{%
        \doifnotempty{##1}{\getparameters[\????nms @\tag @##1@][#2]}
    }
    \expandafter\def\csname\????nms _csvs\endcsname{#1}
    \expandafter\expandafter\expandafter\processcommacommand\expandafter\expandafter
        \expandafter[\expandafter\csname\????nms _csvs\endcsname\expandafter]%
        \csname\????nms _params\endcsname
    % The above should simplify to
    % % \processcommacommand[\<namespace>_csvs]\<namespace>_params,
    % ...but in practice it throws a "Missing \endcsname inserted" error.
}

% ... other module code ...

\protect
\endinput

However, if I set the namespace at the top to be defined by \def\????nms{@@@@nms} instead and replace the problematic line, which begins with the 3 \expandafter calls, with direct references to the namespace prefix (e.g. \processcommacommand[\@@@@nms_csvs]\@@@@nms_params), the entire module works just fine, which is baffling to me, since this shouldn't be any different in theory. Am I using \expandafter incorrectly? I tried rewriting that particular line a few times and it came out the exact same way, so I can't tell what mistakes, if any, I'm making there.

Answer 1

As I said in the comments, I'm the Jon Snow of ConTeXt, so this answer is about the \expandafter chain.

The first \expandafters you used are correct. The last one is the problem:

\expandafter\expandafter\expandafter\processcommacommand\expandafter\expandafter
    \expandafter[\expandafter\csname\????nms _csvs\endcsname\expandafter]
    \csname\????nms _params\endcsname

Let's walk through it. The first chain expands the highlighted tokens:

So the first chain expanded onlt \????nms and stopped, so we are left with:

\expandafter\processcommacommand\expandafter
    [\csname<\????nms>_csvs\endcsname\expandafter]
    \csname\????nms _params\endcsname

where <\????nms> denotes one expansion of \????nms. Let's say that \????nms expands to NMS, just for the example.

Then, the second chain:

expands the \csname NMS_csvs\endcsname and stops, now we're left with:

\processcommacommand
    [\NMS_csvs\expandafter]
    \csname\????nms _params\endcsname

And the next thing executed is \processcommacommand with some strange arguments. Not what we wanted.

A correct \expandafter chain would be:

\expandafter\processcommacommand\expandafter
  [\csname\????nms _csvs\expandafter\endcsname\expandafter]%
   \csname\????nms _params\endcsname

but why?

Firstly, you don't need to expand things inside a \csname...\endcsname. The \csname primitive performs full expansion of everything until the next \endcsname, so:

\def\my{MY}
\def\cmd{COMMAND}
\expandafter\def\csname\my\cmd\endcsname{Hello}
\MYCOMMAND

works with only one \expandafter.

So in your code you only need to expand both \csnames, and they will expand \????nms when needed.

Second (and consequence of the first), is that you can "continue" an \expandafter chain from inside a \csname (it performs full expansion, remember?). So \csname\expandafter\endcsname\hello will expand \hello before the \csname.

Now, writing it down. We start with:

\processcommacommand
  [\csname\????nms _csvs\endcsname]%
   \csname\????nms _params\endcsname

First, we expand the first \csname. Only two \expandafters are needed:

\expandafter\processcommacommand\expandafter
  [\csname\????nms _csvs\endcsname]%
   \csname\????nms _params\endcsname

the first will skip \processcommacommand, and the second will skip [. Now we're in a full expansion context inside the \csname. We can exploit this to expand the next \csname by inserting one \expandafter right before \endcsname and continuing the chain from here:

\expandafter\processcommacommand\expandafter
  [\csname\????nms _csvs\expandafter\endcsname\expandafter]%
   \csname\????nms _params\endcsname

the first \expandafter will skip the \endcsname, and the next will skip the ].