# Get macro name from a peeked token

I have a question concerning the decomposition of macro definitions:

In chapter XV, section 7 in the interface3 manual is a list of possible decompositions:

For example, there is a command \token_get_prefix_spec:N listed in the interface3 manual. As described, it leaves the prefixes in the input-stream.

A token defined by \cs_new_protected:Nn \someToken:n { do~something } would thereby leave "\protected".

The other two, \token_get_arg_spec:N and \token_get_replacement_spec:N, leave the raw arguments or the replacement. All three of them work well with a \l_peek_token as "input".

But is there a method to leave the token name? The \token_to_str:N macro is not an option, as i want to know the (once expanded) name of a \l_peek_token: \token_to_str:N \l_peek_token just returns "\l_peek_token" and not "\someToken:n" (if the peeked at token is "\someToken:n").

Here is a small example showing what I want (compile with XeTeX):

\documentclass[varwidth,border=10pt]{standalone}
\usepackage{expl3}
\usepackage{newunicodechar}
\ExplSyntaxOn

\cs_new_protected:Nn \someToken:n {
\peek_catcode:NF ## {
char: #1\\
-~Meaning~of~next~token:~\token_to_meaning:N \l_peek_token\\
-~Prefix~of~next~token:~\token_get_prefix_spec:N \l_peek_token\\
-~Name~of~next~token:~\token_to_str:N \l_peek_token\\ % this is not working as wanted, I want \someToken:n !
}
}

\newunicodechar{➀}{\someToken:n{a}}
\newunicodechar{➁}{\someToken:n{b}}
\ExplSyntaxOff
\begin{document}
➁➀
…
\end{document}


(This question came up while I was searching for solutions for my other questions Compare macro names instead of meaning and Peek ahead and process characters.)

• That is not possible I guess, because \l_peek_token is just \let to the next token, i.e. it is assigned the same \meaning but the original “name” is lost. Oct 22, 2017 at 11:52
• @HenriMenke But how can \token_to_meaning:N \l_peek_token return the meaning (which includes the macro name: "\protected macro:->\someToken:n {a}") if it's lost? Could I somehow parse the meaning in such a way, that everything between ">" and "{" is "returned"? Oct 22, 2017 at 11:55
• That's not what's happening: ➁ is made active, and searches its next token; the next token is ➀, now that's the info you loose when you use peek functions, you no longer see the “➀” but have something like \let\peektoken➀; the info that's inside ➀ is what you now have, and inside it you have \sometoken:n {a}. It's not clear to me what you want to do with that? Oct 22, 2017 at 13:22
• I want to peek and remove the following token, if the token name of the "info inside" is \someToken:n. The use case is mentioned in the link below the source code: unicode replacement of composite-emoji (like flags or skin color variations). Other methods (also mentioned in my other questions/link below) don't work in some cases. Oct 22, 2017 at 13:49

If you want to extract the first token inside the definition of a macro (this needs to be an user defined macro and not a primitive, otherwise there might be situations in which it gives an error).

\ExplSyntaxOn

\cs_new:Npn \fjs_string_first_delimit_q_nil:Nw #1 #2 \q_nil { \token_to_str:N #1 }

\cs_new_protected:Nn \fjs_sometoken:n
{
\peek_catcode:NF ##
{
char: ~ #1 \\
Meaning ~ of ~ next ~ token: ~ \token_to_meaning:N \l_peek_token \\
\token_if_macro:NT \l_peek_token
{
First ~ token ~ inside ~ the ~ next ~ token: ~
\exp_after:wN \fjs_string_first_delimit_q_nil:Nw \l_peek_token aaaaaaaaa \q_nil
}
\par
}
}

\newunicodechar{➀}{\fjs_sometoken:n{a}}
\newunicodechar{➁}{\fjs_sometoken:n{b}}

\ExplSyntaxOff


This is not entirely safe (e.g., \def\foo{{}} will cause problems if the next token is \foo), but for now this works well in your cases ➀➁.

• Thanks, that seems like exactly what I was looking for. I'm trying to understand your code and have a few questions: What's the purpose of the "aaaaaaaaa"? It seems I can just delete it and nothing changes… What's the purpose of the \q_nil? I've read the section in interface3.pdf but can't make sense of it in your function \fjs_string_first_delimit_q_nil:Nw. In \exp_after:wN it's a placeholder to be not expanded, isn't it?! Oct 22, 2017 at 14:55
• Just in case the next token needs to grab arguments so that it doesn't fail and has 9 possible arguments to grab in the first expansion. This works with delimited arguments, it takes the first token and gobbles all the rest until \q_nil. Oct 22, 2017 at 15:35

@HenriMenke (unintentionally?) gave me the idea to parse a substring of the \meaning to check if \l_peek_token is "\someToken:n". With some explanations from this answer I got the following working code.

\documentclass[varwidth,border=10pt]{standalone}
\usepackage{expl3}
\usepackage{newunicodechar}
\ExplSyntaxOn

\tl_new:N \l_my_tl
\seq_new:N \l_my_seq

\cs_generate_variant:Nn \regex_extract_once:nnNTF { nV }

\cs_new_protected:Nn \someToken:n {
\peek_catcode:NF ## {
char: #1\\
-~Meaning~of~next~token:~\token_to_meaning:N \l_peek_token\\
\tl_set:Nx \l_my_tl { \token_to_meaning:N \l_peek_token }
\regex_extract_once:nVNTF { macro:->\\someToken:n } \l_my_tl \l_my_seq {
-~Real~name~of~next~token:~\textbackslash someToken:n
}{
-~Real~name~of~next~token:~unknown
}\\
}
}

\newunicodechar{➀}{\someToken:n{a}}
\newunicodechar{➁}{\someToken:n{b}}
\begin{document}
➁➀
…
\end{document}


This solution seems like a hack. Unless someone tells me that there is no better solution, I can't mark this as an accepted answer :-(

• "Real name of the next token" is ➀, \sometoken:n {a} is the meaning of that token. Oct 22, 2017 at 13:23
• @Manuel That's the point. I do not want the meaning, because it contains the parameter {a}. When I check for the meaning, ➁➀ would match, but ➀➁ wouldn't. But I want to match every kind of following \someToken:n regardless of the complete meaning. In that sense \someToken:n is the "Real name of the next token" and ➀ is only a symbol. My goals become more apparent in the linked questions at the end of this question. Oct 22, 2017 at 13:55
• It's not. May be the token that will end up eventually expanding after the current one, but in the same sense \sometoken will expand to \peek_catcode and \peek_catcode wil expand to something else. So, I do understand what you mean, and tried to answer that in the answer, but the real name of the next token is ➀ :P Oct 22, 2017 at 14:37
• You're right. But I'm just a self-tought amateur and not very fluent in the exact "vocabulary". Thanks for your answer :-) Oct 22, 2017 at 14:56