# How do I examine a token?

... let me count the ways.
I can test it with \if and \ifx
and \show its \meaning

And there I stop, both in my knowledge and my poetic ability (though some might argue that the latter never started).

Background: I'm still trying to get my head around the statement of what \noexpand does:

\noexpand temporarily sets the next token to \relax.

The purpose of setting it to \relax seems to be that since \relax is not expandable, the next token does not get expanded the next time that TeX tries to expand it. But it does get expanded the time after that, so that seems to be the meaning of "temporarily".

But is the fact that it is temporarily set to \relax detectable? To test this, I want to examine what TeX thinks \noexpand\foo is, but that "temporarily" keeps getting in the way: it's all a bit quantum since the act of observing a \noexpand\foo seems to change it in some way.

Anyway, so I want to build a test suite to see if the value of \noexpand\foo is detectable in a significant way and for that, I need to know all the ways that I can get at its value, such as conditionals and \show. And that's my question.

Question: What ways are there of examining a macro or active token in TeX which reveal something about what it points/expands to?

-
As no assignment ever takes place, I think your out of luck: \tracingall just shows you '{\noexpand}'. –  Joseph Wright Mar 16 '11 at 10:59
@Joseph: Well, I may be out of luck on my motivating question, but please note that that wasn't my actual question! –  Loop Space Mar 16 '11 at 11:11
@Andrew: Well, \meaning shows you the meaning of a token, but it will never show you the result of \noexpand because the later is a primitive acting on a token, rather than an assignment. –  Joseph Wright Mar 16 '11 at 11:17
As soon as you pass a \notexpanded: \foo as an argument, it becomes \foo. Also, you can only test it once, so I guess that your best bet is \expandafter\ifx\noexpand\foo\yourtokenwhichisnotatoken. –  Bruno Le Floch Mar 16 '11 at 11:21
@Joseph, @Bruno: Stop trying to answer the not-question! I tried hard to make a general question out of my specific case of interest. If you want to discuss the background further, may I suggest we move the discussion to my room! Here, I wanted to get a list of things like \if and so on which can detect the value of a token. –  Loop Space Mar 16 '11 at 12:04

Hoping that I understand the question, the options are:

• \show, which displays the meaning of a token in the terminal. For registers, this shows the register number rather than the content of the register. Primitives give you an identity, for example \show = \show, whereas for macros you get the expansion.
• \showthe, which displays the content of a register at the terminal.
• \meaning, which places very similar information in the input stream, with category codes for all tokens 'other' except for spaces. This is useful for both displaying the meaning of macros and for various tests and tokenization tricks. For example, LaTeX uses \meaning to sanitize category codes (although nowadays \scantokens might be a better choice.)
• \ifx, which does a comparison based on the meaning of two tokens. The two do not have to be \let to one another, but do have to be identically-defined for the test to be true. So something like

\def\testa{stuff}
\def\testb{stuff}
\ifx\testa\testb


is true but

\def\testa{stuff}
\long\def\testb{stuff}
\ifx\testa\testb


is false.

• \if, which tests if two unexpandable tokens are identical, ignoring category codes. The two tokens business means that

\def\test{aa}
\if\test\SomeOtherThing


will compare the two a characters here, and that \SomeOtherThing is part of the true branch.

-
you can add \ifcat\foo\bar and \ifcat\noexpand\foo\bar. And after \catcode\~=13\let~*, * and ~ will have the same \string, \meaning, will be equal under \ifx, \if, and \ifcat, also under \ifcat\noexpand, but somehow, ted.sty distinguishes them. –  Bruno Le Floch Mar 16 '11 at 14:56
@Andrew: Sorry, the code above is garbled. Assume that ~ has catcode 13 (active). Then do \let~*, and \show~. This will show ~=the character *. Now, both * and ~ have the same \meaning, the character *. Also, \if~* is true, \ifcat~* is true, \ifx~* is true, and \ifcat\noexpand~* is also true. If on top of that we had used an active * instead of an active ~, then \string would be the same for the character * and the active token that was \let` to it. Essentially, they are indistinguishable. –  Bruno Le Floch Mar 21 '11 at 12:11