expl3 provides predicates to test whether variables of various data types exist, e.g. \cs_if_exist:NTF, \tl_if_exist:NTF, \str_if_exist:NTF, \int_if_exist:NTF, etc. However, I couldn't find a similar function for the data type token (Part XV in the Interfaces3 documentation). So How can I test whether a token variable exists (for instance, one that was possibly created using \token_new:Nn)?

Here's a use case. Whenever I write LaTeX3 functions that take parameters, first thing I do inside the function's body is name the parameters, i.e. I initialize local variable with the raw parameters #1, #2. After this point I never use the raw parameters again; only the named variables.

So suppose I wish to write a function, \g, that takes a single argument that should be of type "function variable" that may or may not be defined, and then I wish to test whether this variable is defined. So \g's structure would be something like this:

\cs_new:Npn \g #1
    \token_new:Nn \f {#1}
    \token_if_exist:NTF \f {...} {...}

Except \token_if_exist:NTF is not available from expl3.

  • I don't think that's what token is for (I guess there \cs_set_eq:NN and \cs_if_exist:NTF is the logical answer), but I'm not capable of answering with security. In any case, I remember that you have been told that the declarations _new: should not be inside of a function; and I see that you keep writing _new: statements inside of functions, why?
    – Manuel
    Oct 17 '17 at 8:19
  • On the question itself: \token_new:Nn is mainly for saving 'copies' of character tokens. It therefore doesn't have unique 'place' to put tokens. I'm not sure how the result of your proposed \token_if_exist:N(TF) differs from \cs_if_exist:N(TF), other than that presumably the former should allow character tokens.
    – Joseph Wright
    Oct 17 '17 at 8:24
  • @JosephWright: The problem with the \cs... functions is that I cannot assign to a variable of type cs an argument that is a control sequence that may not be defined. So in the above example, I couldn't make \f a variable of type cs.
    – Evan Aad
    Oct 17 '17 at 8:30
  • 1
    I've moved the 'general expl3' comments to chat: I've retained those that seem directly focussed on the question.
    – Joseph Wright
    Oct 17 '17 at 8:42
  • 3
    @EvanAad Are you sure you can't \cs_set_eq:NN \f \whateverundefined?
    – Manuel
    Oct 17 '17 at 8:48

There are not really any 'token variables': the l3token module is a set of low-level functions for 'token stuff', reflecting the fact that TeX works with tokens and so we sometimes need to think at this level.

We cannot create/manipulate for example a letter token in the same way we can a cs token, so it is not really clear what \token_if_exist:N(TF) might do. For example, for a Unicode engine we can have the 'existence' of 1000s of character tokens, but presumably this is not what is to be tested. As such, it is very likely the case that \cs_if_exist:N(TF) will provide here the same useful information.

If you wish to store the exact token passed in a variable, you need to do that using a tl: this is irrespective of the nature of the argument. Thus we might have

\tl_new:N \l__my_tmp_tl
\cs_new_protected:Npn \my_demo:N #1
    \tl_set:Nn \l__my_tmp_tl {#1}

Later, if you want to know if the cs given as #1 is defined, and if you don't want to just use #1, you'll need to get the value back from the `tl

\exp_after:wN \cs_if_exist:NTF \l__my_tmp_tl

Note that as \cs_if_exist:NTF needs the cs 'as is', I've had to use \exp_after:Nw to expand the tl. Personally I'd stick to just using #1 but this is ultimately your call!

Notably, l3token has not to-date (October 2017) been reviewed as carefully as some other parts of expl3. This review work has taken place as use of expl3 has spread and the ideas have therefore had a good 'airing'. It is likely that the team will look over l3token in the near future, and it is quite possible that some functions will be removed/altered.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.