4

In LaTeX3, I could define a function for comparing a token list to a string

\cs_generate_variant:Nn \tl_if_eq:nnTF { V }
\cs_new_nopar:Npn \module_compare:n #1
  {
    \tl_if_eq:VnTF \g_some_tl { #1 } { 1 } { 0 }
  }

But now I want to make this function exhaustively expandable. Are there any other similar functions other than \tl_if_eq:nnTF that I could use?


Edit: to give more context to my problem, I added the following unworking example. (Note that I need to use \ifnum for some reasons.)

\documentclass{article}
\usepackage{expl3}
\ExplSyntaxOn
\tl_set:Nn \g_some_tl {abc}
\cs_generate_variant:Nn \tl_if_eq:nnTF { V }
\cs_new_nopar:Npn \mycompare #1
  {
    \tl_if_eq:VnTF \g_some_tl { #1 } { 1 } { 0 }
  }
\ExplSyntaxOff
\begin{document}
\def\mytest#1{\ifnum \mycompare{#1} > 0 do some\else do other\fi}
\mytest{uvw}
\mytest{abc}
\end{document}

According to Joseph Wright's comments, replacing \tl_if_eq:VnTF with \str_if_eq:VnTF solves my problem.

8
  • 1
    Your title say 'string' but your code is a token list comparison: which one do you want? A string-based test is easy using \str_if_eq_x:nn(TF).
    – Joseph Wright
    Feb 11, 2015 at 9:55
  • \ifx is an expanable primitive for comparing a token lists in TeX.
    – wipet
    Feb 11, 2015 at 10:02
  • @JosephWright Sorry, now I have edited my title.
    – Z.H.
    Feb 11, 2015 at 10:03
  • Could we have a bit more context here? I can think of at least one approach to the problem but it is rather tricky so before I put that down wonder if this is an 'X-Y' problem.
    – Joseph Wright
    Feb 11, 2015 at 10:47
  • @JosephWright I have added a complete example.
    – Z.H.
    Feb 11, 2015 at 11:44

3 Answers 3

5

Token list comparison cannot be expandable, because the only way TeX has for comparing them is to make them the replacement text of macros.

However, all the engines (except for Knuth TeX) implement an expandable “string comparison” that basically works with \detokenize (details are a bit more involved).

With \usepackage{pdftexcmds} you have available \pdf@strcmp (it is called \pdfstrcmp by pdftex, \strcmp by XeTeX and it's emulated with a Lua script in LuaTeX). The code

\pdf@strcmp{<string-A>}{<string-B>}

returns zero if the strings are equal (after expansion, here the details become involved), 1 or –1 otherwise.

This is also available with expl3 as \str_if_eq:nn(TF) (the parentheses mean that one or both the specifiers can be omitted) or \str_if_eq_x:nn(TF). The former doesn't perform expansion on its first two arguments, the latter does.

In your case you could do

\documentclass{article}
\usepackage{xparse}

\ExplSyntaxOn
\DeclareExpandableDocumentCommand{\mytest}{m}
 {
  \zh_mytest:n { #1 }
 }

\cs_new:Npn \zh_mytest:n #1
 {
  \str_if_eq:VnTF \g_zh_fixed_tl { #1 } { 1 } { 0 }
 }
\cs_generate_variant:Nn \str_if_eq:nnTF { V }

\tl_gclear_new:N \g_zh_fixed_tl
\tl_gset:Nn \g_zh_fixed_tl { abc }
\ExplSyntaxOff

\begin{document}
\edef\test{\ifnum\mytest{x}>0 Equal\else Unequal\fi}

\texttt{\meaning\test}

\edef\test{\ifnum\mytest{abc}>0 Equal\else Unequal\fi}

\texttt{\meaning\test}
\end{document}

enter image description here

The usage of \edef is just to show that the macro is fully expandable. However, mixing in this way expl3 code and old style code is not really recommended.

I'd prefer something like

\documentclass{article}
\usepackage{xparse}

\ExplSyntaxOn
\DeclareExpandableDocumentCommand{\mytestTF}{mmm}
 {
  \zh_mytest:nnn { #1 } { #2 } { #3 }
 }

\cs_new:Npn \zh_mytest:nnn #1 #2 #3
 {
  \str_if_eq:VnTF \g_zh_fixed_tl { #1 } { #2 } { #3 }
 }
\cs_generate_variant:Nn \str_if_eq:nnTF { V }

\tl_gclear_new:N \g_zh_fixed_tl
\tl_gset:Nn \g_zh_fixed_tl { abc }
\ExplSyntaxOff

\begin{document}
\edef\test{\mytestTF{x}{Equal}{Unequal}}

\texttt{\meaning\test}

\edef\test{\mytestTF{abc}{Equal}{Unequal}}

\texttt{\meaning\test}
\end{document}
3
  • I was thinking for a token-based version one might go for \str_if_eq:nn(TF) first then using a modified version of the old 'almost OK' string comparison based on inserting multiple \if tests and using \ifx instead. I guess the OP didn't need a token test after all!
    – Joseph Wright
    Feb 11, 2015 at 13:23
  • @JosephWright I could not understand what's the differences between token-based and string-based comparisons? Sorry for misleading you.
    – Z.H.
    Feb 11, 2015 at 13:52
  • @Z.H. That probably constitutes a new question (if we don't already have one on this). Basically, TeX deals with tokens so for example typically text is made up of four 'letters' but with some \catcode changes they might not be. Matching the characters and matching the tokens are not the same thing.
    – Joseph Wright
    Feb 11, 2015 at 13:54
4

You can use the etl package. It can expandably compare token lists. It has a few limits though:

  • it ignores the character code of group begin and group end tokens (normally { and }, but one could also do \catcode`\(=1 \catcode`\)=2 and then use (something) and etl couldn't differentiate that from {something})

  • it can't tell apart an active character which was let to the same character with another category, so after

    \lccode`\~=`\:
    \lowercase{\let~:}
    \catcode`\:=13
    

    And now the package couldn't detect whether : is the active token or one of category 12.

  • spaces are normalised (but this happens at an early stage of TeX's parsing)

  • as Ulrich Diez points out in the comments (and his answer) another issue is if the \escapechar is negative. To give a concise summary what is distinguished by etl: Tokens which have the same meaning and string representation are considered equal.

See the following:

\documentclass[]{article}

\usepackage{etl}

\ExplSyntaxOn
\tl_const:Nn \c_my_cmp_tl { some~ {complex}~ token~ \list }
\cs_generate_variant:Nn \etl_if_eq:nnTF { V }
\NewExpandableDocumentCommand \mycompare { m }
  {
    \etl_if_eq:VnTF \c_my_cmp_tl {#1} { 1 } { 0 }
  }
\ExplSyntaxOff

\begin{document}
% completely wrong
\mycompare{Something wrong}

% missing braces
\mycompare{some complex token list}

% missing braces
\mycompare{some complex token \list}

% last not a macro
\mycompare{some {complex} token list}

% last not a macro
\edef\tmp{\noexpand\mycompare{some {complex} token \string\list}}
\tmp

% this is the only correct one
\mycompare{some {complex} token \list}

% macro parameters are no problem
\mycompare{## macro#1 parameters}

% proving it's fully expandable
\edef\tmp{\mycompare{wrong}}\texttt{\meaning\tmp}

% proving it's fully expandable
\edef\tmp{\mycompare{some {complex} token \list}}\texttt{\meaning\tmp}
\end{document}

enter image description here

0
0

I think an expandable test entirely in TeX, not using Lua, that is reliable to 100% is not feasible—at least not feasible with unicode-TeX-engines like XeTeX or LuaTeX where 1114112 different character-codes are possible.

(I think handling explicit character-tokens of category 1/2 is feasible if you are willing to have a bunch of code just for doing some brace-hacking.)

The hard problems are:

  • Finding an expandable way of, e.g., distinguishing an explicit non-active character token from its active pendant when the active-pendant is let equal to the non-active character token in question.
    E.g.,
    \begingroup\catcode`\Z=13\def\temp{\endgroup\letZ= }\temp Z
    
    How to expandably distinguish active-Z from letter-Z under these conditions without using macros that process delimited arguments?
    (In this scenario applying \string or \meaning either to active-Z or to letter-Z yields the same; comparing active-Z to letter-Z via \ifx/\if/\ifcat yields the true-branch. What else do we have for distinguishing tokens expandably?)
  • Finding an expandable way of, e.g., distinguishing a one-letter-control-sequence whose name equals an explicit character token which it is let equal to while \escapechar is negative.
    E.g.,
    \let\!=!\escapechar=-1
    
    How to expandably distinguish \! from other-! under these conditions without using macros that process delimited arguments?
    (In this scenario applying \string or \meaning either to other-! or to \! yields the same; comparing other-! to \! via \ifx/\if/\ifcat yields the true-branch. What else do we have for distinguishing tokens expandably?)

I introduced the restriction of not using macros that process delimited arguments.

For each possible character-code you could write a macro for detecting by means of delimited arguments whether a token of that character-code is an active character of that character-code/is a one-letter-control-sequence whose name equals the character with the character-code in question/is something else.

But on a unicode-machine like xetex or luatex 1114112 character-codes from 0 to 1114111 are possible, thus 1114112 such macros would be needed. (Defining such macros "on the fly" is not an option as defining contradicts expandability.)

On a traditional 8-bit-engine where only 256 character-codes are possible, one could nowadays probably define 256 macros.


Outline of a mechanism for cranking out active-character and—in case of \escapechar being negative—one-letter-control-sequence via many macros that process delimited arguments—only applicable with 8-bit-engines if applicable at all:

Assuming that you wish to examine a macro-argument where you already know that it is

  • either an explicit character token not of category 1 or 2
  • or an active character token let equal to a pendant not of category 1 or 2
  • or a single-letter-control-sequence let equal to a character-token not of category 1 or 2 whose character-code corresponds to the character forming the name of the single-letter-control-sequence while \esapechar is negative,

i.e., in any case something not of category 1/2 whose stringification yields a single character-token of category 10 or 12

, you can expandably crank out active characters and (in the edge case of \escapechar being negative) single-letter-control-sequences expandably by means of something like

\long\def\tokenfork#1<token1><token2>#2#3\SEP{#2}%
\long\def\forktoken#1{%
  \tokenfork
  #1<token2>{tokens in case #1 = <token1>}%
  <token1>#1{tokens in case #1 = <token2>}%
  <token1><token2>{<tokens in case #1 is s.th. else}%
  \SEP
}

where <token 1> is the active character and <token 2> is the single-letter-control-sequence.

Instead of \tokenfork and \forktoken you choose macro-names wherein the character occurs, e.g. \Afork and \forkA for cranking out active-A and \A via \csname fork\string#1\endcsname{#1} whereby #1 is something whose character-code is A and whose stringification yields the single letter A:

\long\def\firstofthree#1#2#3{#1}%
\long\def\secondofthree#1#2#3{#2}%
\long\def\thirdofthree#1#2#3{#3}%
\long\def\forkdefiner#1{\begingroup\lccode`\X=#1 \lccode`\~=#1 \lowercase{\let~\relax\definefork{X}{~}}}%
\long\def\definefork#1#2{%
  \expandafter\let\csname #1fork\endcsname\relax
  \expandafter\let\csname fork#1\endcsname\relax
  \expandafter\let\csname#1\endcsname\relax
  \expandafter\defineforkB\csname #1fork\expandafter\endcsname
                          \csname fork#1\expandafter\endcsname
                          \csname#1\endcsname{#2}%
}%
\long\def\defineforkB#1#2#3#4{%
  \long\gdef#1##1#3#4##2##3\SEP{##2}%
  \long\gdef#2##1{#1##1#4{\thirdofthree}#3##1{\secondofthree}#3#4{\firstofthree}\SEP}%
  \endgroup
}%
% In a loop define two macros forming forking mechanism for the character-codes 0..255
% that are possible in traditional 8-bit engines:
\newcount\scratchy
\scratchy=-1
\loop
\ifnum\scratchy<255 %
  \advance\scratchy by 1 %
  \forkdefiner{\scratchy}%
\repeat
% Now test with letter-A, one-letter-\A, active-A
\message{%
  ^^J%
  \csname fork\string A\endcsname{A}{neither active char nor one-letter-cs}%
                                    {active char}%
                                    {one-letter-cs}%
}%
{\escapechar=-1 \message{%
  ^^J%
  \csname fork\string\A\endcsname{\A}{neither active char nor one-letter-cs}%
                                     {active char}%
                                     {one-letter-cs}%
}}%
{\catcode`\A=13 \message{%
  ^^J%
  \csname fork\string A\endcsname{A}{neither active char nor one-letter-cs}%
                                    {active char}%
                                    {one-letter-cs}%
}}%
% Now test with other-!, one-letter-\!, active-!
\message{%
  ^^J%
  \csname fork\string!\endcsname{!}{neither active char nor one-letter-cs}%
                                   {active char}%
                                   {one-letter-cs}%
}%
{\escapechar=-1 \message{%
  ^^J%
  \csname fork\string\!\endcsname{\!}{neither active char nor one-letter-cs}%
                                     {active char}%
                                     {one-letter-cs}%
}}%
{\catcode`\!=13 \message{%
  ^^J%
  \csname fork\string!\endcsname{!}{neither active char nor one-letter-cs}%
                                   {active char}%
                                   {one-letter-cs}%
}}%
\bye

Output:

$ pdftex test.tex
This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live 2020) (preloaded format=pdftex)
 restricted \write18 enabled.
entering extended mode
(./test.tex 
neither active char nor one-letter-cs 
one-letter-cs 
active char

neither active char nor one-letter-cs 
one-letter-cs 
active char )
No pages of output.
Transcript written on test.log.

When examining tokens, the nameless control-sequence, producible via \csname\endcsname or via ending a line of .tex-input with an escape-char (backslash) while \endlinechar is negative, might also need special treatment via delimited arguments because applying \string to it yields <escapechar>csname<escapechar>endcsname. You get the same stringification for a control-sequence-token whose name is csname<escapechar>endcsname, producible via \csname csname\string\endcsname\endcsname. There is the very very edge case of these two different tokens having assigned the same meaning so that examining meaning and \string-representation alone is not sufficient for distinguishing them.
There are more weird things, e.g., frozen-\relax (which cannot be redefined) versus normal \relax, ...

6
  • To my knowledge (and also according to Bruno Le Floch) it is impossible to distinguish two tokens in an expandable way if the first is the active character let to the other (except in LuaTeX, where you can do expandable assignments).
    – Skillmon
    Dec 26, 2021 at 21:43
  • The brace hacking wouldn't even be that hard, but it would effectively slow the code down considerably just to catch uninteresting edge cases, is it really that important whether your group is delimited by curly braces or parenthesis?
    – Skillmon
    Dec 26, 2021 at 21:48
  • @Skillmon I just added an outline to my answer of how cranking out active-characters and single-letter-control-sequences while \escapechar is negative could be done by means of delimited arguments on 8-bit engines. || I don't know if cranking out different explicit character-tokens of category 1/2 is important. Neither do I need this in daily life, nor do I know about the purposes the questioner had in mind when asking for such a routine.:-) Dec 26, 2021 at 23:42
  • Well, defining 255 * 13 (or fewer? Maybe only one or two per character suffice) control sequences for this purpose seems a bit over the top, don't you think? :)
    – Skillmon
    Dec 27, 2021 at 16:23
  • @Skillmon This looks like 255 × 2 to me although it can probably be reduced to 255 by changing \forktoken#1 into \fork#1 or \fork#1#2 (talk about how :c and :e argument types in expl3 helps with code readability a lot, although the latter is incompatible with non-eTeX engines)
    – user202729
    Dec 27, 2021 at 17:07

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .