4

While trying to answer another question on this site, I was experimenting with \peek_after:Nw and \l_peek_token.

I can set \l_peek_token OK, but then I want to test if this token is one of several characters, rather than just one character. (Just one character is easy and I could use \peek_meaning:NTF.)

I thought that I would be able to easily convert l_peek_token to a string and then use \str_if_in:nnTF for the test. But, I don't know how to do this.

I know I could use a bunch of nested \token_if_eq_meaning:NNTF, but I find this syntax clumsy.

How can I do this?

MWE

\documentclass{article}
\usepackage{xparse}
\ExplSyntaxOn
\cs_new:Nn \__dcp_test_punct:
  {
    % What goes here to convert \l_peek_token to the right format?
    \str_if_in:nnTF {.,} { \l_peek_token }
      { ~Punctuation~found:~ }
      { ~Punctuation~not~found~ }
    ---
    % This works, but I find the syntax clumsy.
    \token_if_eq_meaning:NNTF \l_peek_token .
      { ~Punctuation~found:~ }
      {
        \token_if_eq_meaning:NNTF \l_peek_token ,
          { ~Punctuation~found:~ }
          { ~Punctuation~not~found~ }
      }
  }
\cs_new:Nn \__dcp_test_punct:n
  {
    \peek_after:Nw \__dcp_test_punct:
  }
\DeclareDocumentCommand {\testpunct} { m }
  {
    \__dcp_test_punct:n {#1}
  }
\ExplSyntaxOff
\begin{document}
\testpunct{A}.

\testpunct{A},

\testpunct{A}
\end{document}
|improve this question
  • \l_peek_token is defined with \futurelet, so in case the next token is a character \l_peek_token is an “implicit character”, which is not the same as a character; in particular, it is not expandable. – egreg Feb 12 at 10:20
  • @egreg, does this mean that it's basically impossible to use the kind of construction I have in \__dcp_test_punct:? (I have now come up with a slightly different way of doing it, that I'm just as happy with.) – David Purton Feb 12 at 10:28
  • You have to do a loop over the characters to check. Be careful that neither \str_if_in:nnTF nor \peek_after:Nw are expandable, so \cs_new_protected:Nn should be used. – egreg Feb 12 at 10:33
  • @egreg, does that mean that in my answer here, I should use \cs_new_protected:Nn \__realscripts_footnote_with_punct: and \cs_new_protected:Nn \__realscripts_footnote_with_punct:n? – David Purton Feb 12 at 10:55
  • Yes: any function that calls unexpandable functions should be _protected – egreg Feb 12 at 11:39
4

Further thinking along with @egreg's comments have led me to do things in a slightly different way and use an inline map. It's not quite as compact as I might have hoped to achieve with \str_if_in:nnTF, but at least it doesn't involve nested boolean checks.

\documentclass{article}
\usepackage{xparse}
\ExplSyntaxOn
\bool_new:N \l__dcp_punct_found_bool
\cs_new_protected:Nn \__dcp_test_punct:
  {
    \bool_set_false:N \l__dcp_punct_found_bool
    \str_map_inline:nn { .,:;!? }
      {
        \token_if_eq_meaning:NNT \l_peek_token ##1
          {
            ~Punctuation~found:~
            \bool_set_true:N \l__dcp_punct_found_bool
            \str_map_break:
          }
      }
    \bool_if:NF \l__dcp_punct_found_bool
      { ~Punctuation~not~found~ }
  }
\cs_new_protected:Nn \__dcp_test_punct:n
  {
    \peek_after:Nw \__dcp_test_punct:
  }
\DeclareDocumentCommand {\testpunct} { +m }
  {
    \__dcp_test_punct:n {#1}
  }
\ExplSyntaxOff
\begin{document}
\testpunct{A}.

\testpunct{A},

\testpunct{A}:

\testpunct{A};

\testpunct{A}!

\testpunct{A}?

\testpunct{A}
\end{document}

output

|improve this answer
1

Sorry, I was just playing around with \futurelet. The idea of turning the resulting implicit-character token (related: A Question About \futurelet) into something explicit with the use of \meaning was suggested to me by Christian Tellechea.

Christian actually provided (under separate cover) a macro \implicittomacro that converts catcode 11/12 implicit-character tokens into a macro. You can go to my answer A Question About \futurelet, to see his macro.

My goal was to be able to use the features of the listofitems package to search the implicit token's value for multiple punctuation values all at once, as detailed in the specification \setsepchar{.||,||:||;||!||?}. Success!

\documentclass{article}
\usepackage{listofitems,xcolor}
\newcommand\testpunct[1]{#1\futurelet\xfl\pdecide}
\def\pdecide{\bgroup\tiny\color{cyan}%
  \edef\ximp{\meaning\xfl}%
  \edef\tmp{\detokenize{the character }}%
  \expandafter\setsepchar\expandafter{\tmp}%
  \readlist\mylist{\ximp}%
  \ifnum\listlen\mylist[]<2\relax (Non catcode=12 follows)\else%
    \setsepchar{.||,||:||;||!||?}%
    \itemtomacro\mylist[2]\thecharacter%
    \readlist*\mychar{\thecharacter}%
    \ifnum\listlen\mychar[]=1\relax (Non-punctuation character follows \mychar[1])\else%
      (This Punctuation found: \mycharsep[1])%
    \fi%
  \fi%
\egroup}
\begin{document}
\testpunct{A}\relax

\testpunct{A}B

\testpunct{A}@

\testpunct{A}.

\testpunct{A},

\testpunct{A}:

\testpunct{A};

\testpunct{A}!

\testpunct{A}?

\testpunct{A}

\end{document}

enter image description here

|improve this answer
  • 1
    Haha. This is exactly why expl3 is so good for people like me :). It turns the incomprehensible into something that I can understand. – David Purton Feb 12 at 22:38
  • @DavidPurton Haha indeed. I understood you were asking vis-a-vis expl3. But I was just having fun and ended up on a quest to digest implicit tokens. Please forgive my presumptuousness, – Steven B. Segletes Feb 12 at 23:54
  • @DavidPurton But, just to put into context, when you have, for example, \let\z=., it is awful hard to actually deduce, after the fact, what is in \z. But \meaning\z gives the response the character . except everything in that string is catcode 12 (even the alphabetic letters). So the trick (and technique here) is to issue a \meaning and save and parse the output of it to recapture what \z actually was. – Steven B. Segletes Feb 12 at 23:58
  • Oh, no worries, I just despair of ever getting to the point of understanding enough to come up with something like that. And even when I see it, it's still hard to follow! At least expl3 hides away a lot of stuff from me – David Purton Feb 13 at 7:19

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.