6

This is a question about token processing in TeX. What am I doing wrong when implementing \FutureLetNoSpace?

The book TEX in Practice: Volume III: Tokens, Macros, by Stephan v. Bechtolsheim, really does a good job explaining TeX in detail. However, I ran into a stumbling block that I cannot solve using \tracingall alone.

Could someone demonstrate how I might use the code from the book. I also thought \FutureLetNoSpace is the user-level/document-level command. It seems to imitate \futurelet.

Code

Here I write the text "Hello there!", and inject my token checker immediately after the "t". It checks for the token "h". If found, it should yield a nice log message and "TRUE" in the document.

\documentclass{article}
\usepackage{fontspec}% xelatex

%\tracingall % all hell breaks loose

\catcode`@=11 % or \makeatletter to change category code of @ to 11 and temporarily to access kernel macro \@tabularcr

\long\def\DoLongFutureLet #1#2#3#4{%
  \def\@FutureLetDecide{% hangs here
    #1#2\@FutureLetToken% becomes \ifx#2\@FutureLetToken, which compares two expanded tokens
      \def\@FutureLetNext{#3}%
    \else
      \def\@FutureLetNext{#4}%
    \fi% the \@FutureLetNext gets grabbed into \futurelet below
    \@FutureLetNext
  }%
    \futurelet\@FutureLetToken\@FutureLetDecide
}

\def\DoFutureLet #1#2#3#4{%
  \DoLongFutureLet{#1}{#2}{#3}{#4}
} % identical to \DoLongFutureLet

\def\FutureLetNoSpace #1#2{%
  \def\@FutureLetNoSpaceA{#1}% save arg
  \def\@FutureLetNoSpaceB{#2}% save arg
  \@FutureLetOne
}

\def\@FutureLetOne{%
  \DoFutureLet{\ifx}{ }%
    {\@FutureLetThree}{\@FutureLetOk}% \@FutureLetThree if it is a space token.
}

\edef\@FutureLetNoSpaceTemp{%
  \def\noexpand\@FutureLetThree\space{\noexpand\@FutureLetOne}% force expansion of space into a space token and recall \@FutureLetOne
}
\@FutureLetNoSpaceTemp% why are we calling this macro here?

\def\@FutureLetOk{% called when no space is found
  \expandafter\futurelet\@FutureLetTokenA\@FutureLetTokenB
}

\long\def\DoLongFutureLetNoSpace #1#2#3#4{%
  \def\@FutureLetDecideNoSpace{% \@FutureLetTokenNoSpace is self-contained by this macro
    #1#2\@FutureLetTokenNoSpace% becomes \ifx#2\@FutureLetTokenNoSpace, which compares two expanded tokens
      \def\@FutureLetNextNoSpace{#3}%
    \else
      \def\@FutureLetNextNoSpace{#4}% whatever should get executed on match
     \fi
     \@FutureLetNextNoSpace
    }
    \FutureLetNoSpace{\@FutureLetTokenNoSpace}%
      {\@FutureLetDecideNoSpace}% call \@FutureLetDecideNoSpace instead of \futurelet
}
\def\DoFutureLetNoSpace #1#2#3#4{%
  \DoLongFutureLetNoSpace{#1}{#2}{#3}{#4}%
}

\catcode`@=12 % or \makeatother to restore category code of @ to 12

\begin{document}
Hello t\DoFutureLetNoSpace{\ifx}{h}{\typeout{\noexpand\@FutureLetTokenNoSpace value: \meaning\@FutureLetTokenNoSpace}}TRUE}{\typeout{\noexpand\@FutureLetTokenNoSpace value: \meaning\@FutureLetTokenNoSpace}FALSE}here!
\end{document}

Trace

The last four lines of the trace in the console:

\DoFutureLet #1#2#3#4->\DoLongFutureLet {#1}{#2}{#3}{#4}
#1<-\ifx
#2<-
#3<-\@FutureLetThree
  • 1
    "futurelet that ignores spaces" is more or less the definition of latex's \@ifnextchar – David Carlisle May 18 '18 at 13:04
  • As @DavidCarlisle says, plus see The TeXbook, p. 376. – GuM May 18 '18 at 13:16
  • @GuM So awesome. Thanks! It is not the same implementation as in TEX in Practice, but useful. It uses \afterassignment like in Manuel's answer. – Jonathan Komar May 18 '18 at 13:41
  • What's the problem with \afterassignment? It's more secure to do \def\foo{\afterassignment\nextstep\let\gobble= } than \expandafter\def\expandafter\foo\space{\nextstep}. – Manuel May 18 '18 at 16:35
  • @Manuel I have no problem with it. As I said the comments to your answer, it seems more elegant. I am just trying to grasp the code from the book. I thought I understood its recursive calls, but it keeps choking. Safer is better, of course! – Jonathan Komar May 18 '18 at 16:42
2
\makeatletter
\def\futureletignorespaces#1#2{\def\nextaction{#2}%
  \def\checkspace{\ifx\nexttoken\@sptoken
    \expandafter\@firstoftwo
   \else
    \expandafter\@secondoftwo
   \fi
   {\afterassignment\futureletagain\let\nexttoken= }
   {\futurelet#1\nextaction}}%
  \futurelet\nexttoken\checkspace}
\def\futureletagain{\futurelet\nexttoken\checkspace}
\makeatother

This started as just a substitute for \futurelet\nexttoken\action ignoring spaces \futureletignorespaces\nexttoken\action. But I changed the code a bit so that it lets you define the second argument on the fly:

\futureletignorespaces\nexttoken\action % or you could define \action in the argument
\futureletignorespaces\nexttoken{\ifx\nexttoken!yes\fi}
  • Doesn't work yet. – Manuel May 18 '18 at 12:10
  • +1 for effort haha. I am well aware that my question is a pain. – Jonathan Komar May 18 '18 at 12:13
  • It does work now. – Manuel May 18 '18 at 12:16
  • Note that this has the same syntax as \futurelet. \futurelet\nexttoken\action or \futureletignorespaces\nexttoken\action. – Manuel May 18 '18 at 12:17
  • Then why switch to \afterassignment? I seem to remember one of them puts a token back into the token stream. – Jonathan Komar May 18 '18 at 12:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.