13

I am still struggling with understanding the f-type expansion. What is it all about? The explanation on page 2 in interface3.pdf is not really satisfying.

In the given example

\tl_set:Nn \l_mya_tl { A }
\tl_set:Nn \l_myb_tl { B }
\tl_set:Nf \l_mya_tl { \l_mya_tl \l_myb_tl }

, how can a check that the content of \l_mya_tl is actually A\l_myb_tl?

Does it matter that \l_mya_tl is re-used in order to be set on the third line, and not another, hitherto unused token list variable, say \l_myc_tl?

Why does expansion stop after expanding \l_mya_tl as it is expandable after all?

Is there any thinkable scenario where f-expansion would continue after expanding the first token (\l_mya_tl, here)? How would \l_mya_tl need to be crafted in order to not interrupt further expansion?

Why would someone want to use f-expansion, which stops at some unpredictable place, when the argument is expected to be really fully expanded? (This is what f as in "fully" means to me.)

  • f-type expansion ends with the first-encountered unexpandable token. If this token is a space, it is gobbled. You want e-type expansion, I guess. – egreg Jun 5 at 9:12
  • Thank you, it is more a general question about the purpose of this expansion type. Under which circumstances would I want to use f instead of x or e expansion? – AlexG Jun 5 at 9:15
  • 1
    it is a thin wrapper around \romannumeral with e now available it is possibly less useful than it was, but advantage over x that it can be used in expansion contexts – David Carlisle Jun 5 at 10:13
12

f-type expansion ends upon finding an unexpandable token; if this token is a space (character code 32, category code 10) it is gobbled.

Your \tl_set:Nf \l_mya_tl { \l_mya_tl\l_myb_tl } will first do recursive expansion of \l_mya_tl, leading to A. This is unexpandable, so the business stops here. The token list to assign is evaluated to A\l_myb_tl and \l_mya_tl is updated to contain this list.

Changing the contents of \l_myb_tl will also change the expansion of \l_mya_tl, because this one contains a pointer to \l_myb_tl, rather than the value this variable had at definition time.

If you want to freeze the value of the updated \l_mya_tl variable to the values of \l_mya_tl and \l_myb_tl you have to use either x-type or e-type expansion.

These last two types lead to the same result, but with a big difference: e-type expansion can appear in expansion contexts, x-type cannot. Not so much of a difference in this case, because you're doing an assignment. Actually, there is no predefined \tl_set:Ne function, because it turns out that \tl_set:Ne would take twice as much time as needed by \tl_set:Nx.

\documentclass{article}
\usepackage{expl3,l3benchmark}

\ExplSyntaxOn

\tl_set:Nn \l_tmpa_tl { A }
\tl_set:Nn \l_tmpb_tl { B }
\tl_new:N \l_tmpc_tl
\cs_generate_variant:Nn \tl_set:Nn { Ne }

\benchmark:n { \tl_set:Nx \l_tmpc_tl { \l_tmpa_tl \l_tmpb_tl } }

\benchmark:n { \tl_set:Ne \l_tmpc_tl { \l_tmpa_tl \l_tmpb_tl } }

\stop

yields, on my machine,

3.16e-7 seconds (1.01 ops)
7.78e-7 seconds (2.39 ops)

In either case, \l_tmpc_tl is assigned AB.

Why would someone want f-expansion? Good question! Until a few months ago, there was no way to do full recursive expansion in expansion contexts. Things changed when the primitive \expanded was added to all engines (it used to be allowed only in LuaTeX), except Knuth TeX, of course.

  • I do agree with OP that "full expansion" is rather misleading in this context. Would it not be better to change that to "expansion up to first obstacle" and claim that the f stands for "first"? – schtandard Jun 5 at 9:42
  • @schtandard Possibly. – egreg Jun 5 at 9:51
  • @schtandard I thought about that several times. To defend the “full”, I'd say that with “first”, one might understand that it acts as o, i.e., does only one expansion step. So, f-expansion is full expansion (read: recursive), but unlike that made by \edef, stops at the first non-expandable token (+ gobbles it if it's a space token). f could mean “front” too. :-) – frougon Jun 5 at 9:59
  • 1
    Perhaps see tex.stackexchange.com/q/489063 on the question of \expanded versus \romannumeral. – Joseph Wright Jun 5 at 10:32
  • 1
    @AlexG Better than that: functions marked with a red star in interface3.pdf (aka, “fully expandable functions”) are safe to use inside an x-type argument (but those with a hollow star are not good to use inside an f-type argument). This is what the bottom of page 4 of interface3.pdf says. – frougon Jun 5 at 14:14
7

Compare is with x-expansion:

\documentclass{article}
\usepackage{expl3}

\begin{document}
\ExplSyntaxOn
\tl_set:Nn \l_mya_tl { A }
\tl_set:Nn \l_myb_tl { B }
\tl_set:Nf \l_myc_tl { \l_mya_tl STOP \l_myb_tl }
\tl_show:N \l_myc_tl 

\tl_set:Nx \l_myc_tl { \l_mya_tl STOP \l_myb_tl }
\tl_show:N \l_myc_tl

\ExplSyntaxOff\end{document}

This will give

> \l_myc_tl=ASTOP\l_myb_tl .
<recently read> }

l.207 \tl_show:N \l_myc_tl

? 
> \l_myc_tl=ASTOPB.
<recently read> }

l.210 \tl_show:N \l_myc_tl
  • Thank you, @Ulrike. Thus, \show is the wanted tool to display the content of a control sequence. How could \l_mya_tl look like such that expansion continues? – AlexG Jun 5 at 9:31
  • 1
    @AlexG One easy case is if the result of recursively expanding \l_mya_tl is empty. You can also use macros inside that don't “produce” any unexpandable content but still have desirable side effects, such as gobbling specific things further in the token list (cf. macros such as \use_none:nnn). – frougon Jun 5 at 9:50
7

As pointed out in the other answers and comments, f-expansion is implemented using \romannumeral which was sometimes needed in expansion contexts before the \expanded primitive was available. This answer also mentions two use cases where it might still be of use, namely expansion without a known end point and lookaheads of the next unexpandable token.

Additionally, I'd like to point out a common use case where it's even wrong to use, as it gives undesired results. This is based on the fact that, while x-expansion continues fully expanding tokens beyond the first unexpanable token, f-expansion is more eager in the case \exp_not:n is used in the token stream.

If we look at the following examples, we see that expansion is the same when \exp:not:N (\noexpand) is used:

\cs_set:Npn \foo { [FOO] }

\tl_set:Nx \l_tmpb_tl { \exp_not:N \foo bar }
\tl_show:N \l_tmpb_tl

\tl_set:Nf \l_tmpb_tl { \exp_not:N \foo bar }
\tl_show:N \l_tmpb_tl

outputs

> \l_tmpb_tl=\foo bar.

> \l_tmpb_tl=\foo bar.

On the other hand, using \exp_not:n (\unexpanded) gives different results:

\tl_set:Nx \l_tmpb_tl { \exp_not:n { \foo } bar }
\tl_show:N \l_tmpb_tl

\tl_set:Nf \l_tmpb_tl { \exp_not:n { \foo } bar }
\tl_show:N \l_tmpb_tl

outputs

> \l_tmpb_tl=\foo bar.

> \l_tmpb_tl=[FOO]bar.

This is especially important when dealing with parts of the contents of token list variables via the \tl_head:, \tl_tail:, \tl_range: etc. functions. All those wrap their result in \exp_not:n. f-expansion may seem appropriate here, but it's actually not:

\tl_set:Nn \l_tmpa_tl { \foo bar }
\tl_set:Nx \l_tmpb_tl { \tl_head:V \l_tmpa_tl }
\tl_show:N \l_tmpb_tl

\tl_set:Nf \l_tmpb_tl { \tl_head:V \l_tmpa_tl }
\tl_show:N \l_tmpb_tl

outputs

> \l_tmpb_tl=\foo .

> \l_tmpb_tl=[FOO].

As pointed out by Phelype Oleinik, protected macros behave differently as well:

\cs_new_protected:Npn \protected_foo { \foo }

\tl_set:Nx \l_tmpb_tl { \protected_foo bar }
\tl_show:N \l_tmpb_tl

\tl_set:Nf \l_tmpb_tl { \protected_foo bar }
\tl_show:N \l_tmpb_tl

outputs

> \l_tmpb_tl=\protected_foo bar.

> \l_tmpb_tl=[FOO]bar.
  • f expansion is reserved for TeX connoisseurs, it seems. Especially the last result, the verbatim foo is counter-intuitive. Could it be used as a reverse operation to \csname ... \endcsname (getting a command sequence's name)? – AlexG Jun 6 at 7:08
  • 1
    Note that the expansion of \foo was foo too. I changed it to [FOO] to avoid confusion. So it's not resulting in the macro name. The point is that it's expanding \foo even though it is hidden in \unexpanded. – siracusa Jun 6 at 7:46
  • Ok, thank you. It is perhaps beyond the question's scope, but would it be difficult to translate the last f-type expansion example into TeX? (To see what happens behind the scene.) – AlexG Jun 6 at 8:06
  • @AlexG There are a lot of low-level expl3 macros involved here, rewriting all them to TeX is no fun. But you can see what's going on in more detail when you add \tracingmacros=1\relax before those lines. Each macro expansion is then written to the log file along with its current parameter values. In essence it's \edef\l_tmpa_tl{\unexpanded{\foo}bar} \show\l_tmpa_tl (x-expansion) vs. \expandafter\def\expandafter\l_tmpa_tl\expandafter{\romannumeral-0\unexpanded{\foo}bar} \show\l_tmpa_tl (f-expansion). – siracusa Jun 6 at 8:28
  • Ah I see. This infamous romannumeral trick is involved. Thank you! – AlexG Jun 6 at 10:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.