7

I am sure this has been answered already, but I cannot seem to find any answer that explains it in core TeX primitives and that I understand on a token level, e.g. I definitely don't want to get into expl3 before I got a grip on TeX itself.

What I am trying to do is basically just being able to write macros that can be used in a nested fashion using braces like \amacro{\bmacro{bla}}, first evaluating \bmacro{bla} to e.g. la (argument without first token = \@cdr) and then plugging that into \amacro, evaluating to l (only first token of argument = \@car).

\catcode`\@=11 %
\def\@car#1#2\@nil{#1}
\def\@cdr#1#2\@nil{#2}
\def\amacro#1{%
    \edef\tempa{#1}%
    \@car\tempa\@nil%
}
\def\bmacro#1{%
    \edef\tempa{#1}%
    \@cdr\tempa\@nil%
}
\catcode`\@=12 %

    a: \amacro{bla}\par
    b: \bmacro{bla}\par
    a(b): \amacro{\bmacro{bla}}

\bye

Using \edef I try to ensure that the argument is fully expanded before I go and operate on it, but while the individual macro calls work fine, compilation fails with the nested way with

! Missing control sequence inserted.
<inserted text> 
                \inaccessible 
l.16 a(b): \amacro{\bmacro{bla}}

? 

It's not that unexpected, since it kind of is a nested \edef on the same name so I could very well see it breaking that way, but I didn't really get around what \inaccessible means in this context and why it arises.

And then, of course, the main point, there must be a proper way to do this in TeX core primitives, right? There has to be some way of fully expanding an argument before doing something with it.

  • 1
    Add \tracingmacros=1 right after \begin{document}, recompile, and look at the transcript (.log) file: you’ll be illuminated. Anyhow, this looks like an X-Y question: could you illustrate exactly what you are trying to achieve? – GuM Jul 31 '18 at 6:55
  • I want to be able to design nestable macros such that, when supplied with another macro, it does not operate on a argument token list like \bmacro { b l a } or \@cdr b l a \@nil but the argument fully expanded to l a so whatever it does (here: just grab the first token, discard the rest) is applied to the result of the inner macro. – Peter Nerlich Jul 31 '18 at 7:07
  • 4
    if your macros are all expandable you can fully expand them but most tex commands are not just defined via expansion and like your \amacro and \bmacro use non-expandable assignments (\edef here) so they have no "full expansion" as they are not expandable. the \inaccessible is added as you have \edef bla{ (as you expanded \tempa) and \edef requires a control sequence to define so that is inserted as an error recovery – David Carlisle Jul 31 '18 at 7:20
  • 2
    no you pre-expanded it in a surrounding edef so you have, as I said, \edef blah .The answer to your final sentence is "No, there is no way to fully expand an argument, unless it is expandable". – David Carlisle Jul 31 '18 at 7:33
  • 1
    This is basically a duplicate of tex.stackexchange.com/a/66168 – David Carlisle Jul 31 '18 at 7:36
6

First of all, you're missing \expandafter:

\documentclass{article}

\makeatletter
\def\amacro#1{%
  \edef\tempa{#1}%
  \expandafter\@car\tempa\@nil 
}
\def\bmacro#1{%
  \edef\tempa{#1}%
  \expandafter\@cdr\tempa\@nil 
}
\makeatother

\begin{document}

a: \amacro{bla}

b: \bmacro{bla}

%    a(b): \amacro{\bmacro{bla}}

\end{document}

otherwise \@car will just produce \temp and \@cdr nothing.

You cannot nest \bmacro in \amacro, because \edef is not expandable.

On the other hand, with LuaTeX there is \expanded (and it will be also in the other engines in a near future).

% compile with lualatex
\documentclass{article}

\makeatletter
\def\amacro#1{%
  \expandafter\@car\expanded{#1}\@nil
}
\def\bmacro#1{%
  \expandafter\@cdr\expanded{#1}\@nil
}
\makeatother

\begin{document}

a: \amacro{bla}

b: \bmacro{bla}

a(b): \amacro{\bmacro{bla}}

\end{document}

enter image description here

  • Interesting to hear, though I won't mark this answer as accepted until \expanded is available in LaTeX, at least. Is it also planned for plain TeX? – Peter Nerlich Jul 31 '18 at 10:46
  • 1
    @PeterNerlich \expanded is a primitive: it's available in plain LuaTeX today, and will be in the other engines for TL'19 (we are just waiting on a code change for XeTeX). – Joseph Wright Jul 31 '18 at 10:47
  • 1
    @PeterNerlich Your question uses LaTeX, so the link to plain is a bit odd. – Joseph Wright Jul 31 '18 at 10:47
  • 1
    @Manuel It is in development pdfTeX for TL'19, so we will get it. The expansion \expanded does isn't quite x-type, hence the new e-type in expl3: try a # token!. We also don't want to change the expandability of existing x-type functions. The next expl3 release will rename a few functions as a result, including \str_if_eq_x:nn to \str_if_eq_e:nn (\str_case_x:nn is a more 'interesting' one). We won't entirely drop f-type expansion: it can be used to do 'look ahead' without grabbing arguments (as TeX does at the start of a table cell, for example) ... – Joseph Wright Jul 31 '18 at 13:04
  • 1
    ... and has different behaviour on 'hitting' a non-expandable token. However, we do expect over time to move to using more e-type and less f-type code, but only once \expanded is well-established (several years). (We are still waiting on feedback from the XeTeX maintainers.) – Joseph Wright Jul 31 '18 at 13:05
5

Basically, I’m posting this answer because it is way too long for a comment.

As usual, @egreg’s post already answers your question in essence. However, I think that you are having difficulties in understanding exactly what expansion means, and that you tend to confuse expansion with digestion (i.e., execution in the stomach), two concepts that are quite different from each other. This is absolutely excusable, since TeX’s behavior in this regard is really hard to understand, and the answer that David Carlisle has linked to is very helpful in explaining this subtle topic; nevertheless, I think that it could also be helpful to show you exactly what happens when your code is executed, in a step-by-step fashion. This is precisely what this answer does.


What happens with \amacro{bla}

The tokens

\amacro{bla}

are replaced with

\edef\tempa{bla} \@car\tempa\@nil

Then TeX processes \edef: this is an unexpandable token, so it is forwarded to the “stomach”, where it is executed. Since the tokens in the replacement text are all unexpandable (they are just character tokens), this means that \tempa is defined to expand into bla. Next, TeX finds \@car, which is defined as

\def\@car#1#2\@nil{#1}

In particular, being a macro, \@car is expandable. The first argument of \@car is undelimited, the second is delimited by \@nil; in our case, \tempa (!) is taken as the first argument, and the second argument is empty. Note that \@nil is gobbled as well as part of the expansion process. The result of the expansion is the first argument, that is, \tempa. In conclusion,

\@car\tempa\@nil

is replaced with

\tempa

This is again an expandable token, so it is expanded: the current definition of \tempa is bla, so

bla

is what remains in the token stream after the expansion. These are all unexpandable tokens, so they are sent to the stomach, where they are executed. The final result is that TeX typesets “abc” (note that it was already in horizontal mode) and that a temporary definition has been made.


What happens with \bmacro{bla}

This is very similar to the previous case, hence I‘ll leave it “as an exercise” to you. The final result is that

\bmacro{bla}

is replaced by nothing, i.e., it is removed from the token stream, but always with the side effect of defining \tempa to expanda into bla.


What happens with \amacro{\bmacro{bla}}

This is, of course, the most interesting case. Initially, the token stream contains

\amacro{\bmacro{bla}}

According to the definition of \amacro, this is replaced with

\edef\tempa{\bmacro{bla}} \@car\tempa\@nil

Again, the next thing that TeX processes is \edef, which, I repeat, is not expandable and proceeds therefore to the stomach, where it is executed (not expanded!): \tempa is defined as a macro that will expand to the current full expansion of \bmacro{bla}. So TeX begins to fully expand \bmacro{bla}; according to the current definition of \bmacro, the first expansion step yields \edef\tempa{bla} \@cdr\tempa\@nil, and TeX looks at each of these ten tokens, in turn, to see if it happens to be expandable. Now:

  • \edef is unexpandable, so it is left alone;

  • \tempa, on the other hand, is expandable, so (aha!) it is expanded according to its current definition, which, at this moment, is still the definition set, as a side effect, by the execution of the line

    b: \bmacro{bla}\\
    

    found in your source file, that is, bla; these three tokens are unexpandable and therefore they remain as they are;

  • a { token follows, which is unexpandable, and hence is left alone;

  • similarly, the next four tokens, b, l, a, and } are copied ”verbatim” into the replacement text;

  • \@cdr is expandable, so it is expanded: \tempa becomes its first argument, while the second argument of \@cdr is empty; \@nil is also gobbled as the argument delimiter; according to its definition, \@cdr is expanded into its second argument, that is, into nothing;

  • at this point, the replacement text of the definition that we were executing (recall that it was

    \edef\tempa{\bmacro{bla}}
    

    see above where we said “it is executed (not expanded!)”) has been expanded fully, yielding \edef bla{bla}.

In conclusion, \tempa is defined as a macro whose replacement text is

\edef bla{bla}

(nine tokens). This completes the execution (not expansion) of the code

\edef\tempa{\bmacro{bla}}

What remains now in the token stream is

\@car\tempa\@nil

\@car is expandable, so it is expanded (obviously…): its first argument is \tempa and its second argument is empty; the expansion is the first argument, that is, \tempa. All in all, what remains in the token stream is

\tempa

(of course, \@nil has been gobbled when the arguments of \@car where fetched). This again is an expandable token, whose current expansion is

\edef bla{bla}

because of the definition we have just performed. So TeX processes \edef, which is unexpandable and is hence executed in the stomach… and, well, I think that now the reason of the error is perfectly clear.

(Deep breath.)

  • Thank you! So far, so clear. Then I add the \expandafter, or rather, since I need to get rid of \edef anyways, delete that and \tempa and instead replace it with... the fully expanded character stream, but how? – Peter Nerlich Aug 1 '18 at 14:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.