13

I am looking for an informative answer here. If I want to append to a macro, I have a couple of options, each depending on whether I want the items fully expanded, partially expanded (a certain number of times), or not expanded, right?

Today I am choosing to use the boatload of \expandafters approach.

Storing/Collecting Items in a Macro Conforming to "Userspace" Macro Naming Scheme

no numerals, no symbols, no spaces

\def\appendtolist#1#2{% #1 = Macro Name (list) #2=New Item to Append
  \ifx#1\undefined \def#1{#2}\else% Macro Existence Check  
  \expandafter\def\expandafter#1\expandafter{#1,#2}\fi% #1=Expand Macro List,  #2=New Item
}%

Storing/Collecting Items in a Macro Defined with \csname

contains anything with tradeoff that you have to use a boatload of \expandafters

\def\appendtolist#1#2{% #1 = Macro Name (list) #2=New Item to Append
  \expandafter\ifx\csname#1\endcsname\relax\expandafter\def\csname#1\endcsname{#2}\else% Macro Existence Check  
  %\expandafter\def\expandafter#1\expandafter{#1,#2}\fi% Headache-causing line when trying to surround #1 with \csname
  %\expandafter\def\expandafter\csname #1-names\endcsname% I became lost at this point
}%

Problem / Question Redefined

See headache-causing line above. How can I know the right number of \expandafters beyond the shadow of a doubt without using a trial-and-error approach?

My thoughts up until the stopping point were:

  • Ok, I need \expandafter to expand \csname into a macro
  • Then I \expandafter to expand the new definition of that macro before I TeX defines it.

Notes

  • It's 2^n-1, not 2(n-1) – egreg Jul 25 '16 at 17:01
  • @egreg Fixed! So 4 expansions=(2^4)-1=(16)-1=15. That is so many \expandafters and therefore not elegant. The output of TeX is quite elegant, however. – Jonathan Komar Jul 25 '16 at 17:09
  • @macmadness86 Use etoolbox, it's super useful. – Manuel Jul 26 '16 at 10:49
  • 1
    @Manuel "Today I am choosing to use the boatload of \expandafters approach." ;) Also, egreg already demonstrated the simplified etoolbox version in his answer. But thanks for putting your two cents in anyway. – Jonathan Komar Jul 26 '16 at 11:05
  • @macmadness86 Didn't see that. – Manuel Jul 26 '16 at 11:12
11

The primitive \expandafter carries out exactly one expansion and leaves the result of this in the input stream. Thus

\expandafter\def\csname #1\endcsname

with #1 as foo is equivalent after one expansion to

\def\foo

To get things correct, you have to track how many expansions are required and to know the number of \expandafter commands you need (1, 3, 7, ..., for 1, 2, 3, ... expansions, respectively). We also have some 'tricks' available.

In

\def\appendtolist#1#2{% #1 = Macro Name (list) #2=New Item to Append
....

need to first do a test for existence (which you already have) then to expand \csname #1\endcsname to the command name in two places. This is best done as

\expandafter\def\csname#1\expandafter\expandafter\expandafter\endcsname
   \expandafter\expandafter\expandafter
   {\csname #1\endcsname,#2}%

There is a 'trick' here. We first expand past the \def to start constructing the name of the macro we want for storage. At the end of the \csname ... construct, we need three \expandafters (and two sets as there is a brace to skip over). That's the case as we need to do two expansions: the first to turn \csname #1\endcsname into the macro name (say \foo), and the second to expand \foo into the content of \foo.

Expansion is all about tracking exactly what is in the input stream, and being aware of how many steps you require.

  • 1
    Great answer! You forgot to add a disclaimer: Learning expansion could lead to headaches, unusual outbreaks of aggression including, but not limited to, foul language. – Jonathan Komar Jul 25 '16 at 16:52
  • "two sets as there is a brace to skip over" implies a change to our formula 2^n-1. I am certainly no math expert but it would seem to calculate \expandafters with braces: 1 brace=b=1, sob(2(2^n-1)). If there were two braces, I would need 12 \expandafters. In other words, a boatload of them ;) – Jonathan Komar Jul 26 '16 at 10:11
  • @macmadness86 I don't mean the number of \expandafters required in each place, I mean that to skip over the brace we need them in two places: \expandafter\foo\expandafter\bar\expandafter\baz\target expands the target only once but has to skip over three things, for example. – Joseph Wright Jul 26 '16 at 10:27
  • Yep, we're on the same wavelength (I understood "sets"). Thanks. I just want to be able to predict when I need sets. I would not have guessed to put the first set before the \endcsname, for example. But I guess that is because \endcsname is in itself a second control sequence whereas\def is only one. I found a similar explanation of \csname name\expandafter\endcsname\token here for future reference: tex.stackexchange.com/a/519/13552 – Jonathan Komar Jul 26 '16 at 10:35
  • @macmadness86 You need to \expandafter each token you want to skip. In the case of \csname ...\endcsname the trick is that as this also does expansion we don't have to skip each token in the name, but can just do the \expandafter at the end of the name, immediately before \endcsname. – Joseph Wright Jul 26 '16 at 15:43
10

Formula for the amount of \expandafter that need to be inserted.

If you wish to have TeX "jump" over L tokens in order to have TeX beforehand produce the K-level-expansion outgoing from the (L+1)th token, this requires to have inserted (2K-1) \expandafter-tokens in front of each of these L tokens that are to be "juped" over.

Thus:
If you wish to have TeX "jump" over L tokens in order to have TeX beforehand produce the K-level-expansion outgoing from the (L+1)th token, this requires the insertion of L·(2K-1) \expandafter into the .tex-input.

The basic procedure is successively inserting \expandafter-chains.
Each \expandafter-chain yields another level of expansion.
The \expandafter-chain inserted the last but zero will deliver the first-level-expansion.
The \expandafter-chain inserted the last but one will deliver the second-level-expansion.
...
The \expandafter-chain inserted the last but (K-1) will deliver the K-level-expansion.

The point hereby is:

Having added another \expandafter-chain means having inserted an \expandafter-token in front of each token that should be "jumped" over by the \expandafter-chain that is to be added.
Therefore you need to trace the number of tokens that should be "jumped" over by adding the new \expandafter-chain.

Example:

Having TeX "jump" over four tokens in order to have TeX beforehand produce the 2-level-expansion outgoing from the 5th token requires the insertion of L·(2K-1) \expandafter while L=4 and K=2, thus requires the insertion of 4·(22-1) \expandafter= 12 \expandafter into the .tex-input:

%|the \expand|the \expand|
%|after-chain|after-chain|
%|in this co-|in this co-|
%|lumn deli- |lumn deli- |
%|vers       |vers       |
%|1-level-ex-|2-level-ex-|
%|pansion out|pansion out|
%|-going from|-going from|
%|\Fifth     |\Fifth     |
%|     |     |     |     |
 \expandafter\expandafter
 \expandafter            \First  % _________ 1x(2^K-1)=1x(2^2-1) \expandafter inserted in front of \First
 \expandafter\expandafter
 \expandafter            \Second % _________ 1x(2^K-1)=1x(2^2-1) \expandafter inserted in front of \Second
 \expandafter\expandafter
 \expandafter            \third  % _________ 1x(2^K-1)=1x(2^2-1) \expandafter inserted in front of \Third
 \expandafter\expandafter
 \expandafter            \Fourth % _________ 1x(2^K-1)=1x(2^2-1) \expandafter inserted in front of \Fourth
 \Fifth                          %A total of Lx(2^K-1)=4x(2^2-1) \expandafter inserted into the .tex-input

Let's look at the scenario of successively adding \expandafter-chains in order to have TeX beforehand produce the K-level-expansion outgoing from the (1+1)th token (= the 2nd token).

It is required to insert 1·(2K-1) \expandafter=2K-1 \expandafter into the .tex-input.

(Assume \First is the 1st token before insertion of \expandafter and \Secondis the 2nd token before insertion of \expandafter.)


Case K=0 — Have TeX produce 0-level-expansion outgoing from \Second:
(0-level-expansion outgoing from an expandable token = expansion outgoing from that token does not get triggered.)
0 tokens in front of \Second should be "jumped" over.
There are/is 20 tokens = 1 token in front of \Second. 1 of them is not an \expandafter-token.
You need to insert 20-1 \expandafter = 0 \expandafter:

\First\Second

Case K=1 — Have TeX produce 1-level-expansion outgoing from \Second:
In order to obtain one more level of expansion, another \expandafter-chain needs to be added to the code that was produced for achieving case K=0. This implies the insertion of an \expandafter-token in front of each token that should be "jumped" over.
From case K=0 we know that there are 20 tokens in front of \Second that should be "jumped" over.
Having inserted an \expandafter-token in front of each of these tokens yields 2·(20) tokens=21 tokens in front of \Second. 1 of them is not an \expandafter-token.
You need to insert 21-1 \expandafter:

%|the \expand|
%|after-chain|
%|in this co-|
%|lumn deli- |
%|vers       |
%|1-level-ex-|
%|pansion out|
%|-going from|
%|\Second    |
%|     |     |
 \expandafter\First
 \Second

Case K=2 — Have TeX produce 2-level-expansion outgoing from \Second:
In order to obtain one more level of expansion, another \expandafter-chain needs to be added to the code that was produced for achieving case K=1. This implies the insertion of an \expandafter-token in front of each token that should be "jumped" over.
From case K=1 we know that there are 21 tokens in front of \Second that should be "jumped" over.
Having inserted an \expandafter-token in front of each of these tokens yields 2·(21) tokens=22 tokens in front of \Second. 1 of them is not an \expandafter-token.
You need to insert 22-1 \expandafter:

%|the \expand|the \expand|
%|after-chain|after-chain|
%|in this co-|in this co-|
%|lumn deli- |lumn deli- |
%|vers       |vers       |
%|1-level-ex-|2-level-ex-|
%|pansion out|pansion out|
%|-going from|-going from|
%|\Second    |\Second    |
%|     |     |     |     |
 \expandafter\expandafter
 \expandafter            \First
 \Second

Case K=3 — Have TeX produce 3-level-expansion outgoing from \Second:
In order to obtain one more level of expansion, another \expandafter-chain needs to be added to the code that was produced for achieving case K=2. This implies the insertion of an \expandafter-token in front of each token that should be "jumped" over.
From case K=2 we know that there are 22 tokens in front of \Second that should be "jumped" over.
Having inserted an \expandafter-token in front of each of these tokens yields 2·(22) tokens=23 tokens in front of \Second. 1 of them is not an \expandafter-token.
You need to insert 23-1 \expandafter:

%|the \expand|the \expand|the \expand|
%|after-chain|after-chain|after-chain|
%|in this co-|in this co-|in this co-|
%|lumn deli- |lumn deli- |lumn deli- |
%|vers       |vers       |vers       |
%|1-level-ex-|2-level-ex-|3-level-ex-|
%|pansion out|pansion out|pansion out|
%|-going from|-going from|-going from|
%|\Second    |\Second    |\Second    |
%|     |     |     |     |     |     |
 \expandafter\expandafter
 \expandafter            \expandafter
 \expandafter\expandafter
 \expandafter                        \First
 \Second

Case K=4 — Have TeX produce 4-level-expansion outgoing from \Second:
In order to obtain one more level of expansion, another \expandafter-chain needs to be added to the code that was produced for achieving case K=3. This implies the insertion of an \expandafter-token in front of each token that should be "jumped" over.
From case K=3 we know that there are 23 tokens in front of \Second that should be "jumped" over.
Having inserted an \expandafter-token in front of each of these tokens yields 2·(23) tokens=24 tokens in front of \Second. 1 of them is not an \expandafter-token.
You need to insert 24-1 \expandafter:

%|the \expand|the \expand|the \expand|the \expand|
%|after-chain|after-chain|after-chain|after-chain|
%|in this co-|in this co-|in this co-|in this co-|
%|lumn deli- |lumn deli- |lumn deli- |lumn deli- |
%|vers       |vers       |vers       |vers       |
%|1-level-ex-|2-level-ex-|3-level-ex-|4-level-ex-|
%|pansion out|pansion out|pansion out|pansion out|
%|-going from|-going from|-going from|-going from|
%|\Second    |\Second    |\Second    |\Second    |
%|     |     |     |     |     |     |     |     |
 \expandafter\expandafter
 \expandafter            \expandafter
 \expandafter\expandafter
 \expandafter                        \expandafter
 \expandafter\expandafter
 \expandafter            \expandafter
 \expandafter\expandafter
 \expandafter                                     \First
 \Second

Case ...


Get the picture?

By the way:

There was introduced some indentation providing a visible impression where things are split up in columns.
One column is one \expandafter-chain delivering one level of expansion.
If you count the amount of \expandafter per column/per \expandafter-chain from the rightmost column to the leftmost column, this yields:
1+2+4+8+... = ∑i=1..K{2(i-1)}=2K-1.

In case of obtaining 4-level-expansion outgoing from \Second you have 4 columns of \expandafter holding 1+2+4+8 \expandafter= ∑i=1..4{2(i-1)} \expandafter= 24-1 \expandafter.

Adding another \expandafter-chain for obtaining 5-level-expansion outgoing from \Second means adding an \expandafter in front of the token \First and adding an \expandafter in front of each of the 1+2+4+8 \expandafter that are already there.
This yields 1+2·(1+2+4+8) \expandafter=1+2+4+8+16 \expandafter = ∑i=1..5{2(i-1)} \expandafter = 25-1 \expandafter.

Adding another \expandafter-chain for obtaining 6-level-expansion outgoing from \Second means adding an \expandafter in front of the token \First and adding an \expandafter in front of each of the 1+2+4+8+16 \expandafter that are already there.
This yields 1+2·(1+2+4+8+16) \expandafter=1+2+4+8+16+32 \expandafter = ∑i=1..6{2(i-1)} \expandafter = 26-1 \expandafter.

Now let's think about the scenario of successively adding \expandafter-chains in order to have TeX beforehand produce the K-level-expansion outgoing from the (L+1)th token.

(After inserting \expandafter, the (L+1)th token won't be the (L+1)th token any more. Therefore the token which is the (L+1)th token before insertion of \expandafter will be referred to as "(L+1)-token"—in some places "(L+1)" is an ordinal number, in other places it is a nominal number.)


Case K=0 — Have TeX produce 0-level-expansion outgoing from the (L+1)-token before processing L tokens in front of the expansion-result:
(0-level-expansion outgoing from an expandable token = expansion outgoing from that token does not get triggered.)
The (L+1)-token is the (L+1)th-token in the token-stream.
There are 0 tokens in front of the (L+1)-token that should be "jumped" over.
There are L tokens = L·(20) tokens in front of the (L+1)-token. L of them are not \expandafter-tokens.
You need to insert L·(20)-L \expandafter=L·(20-1) \expandafter=0 \expandafter.


Case K=1 — Have TeX produce 1-level-expansion outgoing from the (L+1)-token before processing L tokens in front of the expansion-result:
In order to obtain one more level of expansion, another \expandafter-chain needs to be added to the code that was produced for achieving case K=0. This implies the insertion of an \expandafter-token in front of each token that should be "jumped" over.
From case K=0 we know that there are L·(20) tokens in front of the (L+1)-token that should be "jumped" over.
Having inserted an \expandafter-token in front of each of these tokens yields 2·(L·(20)) tokens = L·(21) tokens in front of the (L+1)-token. L of them are not \expandafter-tokens.
You need to insert L·(21)-L \expandafter=L·(21-1) \expandafter.


Case K=2 — Have TeX produce 2-level-expansion outgoing from the (L+1)-token before processing L tokens in front of the expansion-result:
In order to obtain one more level of expansion, another \expandafter-chain needs to be added to the code that was produced for achieving case K=1. This implies the insertion of an \expandafter-token in front of each token that should be "jumped" over.
From case K=1 we know that there are L·(21) tokens in front of the (L+1)-token that should be "jumped" over.
Having inserted an \expandafter-token in front of each of these tokens yields 2·(L·(21)) tokens = L·(22) tokens in front of the (L+1)-token. L of them are not \expandafter-tokens.
You need to insert L·(22)-L \expandafter=L·(22-1) \expandafter.


Case K=3 — Have TeX produce 3-level-expansion outgoing from the (L+1)-token before processing L tokens in front of the expansion-result:
In order to obtain one more level of expansion, another \expandafter-chain needs to be added to the code that was produced for achieving case K=2. This implies the insertion of an \expandafter-token in front of each token that should be "jumped" over.
From case K=2 we know that there are L·(22) tokens in front of the (L+1)-token that should be "jumped" over.
Having inserted an \expandafter-token in front of each of these tokens yields 2·(L·(22)) tokens = L·(23) tokens in front of the (L+1)-token. L of them are not \expandafter-tokens.
You need to insert L·(23)-L \expandafter=L·(23-1) \expandafter.


Case K=4 — Have TeX produce 4-level-expansion outgoing from the (L+1)-token before processing L tokens in front of the expansion-result:
In order to obtain one more level of expansion, another \expandafter-chain needs to be added to the code that was produced for achieving case K=3. This implies the insertion of an \expandafter-token in front of each token that should be "jumped" over.
From case K=3 we know that there are L·(23) tokens in front of the (L+1)-token that should be "jumped" over.
Having inserted an \expandafter-token in front of each of these tokens yields 2·(L·(23)) tokens = L·(24) tokens in front of the (L+1)-token. L of them are not \expandafter-tokens.
You need to insert L·(24)-L \expandafter=L·(24-1) \expandafter.


Case ...


There are tricks for reducing the amount of \expandafter.

Some tricks have to do with the fact that expansion is triggered by tokens like \if and \ifcat and \romannumeral and \csname.

Other tricks have to do with having TeX flip macro arguments around.

E.g., using TeX' #{-notation, you can define a macro \name which does take tokens before the first opening brace for an argument and causes another (internal) macro to take tokens nested into the first opening brace and the corresponding closing brace for another/a second argument.

This way you don't need to write \expandafter-chains for starting \csname-expansion.

I elaborated on the \name-macro in the thread "Define a control sequence after that a space matters" which was started at TeX - LaTeX StackExchange in November 10, 2016.

Basically the macro \name is one of those things that are about having TeX flip macro arguments around.

Let's have a look at taking advantage of \romannumeral-expansion.

Actually \romannumeral is there for—well—delivering roman numerals. But there are subtleties in the way in which \romannumeral works which make \romannumeral very interesting for other purposes:

\romannumeral keeps expanding until it finds a number while not delivering any token in case the number found is not positive.

Therefore \romannumeral can be (ab?)used for triggering a lot of expansion-work as long as in the end the first tokens from the expansion-result form a non-positive number.

You can easily create a macro \Expandtimes{<number K>} which takes advantage of \romannumeral-expansion in the way that the sequence
\romannumeral\Expandtimes{<number K>}
needs only one "hit" by \expandafter for producing K "hits" by \expandafter.

\romannumeral\Expandtimes{<number K>} reduces the amount of \expandafter-chains that is needed to only one \expandafter-chain.

Syntax:

\romannumeral\Expandtimes{<number K>}<token sequence>K times the leading token of <token sequence> will be "hit" by \expandafter .
In expansion contexts the leading \romannumeral being "hit" by one \expandafter is sufficient for obtaining these K "hits" by \expandafter on the leading token of <token sequence>.

E.g., with

\def\top{\first 0 }
\def\first{\second1 }
\def\second{\third2 }
\def\third{\fourth3 }
\def\fourth{\fifth4 }
\def\fifth{\sixth5 }
\def\sixth{6 }

\expandafter\string\romannumeral\Expandtimes{0}\top\string\top

\expandafter\string\romannumeral\Expandtimes{1}\top\string\first0

\expandafter\string\romannumeral\Expandtimes{2}\top\string\second1 0

\expandafter\string\romannumeral\Expandtimes{3}\top\string\third2 1 0

\expandafter\string\romannumeral\Expandtimes{4}\top\string\fourth3 2 1 0

\expandafter\string\romannumeral\Expandtimes{5}\top\string\fifth4 3 2 1 0

\expandafter\string\romannumeral\Expandtimes{6}\top\string\sixth5 4 3 2 1 0

\expandafter\string\romannumeral\Expandtimes{7}\top\string6 5 4 3 2 1 0

There are several ways of implementing \Expandtimes.
(A boring way is:

\def\firstoftwo#1#2{#1}
\def\secondoftwo#1#2{#2}

% A check is needed for finding out if an argument is catcode-11-"d" while there are only 
% the possibilities that the argument is either a single catcode-11-"d"
% or a single catcode-12-"m":
\def\innerdfork#1d#2#3dd{#2}%
\def\dfork#1{\innerdfork#1{\firstoftwo}d{\secondoftwo}dd}%

% By means of \romannumeral create as many catcode-12-characters m as expansion-steps are to take place.
% Then by means of recursion for each of these m double the amount of `\expandafter`-tokens and
% add one `\expandafter`-token within \innerExp's first argument.
\def\Expandtimes#1{0\expandafter\innerExp\expandafter{\expandafter}\romannumeral\number\number#1 000d}
\def\innerExp#1#2{\dfork{#2}{#1 }{\innerExp{#1#1\expandafter}}}

Now testing \string\Expandtimes:

\def\top{\first 0 }
\def\first{\second1 }
\def\second{\third2 }
\def\third{\fourth3 }
\def\fourth{\fifth4 }
\def\fifth{\sixth5 }
\def\sixth{6 }

\expandafter\string\romannumeral\Expandtimes{0}\top

\expandafter\string\romannumeral\Expandtimes{1}\top

\expandafter\string\romannumeral\Expandtimes{2}\top

\expandafter\string\romannumeral\Expandtimes{3}\top

\expandafter\string\romannumeral\Expandtimes{4}\top

\expandafter\string\romannumeral\Expandtimes{5}\top

\expandafter\string\romannumeral\Expandtimes{6}\top

\expandafter\string\romannumeral\Expandtimes{7}\top

\bye

)

The example at the bottom of this answer exhibits another way of implementing it.
(The \Expandtimes-variant in the example at the bottom of this answer does trigger a lot of \csname-expansion which in turn does affect allocation of memory related to the semantic nest size. )

E.g., if you wish the macro \top to be "hit" by \expandafter six times while \top—before insertion of \expandafter—being the 16th token in the token-stream, you will need to have inserted six \expandafter-chains for bypassing 15 tokens. That's a lot of \expandafter.
(According to the formula, that would be 15·(26-1) \expandafter = 945 \expandafter.)

Using the \romannumeral\Expandtimes{<number K>}<token sequence>-thingie, you can easily reduce to needing only one \expandafter-chain bypassing 15 tokens. (According to the formula, that would be 15·(21-1) \expandafter = 15 \expandafter.
That makes a difference of 930 \expandafter.)

This would look like this:

\expandafter\tokA
\expandafter\tokB
\expandafter\tokC
\expandafter\tokD
\expandafter\tokE
\expandafter\tokF
\expandafter\tokG
\expandafter\tokH
\expandafter\tokI
\expandafter\tokJ
\expandafter\tokK
\expandafter\tokL
\expandafter\tokM
\expandafter\tokN
\expandafter\tokO
\romannumeral\Expandtimes{6}\top

But 15 still is a lot of \expandafter.

By now only the trick of reducing to needing only one \expandafter-chain by using the \romannumeral\Expandtimes{<number K>}-thingie was applied.
Now there is only a single \expandafter-chain.
But that chain is long.

Keep \expandafter-chains short by having TeX flip around macro arguments.

Another trick for reducing the amount of \expandafter in your code which can be applied in many (but not all!!) situations is keeping \expandafter-chains short simply by having TeX flip macro arguments around/simply by having TeX exchange macro arguments:

\long\def\exchange#1#2{#2#1}%

\expandafter\exchange
\expandafter{%
   \romannumeral\Expandtimes{6}\top
}{%
  \tokA\tokB\tokC\tokD\tokE
  \tokF\tokG\tokH\tokI\tokJ
  \tokK\tokL\tokM\tokN\tokO
}%

Now there is a reduction from needing 945 \expandafter to needing 2 \expandafter.
That makes a difference of 943 \expandafter.

But there is a subtle difference:

With the \romannumeral\Expandtimes{6}-approach bypassing 15 tokens by means of 15 \expandafter, only one expansion-step ("hitting" the first \expandafterof the \expandafter-chain) needs to be triggered for obtaining the 6-level-expansion of \top.

With the approach where the \romannumeral\Expandtimes{6}-thingie was combined with \exchange-ing arguments, two expansion-steps need to be triggered. One expansion-step is needed for "hitting" the first \expandafter of the \expandafter-chain consisting of only two \expandafter. Another one is needed for getting \exchange into doing its job of exchanging arguments.

If needed, you can once more apply good old \romannumeral-expansion for reducing to needing to trigger only one expansion step:

\long\def\exchange#1#2{#2#1}%

\romannumeral0%<-\romannumeral keeps seeking digits and therefore keeps expanding expandable tokens.
\expandafter\exchange
\expandafter{%
   \romannumeral\Expandtimes{6}\top
}{ %<-the space between the opening brace and the percent is needed!!
   %  It gets tokenized as a space-token. After exchanging, it is behind 
   %  the 0-digit-token from \romannumeral0 and causes \romannumeral to
   %  terminate. It gets removed by \romannumeral which in turn does not
   %  deliver any tokens as 0 is not a positive number.
  \tokA\tokB\tokC\tokD\tokE
  \tokF\tokG\tokH\tokI\tokJ
  \tokK\tokL\tokM\tokN\tokO
}%

With this technique the amount of \expandafter is independent from the amount of tokens that are to be "jumped" over.
This technique is useful when the amount of tokens to be "jumped" over is not predictable due to these tokens being delivered via macro-arguments.

Now an example where the tricks are combined for attaching tokens to a list that is stored as a macro whose name is to be given:

\long\def\firstoftwo#1#2{#1}%
\long\def\secondoftwo#1#2{#2}%
\long\def\exchange#1#2{#2#1}%

\long\def\name#1#{\innername{#1}}%
\long\def\innername#1#2{%
  \expandafter\exchange\expandafter{\csname#2\endcsname}{#1}%
}%


\def\rmstop{0 }%
\def\Expandtimes#1{%
  \csname rmstop\expandafter\Expandtimesloop
  \romannumeral\number\number#1 000D\endcsname
}%
\def\Expandtimesloop#1{%
  \if m\noexpand#1%
  \expandafter\expandafter\csname endcsname\expandafter\Expandtimesloop\fi
}%


% The most frugal and most boring thingie without \name and without \Expandtimes
% and without \romannumeral.
\long\def\appendtolist#1#2{%
  \expandafter\ifx\csname #1\endcsname\relax
  \expandafter\firstoftwo\else\expandafter\secondoftwo\fi
  {\expandafter\def\csname#1\endcsname{#2}}%
  {%
    \expandafter\def\csname#1\expandafter\expandafter\expandafter\endcsname
    \expandafter\expandafter\expandafter{\csname#1\endcsname,#2}%
  }%
}%

% Another thingie without \name and without \Expandtimes.
% This time a \romannumeral-trick is used for eliminating the need of
% having \csname launch two \expandafter-chains.
\long\def\otherappendtolist#1#2{%
  \expandafter\ifx\csname #1\endcsname\relax
  \expandafter\firstoftwo\else\expandafter\secondoftwo\fi
  {\expandafter\def\csname#1\endcsname{#2}}%
  {%
    \expandafter\def\csname#1\expandafter\endcsname\expandafter{%
    \romannumeral0\secondoftwo{}{\expandafter\expandafter\expandafter} %
    \csname#1\endcsname,#2}%
  }%
}%

% One more thingie. This time \name and \exchange and \romannumeral-expansion
% are used. On the first glimpse it is confusing. Therefore it is one of my
% favorites.
\long\def\AndOneMoreAppendtolist#1#2{%
  \name\ifx{#1}\relax\expandafter\firstoftwo\else\expandafter\secondoftwo\fi
  {\name\def{#1}{#2}}%
  {\expandafter\exchange\expandafter{\expandafter{\romannumeral\name0 {#1},#2}}%
                                    {\name\expandafter\def\expandafter{#1}\expandafter}%
  }%
}%

% Yet another thingie.
% This time the helper-macro \Expandtimes is used for eliminating the need of
% having \csname launch two \expandafter-chains.
\long\def\yetotherappendtolist#1#2{%
  \expandafter\ifx\csname #1\endcsname\relax
  \expandafter\firstoftwo\else\expandafter\secondoftwo\fi
  {\expandafter\def\csname#1\endcsname{#2}}%
  {%
    \expandafter\def\csname#1\expandafter\endcsname
    \expandafter{\romannumeral\Expandtimes{2}\csname#1\endcsname,#2}%
  }%
}%

% A thingie where you can specify the level of expansion before appending.
\long\def\AppendLevelExpandedTolist#1#2#3{%
  \expandafter\def
  \csname#1%
  \expandafter\endcsname
  \expandafter{%
    \romannumeral
    \expandafter\ifx\csname #1\endcsname\relax
    \expandafter\firstoftwo\else\expandafter\secondoftwo\fi
    {}{%
      \Expandtimes{2}\csname#1%
                  \expandafter\endcsname
                  \expandafter,%
                  \romannumeral
    }%
    \Expandtimes{#2}#3%
  }%
}%    

\tt

\appendtolist{mylist}{element1}

\name\string{mylist}\name\meaning{mylist}

\appendtolist{mylist}{element2}

\name\string{mylist}\name\meaning{mylist}

\otherappendtolist{mylist}{element3}

\name\string{mylist}\name\meaning{mylist}

\AndOneMoreAppendtolist{mylist}{element4}

\name\string{mylist}\name\meaning{mylist}

\yetotherappendtolist{mylist}{element5}

\name\string{mylist}\name\meaning{mylist}

\hrule

Now testing \string\Expandtimes:

\def\top{\first 0 }
\def\first{\second1 }
\def\second{\third2 }
\def\third{\fourth3 }
\def\fourth{\fifth4 }
\def\fifth{\sixth5 }
\def\sixth{6 }

\expandafter\string\romannumeral\Expandtimes{0}\top

\expandafter\string\romannumeral\Expandtimes{1}\top

\expandafter\string\romannumeral\Expandtimes{2}\top

\expandafter\string\romannumeral\Expandtimes{3}\top

\expandafter\string\romannumeral\Expandtimes{4}\top

\expandafter\string\romannumeral\Expandtimes{5}\top

\expandafter\string\romannumeral\Expandtimes{6}\top

\expandafter\string\romannumeral\Expandtimes{7}\top

\hrule

Now testing \string\AppendLevelExpandedTolist:

\AppendLevelExpandedTolist{myotherlist}{0}{\top}

\name\string{myotherlist}\name\meaning{myotherlist}

\AppendLevelExpandedTolist{myotherlist}{1}{\top}

\name\string{myotherlist}\name\meaning{myotherlist}

\AppendLevelExpandedTolist{myotherlist}{2}{\top}

\name\string{myotherlist}\name\meaning{myotherlist}

\AppendLevelExpandedTolist{myotherlist}{3}{\top}

\name\string{myotherlist}\name\meaning{myotherlist}

\AppendLevelExpandedTolist{myotherlist}{4}{\top}

\name\string{myotherlist}\name\meaning{myotherlist}

\AppendLevelExpandedTolist{myotherlist}{5}{\top}

\name\string{myotherlist}\name\meaning{myotherlist}

\AppendLevelExpandedTolist{myotherlist}{6}{\top}

\name\string{myotherlist}\name\meaning{myotherlist}

\AppendLevelExpandedTolist{myotherlist}{7}{\top}

\name\string{myotherlist}\name\meaning{myotherlist}

\bye
  • 2
    Such a great answer! I can't wait to go through it all. Thanks for the time you obviously spent composing this. – Jonathan Komar Nov 18 '16 at 6:59
10

Appending to an existing (parameterless) macro only requires one chain:

\expandafter\def\expandafter\foo\expandafter{\foo<material to add>}

If the parameterless macro is called by name, so requiring \csname, the problem is a bit more complicated, requiring seven \expandafter tokens scattered in various places.

You can do it more easily with various methods.

\makeatletter
\newcommand{\appendtolist}[2]{%
  \@ifundefined{#1}
    {\@namedef{#1}{#2}}% not yet defined
    {\append@to@list{#1}{#2}}%
}
\newcommand{\append@to@list}[2]{%
  \toks0=\expandafter\expandafter\expandafter{\csname #1\endcsname}%
  \toks2={,#2}%
  \expandafter\edef\csname#1\endcsname{\the\toks0 \the\toks2}
}
\makeatother

With the triple \expandafter we reach after the { and expand twice \csname #1\endcsname; the first time we obtain the macro name, the second time its expansion. I use the fact that \the\toks<n> is just expanded once in an \edef.

You can do better, though. First define \append@to@list for the case

\append@to@list{\foo}{bar}

which is easy enough (and is what you already did)

\makeatletter
\newcommand{\append@to@list}[2]{%
  \ifx#1\undefined
    \expandafter\@firstoftwo
  \else
    \expandafter\@secondoftwo
  \fi
  {\def#1{#2}}%
  {\expandafter\def\expandafter#1\expandafter{#1,#2}}%
}
\newcommand{\appendtolist}[2]{%
  \expandafter\append@to@list\csname#1\endcsname{#2}%
}

Do you see the trick? The token obtained by expanding \csname#1\endcsname once is passed to \append@to@list.

This is very similar to the method used by etoolbox with its \appto and \csappto macros. However, whilst the code above doesn't rely on e-TeX primitives, the code in etoolbox does. With etoolbox it's very easy:

\newcommand{\appendtolist}[2]{%
  \ifcsundef{#1}
    {\csappto{#1}{#2}}
    {\csappto{#1}{,#2}}%
}

Last but not least: use expl3: no \expandafter at all, so no need to count them.

\documentclass{article}
\usepackage{xparse}

\ExplSyntaxOn
\NewDocumentCommand{\appendtolist}{mm}
 {% #1 = list name, #2 = tokens to append
  \tl_if_exist:cTF { komar_list_#1_tl }
   {
    \tl_put_right:cn { komar_list_#1_tl } { ,#2 }
   }
   {
    \tl_new:c { komar_list_#1_tl }
    \tl_set:cn { komar_list_#1_tl } { #2 }
   }
 }
\NewExpandableDocumentCommand{\uselist}{m}
 {% #1 = list name
  \tl_use:c { komar_list_#1_tl }
 }
\ExplSyntaxOff

\begin{document}

\appendtolist{test}{a} \uselist{test}

\appendtolist{test}{b} \uselist{test}

\end{document}

For this case it would probably be better to use a clist variable, instead.

enter image description here

  • I feel like I am close to understanding this. I am not seeing how the macro token expanded from the macro name \csname#1\endcsname is passed to \appendtomacro. To me, it looks something like\appendtomacro\#1{#2}. – Jonathan Komar Jul 26 '16 at 10:20
  • @macmadness86 \expandafter\appendtomacro\csname foo\endcsname is the same as \appendtomacro\foo, after \expandafter has performed its duty. – egreg Jul 26 '16 at 10:20
  • That is what I wrote above except I kept it generic with my \foo being \#1. So you can "pass" tokens that way? I was thinking passing would be like \appendtomacro{\foo} – Jonathan Komar Jul 26 '16 at 10:22
  • 1
    @macmadness86 \appendtomacro\foo and \appendtomacro{\foo} are equivalent, if \appendtomacro has arguments. – egreg Jul 26 '16 at 10:24
  • It also looks to me like \appendtomacro is undefined. Where is its definition? – Jonathan Komar Jul 26 '16 at 10:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.