# Extract first, last and rest of tokens

This question is related to Extract first & last characters of macro argument?.

It seems that if, besides getting first and last tokens, one wants to accumulate the rest of the tokens, the solution is nontrivial. My solution appears convoluted. I wonder if David Carlisle can instantly pull a neater one out of his pocket. Full expansion of tokens should be avoided because they may contain undefined controls.

\documentclass{article}
\makeatletter
\usepackage{catoptions}

\def\fl#1{\fl@i#1\@nil\@nil\@nnil}
\def\fl@i#1#2#3\@nnil{%
\def\rest{}\def\last{}%
\edef\first{\ifstrcmpTF{#1}\@nil{}{\unexpanded{#1}}}%
\edef\reserved@b{\unexpanded{#3}}%
\ifcsemptyTF\reserved@b{%
\edef\last{\ifstrcmpTF{#2}\@nil{}{\unexpanded{#2}}}%
}{%
\ifcseqTF\reserved@b\@nnil{%
\edef\last{\ifstrcmpTF{#2}\@nil{}{\unexpanded{#2}}}%
}{%
\edef\last{\unexpanded{#2}}%
\fl@ii#3\@nnil
}%
}%
\ifx\last\rest\def\rest{}\fi
}
\def\fl@ii#1#2#3\@nnil{%
\ifstrcmpTF{#1}\@nil{}{%
\ifstrcmpTF{#2}\@nil{%
\edef\rest{\expandcsonce\last\expandcsonce\rest}%
\edef\last{\unexpanded{#1}}%
}{%
\edef\rest{\expandcsonce\rest\unexpanded{#1}}%
\fl@ii#2#3\@nnil
}%
}%
}

\def\x{\string\x}
\def\y#1{\fl{#1}\immediate\write20{Given token
[\iflacus#1\dolacus empty/null\else#1\fi]
^^J[\first][\last][\rest]}
}
\immediate\write20{[first][last][rest]}
\y{}
\y{\x}
\y{1\x2}
\y{123}
\y{1234\x}

\begin{document}
x
\end{document}

-
Hmph that sounds like a challenge –  David Carlisle Feb 3 '12 at 21:46
You have asked eight questions, but still accepted no answer. Accepting one of the answers provided is not mandatory, but it would be better if you reconsidered them, as people puts their efforts in answering your questions and probably some of the answers did help you. –  egreg Feb 3 '12 at 22:25
Can your token list contain spaces? Can it contain brace groups? What should be returned if the token list starts or ends with a space or group? –  Bruno Le Floch Feb 3 '12 at 23:01
@AhmedMusa (egreg didn't get notified: you need to write @egreg), you probably only voted those answers up. To accept an answer (only one accepted answer per question), click on the green(?) checkmark below the arrows used for voting. By the way, here, feel free to accept a non-LaTeX3 answer if those were more helpful. –  Bruno Le Floch Feb 4 '12 at 23:16

Here is a solution which takes a time linear in the size of the token list, faster than egreg's method. It does not preserve spaces nor brace groups.

The idea is to turn the token list into a sequence using \seq_set_split:Nnn with an empty "delimiter" argument (to split between every token), then use the functions \seq_pop_left:NN and \seq_pop_right:NN to extract (and remove) the first and last items from the sequence. Finally, the sequence holds the contents of the token list without its first and last elements; we need to convert back to a token list: this is done within an \edef (\tl_set:Nx), applying \unexpanded (\exp_not:n) to each item in the sequence, to prevent expansion within this expanding assignment.

\RequirePackage{expl3,xparse}
\ExplSyntaxOn
\seq_new:N \l_your_seq
\DeclareDocumentCommand{\fl}{m}
{
\seq_set_split:Nnn \l_your_seq { } {#1}
\seq_pop_left:NN \l_your_seq \first
\seq_pop_right:NN \l_your_seq \last
\tl_set:Nx \middle { \seq_map_function:NN \l_your_seq \exp_not:n }
}
\ExplSyntaxOff

-
Bruno: This is great, but what shall some of us who have somewhat refused to learn LaTeX3 do? –  Ahmed Musa Feb 4 '12 at 23:05
@Ahmed: (1) code your own or (2) use that code, it works. Seriously, life is short, I'm not going to rewrite everything from scratch every time. –  Bruno Le Floch Feb 4 '12 at 23:13
Bruno: I sometimes rewrite a working code. I've learnt a lot from reinventing the wheel. Life is short, but certainly not for learning. BTW, why does this site drop '@name'? –  Ahmed Musa Feb 5 '12 at 0:08
@AhmedMusa That's a subtlety of the site: when only two people including the post owner (here, me) dialogue in comments, the @<post owner> is removed, since the post owner would be notified anyways. –  Bruno Le Floch Feb 5 '12 at 0:12

The LaTeX3 infrastructure can be used for this: it provides \tl_head:V for grabbing the first token from a token list variable and \tl_tail:V for extracting what remains after removing the first token. It's easy to grab the first token; in order to get the last one we reverse the token list, grab the first element, remove it, and reverse again.

At the end, the token list variables \l_fl_first_tl, \l_fl_last_tl and \l_fl_rest_tl will contain the required elements. If the token list has length one, the rest and the last elements are empty.

\documentclass[a4paper]{article}
\usepackage{xparse}
\ExplSyntaxOn
\NewDocumentCommand{\fl}{m}
{
\tl_set:Nn \l_fl_rest_tl { #1 }
\tl_set:Nx \l_fl_first_tl { \tl_head:V \l_fl_rest_tl }
\tl_set:Nx \l_fl_rest_tl { \tl_tail:V \l_fl_rest_tl }
\tl_set:Nx \l_fl_rest_tl { \tl_reverse:V \l_fl_rest_tl }
\tl_set:Nx \l_fl_last_tl { \tl_head:V \l_fl_rest_tl }
\tl_set:Nx \l_fl_rest_tl { \tl_tail:V \l_fl_rest_tl }
\tl_set:Nx \l_fl_rest_tl { \tl_reverse:V \l_fl_rest_tl }
\iow_term:x { [\l_fl_first_tl] [\l_fl_last_tl] [\l_fl_rest_tl] }
}
\tl_new:N \l_fl_first_tl
\tl_new:N \l_fl_rest_tl
\tl_new:N \l_fl_last_tl
\ExplSyntaxOff
\protected\def\x{aaaa}
\immediate\write20{===1}
\fl{\x}
\immediate\write20{===12}
\fl{1\x2}
\immediate\write20{===123}
\fl{123}
\immediate\write20{===1234}
\fl{1234\x}
\immediate\write20{===1234}
\fl{1234{\x\x}}


The result is

===1
[\x ][][]
===12
[1][2][\x ]
===123
[1][3][2]
===1234
[1][\x ][234]
===1234
[1][\x \x ][234]

-

This code doesn't pretend to be the best solution, but it sure is shorter. The macros read the input one token at a time. Non-marginal tokens are appended to \middle. Handling edge cases is left as an exercise. And hail \expandafter!

\documentclass{article}

\makeatletter
\def\split#1{%
\def\middle{}%
\def\last{}%
\let\next=\split@next
\split@#1\@@end}
\def\split@#1{%
\def\first{#1}%
\split@next}
\def\split@next#1{%
\ifx#1\@@end
\let\next=\relax
\else
\expandafter\expandafter\expandafter\def
\expandafter\expandafter\expandafter\middle
\expandafter\expandafter\expandafter{\expandafter\middle\last}%
\def\last{#1}%
\fi
\next}
\makeatother

\begin{document}

\split{12345}
\first,\middle,\last

\end{document}

-
Andrey Vihrov: Is that you didn't want to use eTeX and preferred a chain of \expandafters instead? –  Ahmed Musa Feb 4 '12 at 23:17
@AhmedMusa: I'm not familiar with e-TeX very well. What command could be used instead? –  Andrey Vihrov Feb 5 '12 at 0:03

Had to slip away, @andrey-vihrov did something similar to I would probably have done, so I thought I'd do something different, this one just works by expansion only, but gets what I hope is the intended answer:

[][][]
[\x ][][]
[1][\x ][2]
[1][2][3]
[1][234][\x ]
[1][234][\x \x ]


plain TeX:

\def\fl#1{%
\toks0\expandafter\expandafter\expandafter{%
\expandafter\fleat\romannumeral\Q\flx#1\relax\flx}%
\immediate\write30{\the\toks0}}

\def\fla#1#2{#1}
\def\flb#1#2{#2}
\def\fleat#1[{[}

\def\flx#1#2\flx{%
\ifx\relax#1%
\expandafter\fla\else\expandafter\flb
\fi
{[][][]}{%
\ifx\relax#2%
\expandafter\fla\else\expandafter\flb
\fi
{[#1][][]}{\fly#2\flx{}{#1}}}}

\def\fly#1#2\flx#3#4{%
\ifx\relax#2%
\expandafter\fla\else\expandafter\flb
\fi
{[#4][#3][#1]}{\fly#2\flx{#3#1}{#4}}}

\def\x{aaaa}
\fl{}
\fl{\x}
\fl{1\x2}
\fl{123}
\fl{1234\x}
\fl{1234{\x\x}}

\end
`
-