# An expandable 'character scanning' command?

Please consider the following MWE, for a "character scanning" command "\looper" which utilizes \@tfor:

\documentclass{minimal}

\usepackage{trace}

\makeatletter
\newcommand\looper[1]{%
% \def\looper#1{%
\typeout{#1}%
\@tfor\next:=#1 \do{\typeout{\next}%
\next%
}
}
\makeatother

\begin{document}

\looper{TEST1}
%% terminal typeout:
% TEST1
% T
% E
% S
% T
% 1

%\typeout{\looper{Test2}}
%% terminal typeout:
% ! Undefined control sequence.
% \next ->\@nil

\edef\tmptest{TEST3}
\looper{\tmptest}
%% terminal typeout:
% TEST3
% TEST3

% \traceon
\edef\outtest{\looper{\tmptest}}
%% terminal typeout:
% ! Undefined control sequence.
% \next ->\@nil

\end{document}


Here the \looper{TEST1} behavior is as I want it, but I'd like also to somehow use \looper{\tmptest} (where \edef\tmptest{TEST3}) and get (in terminal):

\looper{\tmptest}
%% terminal typeout:
% TEST3
% T
% E
% S
% T
% 3


However, the function doesn't really have to expand the argument on its own, as long as I can do something in the call itself, like \looper{\detokenize{\tmptest}} (which I otherwise cannot use here, as it doesn't "unpack" characters)

Furthermore, the function "returns" \next for each character scanned, and thus I'd like to be able to write something like

\edef\outtest{\looper{\tmptest}}


... and get the contents of \outtest to again be "TEST3" - except here, they would be the result of concatenation of \next of the \looper routing (thus, I could instead return -\next instead of \next, and get "-T-E-S-T-3" with a function like this).

Is there something like this built-in already - or can anyone point to a resource that contains a similar function?

EDIT: Just a note, probably this one doesn't expand because \@tfor uses nonexpandable (?) commands (\@nil) - I get this if I activate \traceon:

% with \traceon:
{into \tracingonline=1}
{\edef}

\looper #1->\typeout {#1}\@tfor \next :=#1 \do {\typeout {\next }\next }
#1<-\tmptest

\typeout #1->\begingroup \set@display@protect \immediate \write \@unused {#1}\e
ndgroup
#1<-\tmptest

\set@display@protect ->\let \protect \string
{\string}

\tmptest ->TEST3

\@tfor #1:=->\@tf@r #1
#1<-\next

\@tf@r #1#2\do #3->\def \@fortmp {#2}\ifx \@fortmp \space \else \@tforloop #2\@
nil \@nil \@@ #1{#3}\fi
#1<-\next
#2<- \tmptest
#3<-\typeout {\next }\next

\@fortmp -> \tmptest

\tmptest ->TEST3

\tmptest ->TEST3
{\ifx}
{false}

\@tforloop #1#2\@@ #3#4->\def #3{#1}\ifx #3\@nnil \expandafter \@fornoop \else
#4\relax \expandafter \@tforloop \fi #2\@@ #3{#4}
#1<-\tmptest
#2<- \@nil \@nil
#3<-\next
#4<-\typeout {\next }\next

\next ->\@nil
{undefined}
! Undefined control sequence.
\next ->\@nil

-
It's not clear what you really need here. If you need something like \@tfor that assigns a csname to each item in turn then clearly it can not be expandable or work in an edef, as no TeX assignments are expandable. It is possible of course to have expandable loop commands but it has to expand each item directly you can not assign it to \temp or any other macro. –  David Carlisle May 28 '12 at 12:09
Notice also that \typeout is not expandable, so that cannot go inside the \edef even with some 'expandable' loop. –  Joseph Wright May 28 '12 at 12:16
Hi @DavidCarlisle - \edef is currently more important to me than \csname; I could use the function for many things; one is to append characters "in-between" as written in the OP, and store the result; another thing I have in mind is outputting catcodes of individual characters for debugging. Cheers! –  sdaau May 28 '12 at 12:17
Btw, thanks @DavidCarlisle and @JosephWright for the note on "TeX assignments" and "\typeout" not being expandable. Cheers! –  sdaau May 28 '12 at 12:19

As noted in comments you can not make expandable assignments, but here is one possible definition of an expandable loop that just puts [] around each token:

\def\expandloop#1{\xloop#1\relax}
\def\xloop#1{%
\ifx\relax#1\else[#1]\expandafter\xloop\fi}

\edef\zzzz{\expandloop{TEST1}}

\show\zzzz

\bye


running this produces:

\$ tex loop
This is TeX, Version 3.1415926 (TeX Live 2011/Cygwin)
(./loop.tex
> \zzzz=macro:
->[T][E][S][T][1].
l.8 \show\zzzz

-
Many thanks for that, @DavidCarlisle - great to finally have a working example to consult :) Note to self: the assignment which is not expandable in the OP is the \@tfor\next >> := << #1 ... .. (goddamn comments not supporting nested bold, italic) Thanks again - cheers! –  sdaau May 28 '12 at 12:21
well the := in the latex for isn't an assignment it is just syntax used to delimit the arguments but if you look at your tracing output you will see two instances of \def and \def assignments are not expandable. –  David Carlisle May 28 '12 at 12:24
Ah, there we go (good thing I write notes to self :)) - thanks for correcting me there, @DavidCarlisle! I was also not really aware about \def being not expandable, so thanks for mentioning that, too. Cheers! –  sdaau May 28 '12 at 12:27
Also: if you have a "string" in a command (as in the OP, "\edef\tmptest{TEST3}"), use the following call: "\edef\tmpout{\expandafter\expandloop\expandafter{\tmptest}}" to have all characters processed individually. –  sdaau May 28 '12 at 12:47

In LaTeX3 we have a loop called \tl_map_function:nN which seems to fulfil your requirements. That can of course be recoded in primitives, which for ease of reference I've done here:

\documentclass{article}

\usepackage{pdftexcmds}

\makeatletter
\long\def\expandableloop#1#2{%
\expandableloop@aux#1#2\q@recursion@tail
\break@point
}
\long\def\expandableloop@aux#1#2{%
\break@if@quark@tail{#2}%
#1{#2}%
\expandableloop@aux#1%
}
\long\def\break@if@quark@tail#1{%
\ifnum\pdf@strcmp{\noexpand\q@recursion@tail}{\unexpanded{#1}}=\z@
\expandafter\break@if@quark@tail@aux
\fi
}
\long\def\break@if@quark@tail@aux#1\break@point{}

\makeatletter
\newcommand\looper[1]{%
\expandableloop\detokenize{#1}
}
\makeatother

\begin{document}

\def\tmptest{TEST3}
\edef\outtest{\expandafter\looper\expandafter{\tmptest}}
\show\outtest

\end{document}


Notice that this relies on \pdfstctmp or equivalent functionality: I've loaded the pdftexcmds package to sort out the engine variability here. Notice also that this will not 'accidentally' expand the input, so I've had to do so deliberately using \expandafter.

-
Many thanks for that example, @JosephWright - still haven't jumped to Latex3, but it will be great to have this reference once I do; cheers! –  sdaau May 28 '12 at 12:25