# How can I create a comma-separated list of characters from a text string?

How can I split the string Hello World! - stored into a macro \def\mystring{Hello Word!} - into a comma-separated list of characters (including spaces) which can be used as argument of a \foreach loop (\foreach \char in {\myCSlist}) in order to loop over each character (as with \foreach \char in {H,e,l,l,o,\space,W,o,r,l,d,!})?

## EDIT 1 (Mar 21): Why I want that?

At first, I didn't explain why I want the solution generates a comma-separated list of characters (e.g. \myCSlist) which can be used as argument of a \foreach. It's because I want to create a Tikz \node for each character using a pic. Something like this (from here):

\newcommand{\hsp}{.5}
\tikzset{symbols/.pic={%
\foreach \s[count=\n from 0] in {\myCSlist}{%
\pgfmathsetmacro{\myangle}{360*rnd}%
\node[rotate=\myangle] at (\hsp*\n,0){\s};%
}}}%


## EDIT 2 (Mar 21): Answers review (regarding to my specifications)

Please feel free to edit this part!

The following comparison is made considering the use of LuaLaTeX engine for compilation (so no need of fontenc as pointedd out by Mico in comment).

Answer \foreach requirement Nb Packages* Bonus Particularity
StevenB.Segletes's (listofitems) Yes 2 handles macro in string input, UTF-8, compatible with [tikz]{standalone} class
StevenB.Segletes's (tokcycle only) Yes 1 UTF-8, compatible with [tikz]{standalone} class
Mico's ~Yes 1 UTF-8, compatible with [tikz]{standalone} class LuaLaTeX code
SandyG's Yes 1 UTF-8, compatible with [tikz]{standalone} class
egreg's \stringforeach Yes 2 UTF-8 expl3
egreg's \processtokenlist No? 2 UTF-8 expl3
wipet's \replspaces No? 0 UTF-8 ,compatible with [tikz]{standalone} class TeX code
wipet's \myforeach Yes 0 UTF-8 ,compatible with [tikz]{standalone} class TeX code

Legend:

• "\foreach requirement" = Is the solution pemits to use the output list into a \foreach or something with equivalent features?
• "No?" = I was not able to make it works myself.
• "~Yes" = Not direct because of the use of "special" language (ex. LuaLaTeX or expl3).
• "*" = pgffor not counted because I will use Tikz anyway.

### EDIT 3 (Mar 22): Implementation of the Tikz nodes with pic

\tikzset{%
symbols/.pic={\markletters{Hello World!}%
(\detokenize\expandafter{\mystring})%
\foreachitem\z\in\mylist{%
\pgfmathsetmacro{\myangle}{360*rnd}%
\pgfmathsetmacro{\mystep}{\zcnt-1}% start from 0 (\zcnt macro is provided by listofitems package)
\node[rotate=\myangle] at (\hsp*\mystep,0){\z};%
}}}%


Not working if the string Hello World! is stored into a macro (e.g. \def\myFirststring{Hello World!}) but it seems to be feasible (see StevenB.Segletes's comments).

\newcounter{cnt}
\setcounter{cnt}{-1}
\newcommand\markletters[1]{%
\tokencycle{\stepcounter{cnt}%
\pgfmathsetmacro{\myangle}{360*rnd}%
\pgfmathsetmacro{\mystep}{\thecnt}%
\node[rotate=\myangle] at (\hsp*\mystep,0){##1};}{}{}{\stepcounter{cnt}%
\node[rotate=\myangle] at (\hsp*\mystep,0){##1};}#1\endtokencycle%
\setcounter{cnt}{-1}}%
\tikzset{symbols/.pic={\markletters{Hello World!}}}%


Not working if the string Hello World! is stored into a macro (e.g. \def\myFirststring{Hello World!}) but it seems to be feasible (see StevenB.Segletes's comments).

WARNING: this method is not compatible with \pic[red] {symbols}; (i.e. the text is not colored in red...).

\luaexec{
function DoString ( s )
for i = 1,string.utflength(s) do
tex.sprint ( '\\pgfmathsetmacro{\\mystep}{'.. i .. '-1}% start from 0 ('.. i .. ' acces the loop integer)
\\pgfmathsetmacro{\\myangle}{360*rnd}%
\\node[rotate=\\myangle] at (\\mystep*10 pt,0)  {' .. unicode.utf8.sub ( s , i , i ) .. '};%
' )
end
end
}
\tikzset{symbols/.pic={\DoString{\mystring}}}%

\newcommand{\markletters}[1]{\StrLen{#1}[\strlen]\foreach \chr[count=\n from 0] in {1,...,\strlen}{\pgfmathsetmacro{\myangle}{360*rnd}%
\node[rotate=\myangle] at (\hsp*\n,0){\StrChar{#1}{\chr}};}}%
\tikzset{symbols/.pic={\markletters{\mystring}}}%

\tikzset{symbols/.pic={\stringforeach \char [count=\n from 0] in \mystring {\pgfmathsetmacro{\myangle}{360*rnd}\node[rotate=\myangle] at (\hsp*\n,0){\char};}}}%

\newcounter{cnt}%
\setcounter{cnt}{-1}%
\def\body#1{\stepcounter{cnt}\pgfmathsetmacro{\myangle}{360*rnd}\pgfmathsetmacro{\mystep}{\thecnt}\pgfmathsetmacro{\myangle}{360*rnd}\node[rotate=\myangle] at (\hsp*\mystep,0){#1};}%
\tikzset{symbols/.pic={\replspaces\mystring
\myforeach\mystring\setcounter{cnt}{-1}%
}}%


All these methods give the following result:

If the following code is used (e.g. given with SandyG's solution):

\documentclass{standalone}
\usepackage{tikz}
\usepackage{xstring}

\newcommand{\markletters}[1]{\StrLen{#1}[\strlen]\foreach \chr[count=\n] in {1,...,\strlen}{\pgfmathsetmacro{\myangle}{360*rnd}%
\node[rotate=\myangle] at (\hsp*\n,0){\StrChar{#1}{\chr}};}}%

\def\mystring{Hello World!}%
\newcommand{\hsp}{.5}%
\tikzset{symbols/.pic={\markletters{\mystring}}}%

\begin{document}
\begin{tikzpicture}
\pic {symbols};
\end{tikzpicture}
\end{document}


### EDIT 4 (Mar 23): My final choice (fully opinion-based ;)

I could use all the solutions with a "Yes" (or "~Yes") in the \foreach column of the above table, but I have to choose only one...

It as to be noted that the only solution which creates a macro which can be used as an \foreach argument is SandyG's solution (this feature was explicity mentionned in my question).

The egreg's \stringforeach, StevenB.Segletes's (listofitems) (with the \foreachitem provided by listofitems package) and wipet's \myforeach solutions achieves equivalent functionality but:

My choice is therefore towards the solution of SandyG.

## First attempt

I tried with the \markletters macro of egreg (using xparse and expl3) but the results seems to be not suitable for looping with foreach.

I changed the result print - of the \markletters macro - from (##1) to ##1,, in order to obtain a comma-separated list, but this also results in a unwanted comma , at the end of the comma-separated list.

Here a M(Non)WE:

\documentclass{article}
\usepackage{tikz}% For the \foreach loop
\usepackage{xparse}

% egreg's \markletters macro : https://tex.stackexchange.com/a/359204/262081
\ExplSyntaxOn
\NewDocumentCommand{\markletters}{om}
{
\IfNoValueTF{#1}
{
\kessels_markletters:nn { #2 } { \tl_use:N \l_kessels_marked_letters_tl }
}
{
\kessels_markletters:nn { #2 } { \tl_set_eq:NN #1 \l_kessels_marked_letters_tl }
}
}

\tl_new:N \l_kessels_unmarked_letters_tl
\tl_new:N \l_kessels_marked_letters_tl

\cs_new_protected:Nn \kessels_markletters:nn
{
\tl_set:Nn \l_kessels_unmarked_letters_tl { #1 }
\tl_replace_all:Nnn \l_kessels_unmarked_letters_tl { ~ } { \textvisiblespace }
\tl_clear:N \l_kessels_marked_letters_tl
\tl_map_inline:Nn \l_kessels_unmarked_letters_tl
{
\tl_put_right:Nn \l_kessels_marked_letters_tl { ##1, }
}
#2
}
\ExplSyntaxOff

\def\mystring{Hello World!}

\begin{document}
\markletters[\foo]{\mystring}%
\foreach \char in {\foo}{%
<\char>%
}%
\end{document}


This gives the following result:

But I would prefer this result (note the presence of the space between 'Hello' and 'World'):

– Mico
Mar 20 at 20:43
• If you use lualatex, then you can use a normal looking programming language lua to do this, instead of all this weird \tl_new:N # unreadable stuff :) Mar 20 at 20:50
• @Mico Yes I am (I don't understand Lua or expl3 anyway :)! I don't really care what language is used... Mar 20 at 21:13
• But it’s "Hello, World!" (not "Hello World!"). Mar 21 at 12:03

This can be done using the package xstring. The code also uses \foreach, which is part of tikz but can be loaded with pgffor if your document doesn't use tikz.

\documentclass{article}

\usepackage[T1]{fontenc} % to use accented characters and others
\usepackage{xstring}
\usepackage{pgffor} % in case you're not using tikz

\newcommand{\markletters}[1]{\StrLen{#1}[\strlen]\foreach \chr in {1,...,\strlen}{$<$\StrChar{#1}{\chr}$>$}}

\begin{document}

\markletters{Hello World!}

\markletters{äöü ÄÖÜ // ß}

\end{document}

• +1. How might your code need to be adapted to handle utf8-encoded, as opposed to ascii-encoded, characters?
– Mico
Mar 20 at 21:45
• @Mico: Good thought. This code works fine if you add \usepackage[T1]{fontenc}. I updated my answer. Mar 20 at 21:52
• @SandyG Thanks for the precisons about pgffor. BTW, you will find the reasons I accepted your answer in my question EDIT 4 if you are interrested in. Thanks a lot for this clean an efficient solution! Mar 23 at 8:29

Here's a LuaLaTeX-based solution. It's fully utf8-code aware, i.e., the input string may contain utf8-encoded, as opposed to just ascii-encoded, characters.

The code consists of a Lua function named DoString and a LaTeX utility macros called \DoString, which takes 1 argument -- a character string -- and passes it to the Lua function for further processing.

% !TEX TS-program = lualatex
\documentclass{article}
\directlua{
function DoString ( s )
for i = 1,string.utflength(s) do
tex.sprint ( '<' .. unicode.utf8.sub ( s , i , i ) .. '>' )
end
end
}
%% Define a LaTeX utility macro:
\newcommand\DoString[1]{\directlua{DoString("#1")}}

\begin{document}
\ttfamily
\DoString{Hello World!}

\DoString{äöü ÄÖÜ // ß}
\end{document}


Addendum to address the OP's follow-up question: To create tikz nodes along the lines suggested in the comment, I suggest you (a) load the luacode package (for the \luaexec macro) and (b) change the \directlua{...} chunk to

\luaexec{
function DoString ( s )
for i = 1,string.utflength(s) do
tex.sprint ( '\\node{' .. unicode.utf8.sub ( s , i , i ) .. '};' )
end
end
}

• Is there a way to create a Tikz node for each character? I tried something like this: tex.sprint ( '\node{' .. unicode.utf8.sub ( s , i , i ) .. '};' ) but I got an error. At the end I would like to use these nodes in a Tikz pic as shown in this question (see EDIT 4 and look for the \foreach \s[count=\n] in {a,b,c,f} part. I want to replace the {a,b,c,f} list with the output of your macro). Mar 21 at 12:41
• @SylvainRigal - Pleas see the addendum I just posted. A major caveat: While the code suggested in the addendum should compile, I have no idea if it'll actually work for you as I don't use TikZ myself.
– Mico
Mar 21 at 13:08
• The addendum works like a charm! Thanks a lot! Mar 21 at 18:27
• The font seems to be different from the default Latex font. Is there a way to set the default Latex font with your code? Mar 22 at 13:08
• I learn some things, so it wasn't improductive for me. Thank you very much for your reactivity and support! Mar 22 at 20:08

Here I use a token cycle. No mention was made of macros or groups appearing in the input (only characters and spaces), and so those should be excluded from the input, unless the OP advises on how they should be handled.

If one wishes to then use this list in a for-like loop, one can use \foreachitem from the listofitems package, after reading the comma-separated \mystring into a list.

\documentclass{article}
\usepackage{tokcycle,listofitems}
\makeatletter
\newcommand\markletters[1]{%
\tokcyclexpress{#1}%
\expandafter\expandafter\expandafter\def
\expandafter\expandafter\expandafter\mystring
\expandafter\expandafter\expandafter
{\expandafter\@gobble\the\cytoks}%
}
\makeatother
\begin{document}
\markletters{Hello World!}

(\detokenize\expandafter{\mystring})

\foreachitem\z\in\mylist{$<$\z$>$}

\end{document}


Of course, the whole process could be done more simply inside the token cycle, such that listofitems would not even be needed. And, much to my surprise, if I compile under lualatex, it properly handles UTF-8 input

\documentclass{article}
\usepackage{tokcycle}
\newcommand\markletters[1]{%
\tokencycle{$<$##1$>$}{}{}{$<$##1$>$}#1\endtokencycle%
}
\begin{document}
\markletters{Hello World! äöü ÄÖÜ // ß}
\end{document}


SUPPLEMENT

Based on OP comments, here is a version (based on the first approach) that handles macros and groups. However, macros requiring arguments would have to be handled carefully, since the comma is being inserted after each token/group.

In this implementation, group content is merely echoed as a single entity. If one wanted group content, subdivided by token, tokcycle can do that, but it is not done in this implementation.

\documentclass{article}
\usepackage[T1]{fontenc}
\usepackage{tokcycle,listofitems}
\makeatletter
\stripgroupingtrue
\newcommand\markletters[1]{%
\expandafter\expandafter\expandafter\def
\expandafter\expandafter\expandafter\mystring
\expandafter\expandafter\expandafter
{\expandafter\@gobble\the\cytoks}%
}
\makeatother
\begin{document}
\markletters{Hello World! \today{} is a great day}

(\detokenize\expandafter{\mystring})

\foreachitem\z\in\mylist{$<$\z$>$}

\mylist[7] is the 7th token

\detokenize\expandafter\expandafter\expandafter{\mylist[14]}
is the 14th token
\end{document}


• @Mico I think I was mistaken. If I compile under lualatex and use \markletters{Hello World! äöü ÄÖÜ // ß}, I get the correct result. Please see my edited answer. Mar 20 at 21:55
• @SylvainRigal Please see the SUPPLEMENT to my answer. Perhaps this gives you more of what you seek. Mar 21 at 14:09
• @SylvainRigal In that case, one needs to expand the macro first, as in \def\z{Hello World!}\expandafter\markletters\expandafter{\z}. Alternately, one could build the expansion into the definition of \markletters, with \newcommand\markletters[1]{\expandafter\tokcyclexpress\expandafter{#1}\expandafter\expandafter\expandafter\def \expandafter\expandafter\expandafter\mystring \expandafter\expandafter\expandafter {\expandafter\@gobble\the\cytoks}} Mar 22 at 14:30
• @SylvainRigal In the 2nd example, \newcommand\markletters[1]{\tokencycle{$<$##1$>$}{}{$<$##1$>$}{$<$##1$>$}#1\endtokencycle} will handle macros that don't require arguments. It will ignore anything in a group { }. Mar 23 at 13:32
• @SylvainRigal \documentclass{article} \usepackage{tokcycle} \newcommand\markletters[1]{% \tokencycle{\addcytoks{$<$##1$>$}}{\addcytoks{##1}} {\addcytoks{##1}}{\addcytoks{$<$##1$>$}}#1\endtokencycle% } \begin{document} \markletters{Hello World \today! äö\rule{1ex}{1ex}ü ÄÖÜ // ß} \end{document} will handle both macros and groups UNMODIFIED, that is, they will not add < > around them. Further, macro arguments must be placed in groups, or the argument will get messed up with additions of < >. See ctan.org/pkg/tokcycle for docs and examples. Mar 23 at 13:37

Here's a \stringforeach command, with the same syntax as \foreach.

\documentclass{article}
\usepackage[T1]{fontenc}
\usepackage{geometry}
\usepackage{pgffor}

\makeatletter
\newcommand{\stringforeach}{}
\def\stringforeach#1#2in{%
\def\@foreachvariable{#1}%
\def\@foreachoptions{#2}%
\@ifnextchar\bgroup{\@foreachexplicit}{\@foreachimplicit}%
}
\def\@foreachexplicit#1{\@stringforeachdo{#1}}
\def\@foreachimplicit#1{\expandafter\@stringforeachdo\expandafter{#1}}
\def\@stringforeachdo#1{%
\@transformstringintoclist{#1}%
\begingroup\edef\x{%
\endgroup
\noexpand\foreach
\unexpanded\expandafter{\@foreachvariable}%
\unexpanded\expandafter{\@foreachoptions}%
in
\noexpand\@transformedstring
}\x
}
\makeatother

\ExplSyntaxOn

\cs_new_protected:cpn {@transformstringintoclist} #1
{
\tl_set:Nn \l_tmpa_tl { #1 }
\tl_replace_all:Nnn \l_tmpa_tl { ~ } { \space }
\seq_set_split:NnV \l_tmpa_seq { } \l_tmpa_tl
\tl_set:cx { @transformedstring } { \seq_use:Nn \l_tmpa_seq { , } }
}

\ExplSyntaxOff

\def\mystring{Hello World!}

\begin{document}

\stringforeach \char in \mystring {<\char>}

\stringforeach \char in {Hello World!} {<\char>}

\stringforeach \char [count=\x] in \mystring {<\x=\char>}

\stringforeach \char [count=\x] in {Hello World!} {<\x=\char>}

\end{document}


If you don't need all the \foreach features, it's simpler.

\documentclass{article}
\usepackage[T1]{fontenc}
\usepackage{geometry}

\ExplSyntaxOn

\NewDocumentCommand{\processtokenlist}{smm}
{
\IfBooleanTF{#1}
{% we have a macro
\sylvain_processtl:Vn #2 { #3 }
}
{% we have an explicit token list
\sylvain_processtl:nn { #2 } { #3 }
}
}

\tl_new:N \l__sylvain_processtl_tl

\cs_new_protected:Nn \sylvain_processtl:nn
{
\tl_set:Nn \l__sylvain_processtl_tl { #1 }
\tl_replace_all:Nnn \l__sylvain_processtl_tl { ~ } { \c_space_tl }
\cs_set_protected:Nn \__sylvain_processtl_item:n { #2 }
\tl_map_function:NN \l__sylvain_processtl_tl \__sylvain_processtl_item:n
}
\cs_generate_variant:Nn \sylvain_processtl:nn { V }

\ExplSyntaxOff

\def\mystring{Hello World!}

\begin{document}

\processtokenlist*{\mystring}{<#1>}

\processtokenlist{Hello World!}{<#1>}

\end{document}


• Your solutions seem to not well handle UTF-8 encoding (ß character seems to be not well printed). It was not requested in my original question but pointed out by @Mico in comment. Mar 22 at 20:02
• @SylvainRigal No, currently that's not supported. Mar 22 at 20:39
• @SylvainRigal - The code shown in this answer can handle utf8-encoded characters just fine as long as the document is compiled under LuaLaTeX. (No need to load the fontenc package under LuaLaTeX, though.)
– Mico
Mar 23 at 8:30
• I first tried with fontenc AND LuaLaTeX engine but the "ß" was printed as "SS". Indeed, without fontenc it's OK (if compiled under LuaLaTeX). Thanks. Mar 24 at 9:40

Because we cannot absorb single space by unseparated macro parameter, we must pre-process the given macro by \replspaces macro first. Then you can use \insertcommas which inserts commas between token but not after the last one. Or \insertangles which replaces each token by <token>.

\def\afterfi#1#2\fi{\fi#1}
\def\replspaces#1{\edef#1{\expandafter\replspacesA#1 \end}}
\def\replspacesA #1 #2{#1\ifx\end#2\else { }\afterfi{\replspacesA#2}\fi}
\def\insertcommas#1{\edef#1{\expandafter\insertcommasA#1\end}}
\def\insertcommasA#1#2{#1\ifx\end#2\else,\afterfi{\insertcommasA{#2}}\fi}
\def\insertangles#1{\edef#1{\expandafter\insertanglesA#1\end}}
\def\insertanglesA#1{\ifx\end#1\else<#1>\expandafter\insertanglesA\fi}

% test:

\def\mystring{Hello World!}

\replspaces\mystring
\insertcommas\mystring
\meaning\mystring  % macro:-> H,e,l,l,o, ,W,o,r,l,d,!

\def\mystring{Hello World!}

\replspaces\mystring
\insertangles\mystring
\meaning\mystring  % macro:-> <H><e><l><l><o>< ><W><o><r><l><d><!>

\bye


Edit Your first ask was: how to create a macro H,e,l,l,o, ,w,o,r,d,! from the macro Hello World!. I answered correctly.

Now, your second ask is: how to use such macro in \foreach from Tikz. I don't know how to set effectively the expanded parameter to a somewhat impractical \foreach from Tikz (you can use \expanded{\unexpanded {\foreach ...}{\mystring}} but it seems horrible). It should be better not to use \foreach from Tikz at all and define your own \myforeach:

\def\replspaces#1{\edef#1{\expandafter\replspacesA#1 \end}}
\def\replspacesA #1 #2{#1\ifx\end#2\else { }\afterfi{\replspacesA#2}\fi}
\def\myforeach#1{\expandafter\myforeachA#1\end}
\def\myforeachA#1{\ifx\end#1\else\body{#1}\expandafter\myforeachA\fi}

\def\body#1{<#1>}
\def\mystring{Hello World!}

\replspaces\mystring
\myforeach\mystring % prints: <H><e><l><l><o>< ><W><o><r><l><d><!>

\bye

• If I do something like this \replspaces\mystring \insertcommas\mystring \foreach \char in {\mystring}{<\char>}, it doesn't loop over each character in the string, I just get this: <H,e,l,l,o, ,W,o,r,l,d,!>. Am I doing something wrong? Mar 22 at 8:04
• My first ask was: How to create a macro H,e,l,l,o, ,w,o,r,d,! from the macro Hello World! which can be used as an \foreach argument? I'm sincerely sorry if it was not clear. I'm sure there are many other way to do it but the \foreach syntax is easy to use for me and I can acces easily to the loop integer or start it from 0. Anyway, your solution works fine thank you. Mar 23 at 10:07
• @SylvainRigal The problem is that the parameter for Tikz's \foreach must be expanded before the \foreach is processed. But you cannot use single \expandafter because the parameter is due to the inappropriately \foreach syntax positioned at the very back. I don't want to study the 1400 pages manual of Tikz in order to know, how to do it using Tikz concept. It is more simple for me to write my own \foreach macro. I hoped that you know how to expand the parameter for \foreach from Tikz if you want to use it and that this is not the main problem asked here. Mar 23 at 15:18
• You're right is not the main problem here... But I disturb you again because I tried to used your myforeach with this: \newcounter{cnt}\setcounter{cnt}{-1}\def\body#1{\stepcounter{cnt}\pgfmathsetmacro{\myangle}{360*rnd}\pgfmathsetmacro{\mystep}{\thecnt}\pgfmathsetmacro{\myangle}{360*rnd}\node[rotate=\myangle] at (\hsp*\mystep,0){#1};}% but I got the following error: ! Undefined control sequence. \replspacesA ...->#1\ifx \end #2\else { }\afterfi{\replspacesA #2}\fi. Mar 24 at 10:28
• Undefined control sequence \afterfi? Maybe there is not copied the definition of \afterfi from the firs line of my first example. Ii is my mistake that I didn't copied it from my first example to the second. Mar 24 at 14:53