How can I create a comma-separated list of characters from a text string?

Question

How can I split the string Hello World! - stored into a macro \def\mystring{Hello Word!} - into a comma-separated list of characters (including spaces) which can be used as argument of a \foreach loop (\foreach \char in {\myCSlist}) in order to loop over each character (as with \foreach \char in {H,e,l,l,o,\space,W,o,r,l,d,!})?

EDIT 1 (Mar 21): Why I want that?

At first, I didn't explain why I want the solution generates a comma-separated list of characters (e.g. \myCSlist) which can be used as argument of a \foreach. It's because I want to create a Tikz \node for each character using a pic. Something like this (from here):

\newcommand{\hsp}{.5}
\tikzset{symbols/.pic={%
    \foreach \s[count=\n from 0] in {\myCSlist}{%
    \pgfmathsetmacro{\myangle}{360*rnd}%
    \node[rotate=\myangle] at (\hsp*\n,0){\s};%
}}}%

EDIT 2 (Mar 21): Answers review (regarding to my specifications)

Please feel free to edit this part!

The following comparison is made considering the use of LuaLaTeX engine for compilation (so no need of fontenc as pointedd out by Mico in comment).

Answer	\foreach requirement	Nb Packages*	Bonus	Particularity
StevenB.Segletes's (listofitems)	Yes	2	handles macro in string input, UTF-8, compatible with [tikz]{standalone} class
StevenB.Segletes's (tokcycle only)	Yes	1	UTF-8, compatible with [tikz]{standalone} class
Mico's	~Yes	1	UTF-8, compatible with [tikz]{standalone} class	LuaLaTeX code
SandyG's	Yes	1	UTF-8, compatible with [tikz]{standalone} class
egreg's \stringforeach	Yes	2	UTF-8	expl3
egreg's \processtokenlist	No?	2	UTF-8	expl3
wipet's \replspaces	No?	0	UTF-8 ,compatible with [tikz]{standalone} class	TeX code
wipet's \myforeach	Yes	0	UTF-8 ,compatible with [tikz]{standalone} class	TeX code

Legend:

• "\foreach requirement" = Is the solution pemits to use the output list into a \foreach or something with equivalent features?
• "No?" = I was not able to make it works myself.
• "~Yes" = Not direct because of the use of "special" language (ex. LuaLaTeX or expl3).
• "*" = pgffor not counted because I will use Tikz anyway.

EDIT 3 (Mar 22): Implementation of the Tikz nodes with pic

• StevenB.Segletes's (listofitems):

\tikzset{%
symbols/.pic={\markletters{Hello World!}%
    (\detokenize\expandafter{\mystring})%
    \readlist\mylist\mystring
    \foreachitem\z\in\mylist{%
    \pgfmathsetmacro{\myangle}{360*rnd}%
    \pgfmathsetmacro{\mystep}{\zcnt-1}% start from 0 (\zcnt macro is provided by listofitems package)
    \node[rotate=\myangle] at (\hsp*\mystep,0){\z};%
}}}%

Not working if the string Hello World! is stored into a macro (e.g. \def\myFirststring{Hello World!}) but it seems to be feasible (see StevenB.Segletes's comments).

• StevenB.Segletes's (tokcycle only):

\newcounter{cnt} 
\setcounter{cnt}{-1}
\newcommand\markletters[1]{%
  \tokencycle{\stepcounter{cnt}%
  \pgfmathsetmacro{\myangle}{360*rnd}%
  \pgfmathsetmacro{\mystep}{\thecnt}%
  \node[rotate=\myangle] at (\hsp*\mystep,0){##1};}{}{}{\stepcounter{cnt}%
  \node[rotate=\myangle] at (\hsp*\mystep,0){##1};}#1\endtokencycle%
\setcounter{cnt}{-1}}%
\tikzset{symbols/.pic={\markletters{Hello World!}}}%

Not working if the string Hello World! is stored into a macro (e.g. \def\myFirststring{Hello World!}) but it seems to be feasible (see StevenB.Segletes's comments).

WARNING: this method is not compatible with \pic[red] {symbols}; (i.e. the text is not colored in red...).

• Mico's:

\luaexec{
function DoString ( s )
  for i = 1,string.utflength(s) do 
    tex.sprint ( '\\pgfmathsetmacro{\\mystep}{'.. i .. '-1}% start from 0 ('.. i .. ' acces the loop integer)
    \\pgfmathsetmacro{\\myangle}{360*rnd}%
    \\node[rotate=\\myangle] at (\\mystep*10 pt,0)  {' .. unicode.utf8.sub ( s , i , i ) .. '};%
    ' )
  end
end
}
\tikzset{symbols/.pic={\DoString{\mystring}}}%

• SandyG's:

\newcommand{\markletters}[1]{\StrLen{#1}[\strlen]\foreach \chr[count=\n from 0] in {1,...,\strlen}{\pgfmathsetmacro{\myangle}{360*rnd}%
\node[rotate=\myangle] at (\hsp*\n,0){\StrChar{#1}{\chr}};}}%
\tikzset{symbols/.pic={\markletters{\mystring}}}%

• egreg's \stringforeach:

\tikzset{symbols/.pic={\stringforeach \char [count=\n from 0] in \mystring {\pgfmathsetmacro{\myangle}{360*rnd}\node[rotate=\myangle] at (\hsp*\n,0){\char};}}}%

• wipet's \myforeach:

\newcounter{cnt}%
\setcounter{cnt}{-1}%
\def\body#1{\stepcounter{cnt}\pgfmathsetmacro{\myangle}{360*rnd}\pgfmathsetmacro{\mystep}{\thecnt}\pgfmathsetmacro{\myangle}{360*rnd}\node[rotate=\myangle] at (\hsp*\mystep,0){#1};}%
\tikzset{symbols/.pic={\replspaces\mystring 
\myforeach\mystring\setcounter{cnt}{-1}%
}}%

All these methods give the following result:

If the following code is used (e.g. given with SandyG's solution):

\documentclass{standalone}
\usepackage{tikz}
\usepackage{xstring}

\newcommand{\markletters}[1]{\StrLen{#1}[\strlen]\foreach \chr[count=\n] in {1,...,\strlen}{\pgfmathsetmacro{\myangle}{360*rnd}%
\node[rotate=\myangle] at (\hsp*\n,0){\StrChar{#1}{\chr}};}}%

\def\mystring{Hello World!}%
\newcommand{\hsp}{.5}%
\tikzset{symbols/.pic={\markletters{\mystring}}}%

\begin{document}
\begin{tikzpicture}
\pic {symbols};
\end{tikzpicture}
\end{document}

EDIT 4 (Mar 23): My final choice (fully opinion-based ;)

I could use all the solutions with a "Yes" (or "~Yes") in the \foreach column of the above table, but I have to choose only one...

It as to be noted that the only solution which creates a macro which can be used as an \foreach argument is SandyG's solution (this feature was explicity mentionned in my question).

The egreg's \stringforeach, StevenB.Segletes's (listofitems) (with the \foreachitem provided by listofitems package) and wipet's \myforeach solutions achieves equivalent functionality but:

• SandyG's solution supports UTF-8 (egreg's \stringforeach solution doesn't yet).
• SandyG's solution is compatible with [tikz]{standalone} document class (strange things happen with egreg's \stringforeach solution).
• SandyG's solution handle the use of a string input strored into a macro: \def\mystring{Hello Word!} (I was not able achieve this feature with StevenB.Segletes's (listofitems) but it seems to be feasible, see StevenB.Segletes's comments)
• SandyG's solution uses a very short code that is easily understood by a beginner (which is not the case with expl3 or TeX).

My choice is therefore towards the solution of SandyG.

First attempt

I tried with the \markletters macro of egreg (using xparse and expl3) but the results seems to be not suitable for looping with foreach.

I changed the result print - of the \markletters macro - from (##1) to ##1,, in order to obtain a comma-separated list, but this also results in a unwanted comma , at the end of the comma-separated list.

Here a M(Non)WE:

\documentclass{article}
\usepackage{tikz}% For the \foreach loop
\usepackage{xparse}

% egreg's \markletters macro : https://tex.stackexchange.com/a/359204/262081
\ExplSyntaxOn
\NewDocumentCommand{\markletters}{om}
 {
  \IfNoValueTF{#1}
   {
    \kessels_markletters:nn { #2 } { \tl_use:N \l_kessels_marked_letters_tl }
   }
   {
    \kessels_markletters:nn { #2 } { \tl_set_eq:NN #1 \l_kessels_marked_letters_tl }
   }
 }

\tl_new:N \l_kessels_unmarked_letters_tl
\tl_new:N \l_kessels_marked_letters_tl

\cs_new_protected:Nn \kessels_markletters:nn
 {
  \tl_set:Nn \l_kessels_unmarked_letters_tl { #1 }
  \tl_replace_all:Nnn \l_kessels_unmarked_letters_tl { ~ } { \textvisiblespace }
  \tl_clear:N \l_kessels_marked_letters_tl
  \tl_map_inline:Nn \l_kessels_unmarked_letters_tl
   {
    \tl_put_right:Nn \l_kessels_marked_letters_tl { ##1, }
   }
  #2
 }
\ExplSyntaxOff

\def\mystring{Hello World!}

\begin{document}
\markletters[\foo]{\mystring}%
\foreach \char in {\foo}{%
    <\char>%
}%
\end{document}

This gives the following result:

But I would prefer this result (note the presence of the space between 'Hello' and 'World'):

Answer 1

This can be done using the package xstring. The code also uses \foreach, which is part of tikz but can be loaded with pgffor if your document doesn't use tikz.

\documentclass{article}

\usepackage[T1]{fontenc} % to use accented characters and others
\usepackage{xstring}
\usepackage{pgffor} % in case you're not using tikz

\newcommand{\markletters}[1]{\StrLen{#1}[\strlen]\foreach \chr in {1,...,\strlen}{$<$\StrChar{#1}{\chr}$>$}}

\begin{document}

\markletters{Hello World!}

\markletters{äöü ÄÖÜ // ß}

\end{document}

Answer 2

Here's a LuaLaTeX-based solution. It's fully utf8-code aware, i.e., the input string may contain utf8-encoded, as opposed to just ascii-encoded, characters.

The code consists of a Lua function named DoString and a LaTeX utility macros called \DoString, which takes 1 argument -- a character string -- and passes it to the Lua function for further processing.

% !TEX TS-program = lualatex
\documentclass{article}
\directlua{
function DoString ( s )
  for i = 1,string.utflength(s) do 
    tex.sprint ( '<' .. unicode.utf8.sub ( s , i , i ) .. '>' )
  end
end
}
%% Define a LaTeX utility macro:
\newcommand\DoString[1]{\directlua{DoString("#1")}}

\begin{document}
\ttfamily
\DoString{Hello World!}

\DoString{äöü ÄÖÜ // ß}
\end{document}

---

Addendum to address the OP's follow-up question: To create tikz nodes along the lines suggested in the comment, I suggest you (a) load the luacode package (for the \luaexec macro) and (b) change the \directlua{...} chunk to

\luaexec{
function DoString ( s )
  for i = 1,string.utflength(s) do 
    tex.sprint ( '\\node{' .. unicode.utf8.sub ( s , i , i ) .. '};' )
  end
end
}

Answer 3

Here I use a token cycle. No mention was made of macros or groups appearing in the input (only characters and spaces), and so those should be excluded from the input, unless the OP advises on how they should be handled.

If one wishes to then use this list in a for-like loop, one can use \foreachitem from the listofitems package, after reading the comma-separated \mystring into a list.

\documentclass{article}
\usepackage{tokcycle,listofitems}
\Characterdirective{\addcytoks{,#1}}
\Spacedirective{\addcytoks{,#1}}
\makeatletter
\newcommand\markletters[1]{%
  \tokcyclexpress{#1}%
  \expandafter\expandafter\expandafter\def
  \expandafter\expandafter\expandafter\mystring
  \expandafter\expandafter\expandafter
  {\expandafter\@gobble\the\cytoks}%
}
\makeatother
\begin{document}
\markletters{Hello World!}

(\detokenize\expandafter{\mystring})

\readlist\mylist\mystring

\foreachitem\z\in\mylist{$<$\z$>$}

\end{document}

Of course, the whole process could be done more simply inside the token cycle, such that listofitems would not even be needed. And, much to my surprise, if I compile under lualatex, it properly handles UTF-8 input

\documentclass{article}
\usepackage{tokcycle}
\newcommand\markletters[1]{%
  \tokencycle{$<$##1$>$}{}{}{$<$##1$>$}#1\endtokencycle%
}
\begin{document}
\markletters{Hello World! äöü ÄÖÜ // ß}
\end{document}

SUPPLEMENT

Based on OP comments, here is a version (based on the first approach) that handles macros and groups. However, macros requiring arguments would have to be handled carefully, since the comma is being inserted after each token/group.

In this implementation, group content is merely echoed as a single entity. If one wanted group content, subdivided by token, tokcycle can do that, but it is not done in this implementation.

\documentclass{article}
\usepackage[T1]{fontenc}
\usepackage{tokcycle,listofitems}
\makeatletter
\stripgroupingtrue
\newcommand\markletters[1]{%
  \tokcycle{\addcytoks{,##1}}{\addcytoks{,}\groupedcytoks{##1}}%
    {\addcytoks{,##1}}{\addcytoks{,##1}}{#1}%
  \expandafter\expandafter\expandafter\def
  \expandafter\expandafter\expandafter\mystring
  \expandafter\expandafter\expandafter
  {\expandafter\@gobble\the\cytoks}%
}
\makeatother
\begin{document}
\markletters{Hello World! \today{} is a great day}

(\detokenize\expandafter{\mystring})

\readlist\mylist\mystring

\foreachitem\z\in\mylist{$<$\z$>$}

\mylist[7] is the 7th token

\detokenize\expandafter\expandafter\expandafter{\mylist[14]}
 is the 14th token
\end{document}

Answer 4

Here's a \stringforeach command, with the same syntax as \foreach.

\documentclass{article}
\usepackage[T1]{fontenc}
\usepackage{geometry}
\usepackage{pgffor}

\makeatletter
\newcommand{\stringforeach}{}
\def\stringforeach#1#2in{%
  \def\@foreachvariable{#1}%
  \def\@foreachoptions{#2}%
  \@ifnextchar\bgroup{\@foreachexplicit}{\@foreachimplicit}%
}
\def\@foreachexplicit#1{\@stringforeachdo{#1}}
\def\@foreachimplicit#1{\expandafter\@stringforeachdo\expandafter{#1}}
\def\@stringforeachdo#1{%
  \@transformstringintoclist{#1}%
  \begingroup\edef\x{%
    \endgroup
    \noexpand\foreach
    \unexpanded\expandafter{\@foreachvariable}%
    \unexpanded\expandafter{\@foreachoptions}%
    in
    \noexpand\@transformedstring
  }\x
}
\makeatother

\ExplSyntaxOn

\cs_new_protected:cpn {@transformstringintoclist} #1
 {
  \tl_set:Nn \l_tmpa_tl { #1 }
  \tl_replace_all:Nnn \l_tmpa_tl { ~ } { \space }
  \seq_set_split:NnV \l_tmpa_seq { } \l_tmpa_tl
  \tl_set:cx { @transformedstring } { \seq_use:Nn \l_tmpa_seq { , } }
 }

\ExplSyntaxOff

\def\mystring{Hello World!}

\begin{document}

\stringforeach \char in \mystring {<\char>}

\stringforeach \char in {Hello World!} {<\char>}

\stringforeach \char [count=\x] in \mystring {<\x=\char>}

\stringforeach \char [count=\x] in {Hello World!} {<\x=\char>}

\end{document}

If you don't need all the \foreach features, it's simpler.

\documentclass{article}
\usepackage[T1]{fontenc}
\usepackage{geometry}

\ExplSyntaxOn

\NewDocumentCommand{\processtokenlist}{smm}
 {
  \IfBooleanTF{#1}
   {% we have a macro
    \sylvain_processtl:Vn #2 { #3 }
   }
   {% we have an explicit token list
    \sylvain_processtl:nn { #2 } { #3 }
   }
 }

\tl_new:N \l__sylvain_processtl_tl

\cs_new_protected:Nn \sylvain_processtl:nn
 {
  \tl_set:Nn \l__sylvain_processtl_tl { #1 }
  \tl_replace_all:Nnn \l__sylvain_processtl_tl { ~ } { \c_space_tl }
  \cs_set_protected:Nn \__sylvain_processtl_item:n { #2 }
  \tl_map_function:NN \l__sylvain_processtl_tl \__sylvain_processtl_item:n
 }
\cs_generate_variant:Nn \sylvain_processtl:nn { V }

\ExplSyntaxOff

\def\mystring{Hello World!}

\begin{document}

\processtokenlist*{\mystring}{<#1>}

\processtokenlist{Hello World!}{<#1>}

\end{document}

Answer 5

Because we cannot absorb single space by unseparated macro parameter, we must pre-process the given macro by \replspaces macro first. Then you can use \insertcommas which inserts commas between token but not after the last one. Or \insertangles which replaces each token by <token>.

\def\afterfi#1#2\fi{\fi#1}
\def\replspaces#1{\edef#1{\expandafter\replspacesA#1 \end}}
\def\replspacesA #1 #2{#1\ifx\end#2\else { }\afterfi{\replspacesA#2}\fi}
\def\insertcommas#1{\edef#1{\expandafter\insertcommasA#1\end}}
\def\insertcommasA#1#2{#1\ifx\end#2\else,\afterfi{\insertcommasA{#2}}\fi}
\def\insertangles#1{\edef#1{\expandafter\insertanglesA#1\end}}
\def\insertanglesA#1{\ifx\end#1\else<#1>\expandafter\insertanglesA\fi}

% test:

\def\mystring{Hello World!}

\replspaces\mystring
\insertcommas\mystring
\meaning\mystring  % macro:-> H,e,l,l,o, ,W,o,r,l,d,!

\def\mystring{Hello World!}

\replspaces\mystring
\insertangles\mystring
\meaning\mystring  % macro:-> <H><e><l><l><o>< ><W><o><r><l><d><!>

\bye

Edit Your first ask was: how to create a macro H,e,l,l,o, ,w,o,r,d,! from the macro Hello World!. I answered correctly.

Now, your second ask is: how to use such macro in \foreach from Tikz. I don't know how to set effectively the expanded parameter to a somewhat impractical \foreach from Tikz (you can use \expanded{\unexpanded {\foreach ...}{\mystring}} but it seems horrible). It should be better not to use \foreach from Tikz at all and define your own \myforeach:

\def\replspaces#1{\edef#1{\expandafter\replspacesA#1 \end}}
\def\replspacesA #1 #2{#1\ifx\end#2\else { }\afterfi{\replspacesA#2}\fi}
\def\myforeach#1{\expandafter\myforeachA#1\end}
\def\myforeachA#1{\ifx\end#1\else\body{#1}\expandafter\myforeachA\fi}

\def\body#1{<#1>}
\def\mystring{Hello World!}

\replspaces\mystring 
\myforeach\mystring % prints: <H><e><l><l><o>< ><W><o><r><l><d><!>

\bye