12

How can I split the string Hello World! - stored into a macro \def\mystring{Hello Word!} - into a comma-separated list of characters (including spaces) which can be used as argument of a \foreach loop (\foreach \char in {\myCSlist}) in order to loop over each character (as with \foreach \char in {H,e,l,l,o,\space,W,o,r,l,d,!})?

EDIT 1 (Mar 21): Why I want that?

At first, I didn't explain why I want the solution generates a comma-separated list of characters (e.g. \myCSlist) which can be used as argument of a \foreach. It's because I want to create a Tikz \node for each character using a pic. Something like this (from here):

\newcommand{\hsp}{.5}
\tikzset{symbols/.pic={%
    \foreach \s[count=\n from 0] in {\myCSlist}{%
    \pgfmathsetmacro{\myangle}{360*rnd}%
    \node[rotate=\myangle] at (\hsp*\n,0){\s};%
}}}%

First attempt

I tried with the \markletters macro of egreg (using xparse and expl3) but the results seems to be not suitable for looping with foreach.

I changed the result print - of the \markletters macro - from (##1) to ##1,, in order to obtain a comma-separated list, but this also results in a unwanted comma , at the end of the comma-separated list.

Here a M(Non)WE:

\documentclass{article}
\usepackage{tikz}% For the \foreach loop
\usepackage{xparse}

% egreg's \markletters macro : https://tex.stackexchange.com/a/359204/262081
\ExplSyntaxOn
\NewDocumentCommand{\markletters}{om}
 {
  \IfNoValueTF{#1}
   {
    \kessels_markletters:nn { #2 } { \tl_use:N \l_kessels_marked_letters_tl }
   }
   {
    \kessels_markletters:nn { #2 } { \tl_set_eq:NN #1 \l_kessels_marked_letters_tl }
   }
 }

\tl_new:N \l_kessels_unmarked_letters_tl
\tl_new:N \l_kessels_marked_letters_tl

\cs_new_protected:Nn \kessels_markletters:nn
 {
  \tl_set:Nn \l_kessels_unmarked_letters_tl { #1 }
  \tl_replace_all:Nnn \l_kessels_unmarked_letters_tl { ~ } { \textvisiblespace }
  \tl_clear:N \l_kessels_marked_letters_tl
  \tl_map_inline:Nn \l_kessels_unmarked_letters_tl
   {
    \tl_put_right:Nn \l_kessels_marked_letters_tl { ##1, }
   }
  #2
 }
\ExplSyntaxOff

\def\mystring{Hello World!}

\begin{document}
\markletters[\foo]{\mystring}%
\foreach \char in {\foo}{%
    <\char>%
}%
\end{document}

This gives the following result:

enter image description here

But I would prefer this result (note the presence of the space between 'Hello' and 'World'):

Enter image description here

6
  • 1
    Are you interested in a LuaLaTeX-based solution? Please advise.
    – Mico
    Commented Mar 20, 2022 at 20:43
  • 1
    If you use lualatex, then you can use a normal looking programming language lua to do this, instead of all this weird \tl_new:N # unreadable stuff :)
    – Nasser
    Commented Mar 20, 2022 at 20:50
  • 1
    @Mico Yes I am (I don't understand Lua or expl3 anyway :)! I don't really care what language is used...
    – zetyty
    Commented Mar 20, 2022 at 21:13
  • 1
    @SylvainRigal - I've posted a LuaLaTeX-based answer that can handle utf8-encoded, as opposed to "just" ascii-encoded, characters in the input string.
    – Mico
    Commented Mar 20, 2022 at 21:46
  • 2
    But it’s "Hello, World!" (not "Hello World!"). Commented Mar 21, 2022 at 12:03

6 Answers 6

7

This can be done using the package xstring. The code also uses \foreach, which is part of tikz but can be loaded with pgffor if your document doesn't use tikz.

enter image description here

\documentclass{article}

\usepackage[T1]{fontenc} % to use accented characters and others
\usepackage{xstring}
\usepackage{pgffor} % in case you're not using tikz

\newcommand{\markletters}[1]{\StrLen{#1}[\strlen]\foreach \chr in {1,...,\strlen}{$<$\StrChar{#1}{\chr}$>$}}

\begin{document}

\markletters{Hello World!}

\markletters{äöü ÄÖÜ // ß}

\end{document}
2
  • +1. How might your code need to be adapted to handle utf8-encoded, as opposed to ascii-encoded, characters?
    – Mico
    Commented Mar 20, 2022 at 21:45
  • 1
    @Mico: Good thought. This code works fine if you add \usepackage[T1]{fontenc}. I updated my answer.
    – Sandy G
    Commented Mar 20, 2022 at 21:52
10

Here's a LuaLaTeX-based solution. It's fully utf8-code aware, i.e., the input string may contain utf8-encoded, as opposed to just ascii-encoded, characters.

The code consists of a Lua function named DoString and a LaTeX utility macros called \DoString, which takes 1 argument -- a character string -- and passes it to the Lua function for further processing.

enter image description here

% !TEX TS-program = lualatex
\documentclass{article}
\directlua{
function DoString ( s )
  for i = 1,string.utflength(s) do 
    tex.sprint ( '<' .. unicode.utf8.sub ( s , i , i ) .. '>' )
  end
end
}
%% Define a LaTeX utility macro:
\newcommand\DoString[1]{\directlua{DoString("#1")}}

\begin{document}
\ttfamily
\DoString{Hello World!}

\DoString{äöü ÄÖÜ // ß}
\end{document}

Addendum to address the OP's follow-up question: To create tikz nodes along the lines suggested in the comment, I suggest you (a) load the luacode package (for the \luaexec macro) and (b) change the \directlua{...} chunk to

\luaexec{
function DoString ( s )
  for i = 1,string.utflength(s) do 
    tex.sprint ( '\\node{' .. unicode.utf8.sub ( s , i , i ) .. '};' )
  end
end
}
8
  • Is there a way to create a Tikz node for each character? I tried something like this: tex.sprint ( '\node{' .. unicode.utf8.sub ( s , i , i ) .. '};' ) but I got an error. At the end I would like to use these nodes in a Tikz pic as shown in this question (see EDIT 4 and look for the \foreach \s[count=\n] in {a,b,c,f} part. I want to replace the {a,b,c,f} list with the output of your macro).
    – zetyty
    Commented Mar 21, 2022 at 12:41
  • @SylvainRigal - Pleas see the addendum I just posted. A major caveat: While the code suggested in the addendum should compile, I have no idea if it'll actually work for you as I don't use TikZ myself.
    – Mico
    Commented Mar 21, 2022 at 13:08
  • 2
    The addendum works like a charm! Thanks a lot!
    – zetyty
    Commented Mar 21, 2022 at 18:27
  • The font seems to be different from the default Latex font. Is there a way to set the default Latex font with your code?
    – zetyty
    Commented Mar 22, 2022 at 13:08
  • 1
    I learn some things, so it wasn't improductive for me. Thank you very much for your reactivity and support!
    – zetyty
    Commented Mar 22, 2022 at 20:08
9

Here I use a token cycle. No mention was made of macros or groups appearing in the input (only characters and spaces), and so those should be excluded from the input, unless the OP advises on how they should be handled.

If one wishes to then use this list in a for-like loop, one can use \foreachitem from the listofitems package, after reading the comma-separated \mystring into a list.

\documentclass{article}
\usepackage{tokcycle,listofitems}
\Characterdirective{\addcytoks{,#1}}
\Spacedirective{\addcytoks{,#1}}
\makeatletter
\newcommand\markletters[1]{%
  \tokcyclexpress{#1}%
  \expandafter\expandafter\expandafter\def
  \expandafter\expandafter\expandafter\mystring
  \expandafter\expandafter\expandafter
  {\expandafter\@gobble\the\cytoks}%
}
\makeatother
\begin{document}
\markletters{Hello World!}

(\detokenize\expandafter{\mystring})

\readlist\mylist\mystring

\foreachitem\z\in\mylist{$<$\z$>$}

\end{document}

enter image description here

Of course, the whole process could be done more simply inside the token cycle, such that listofitems would not even be needed. And, much to my surprise, if I compile under lualatex, it properly handles UTF-8 input

\documentclass{article}
\usepackage{tokcycle}
\newcommand\markletters[1]{%
  \tokencycle{$<$##1$>$}{}{}{$<$##1$>$}#1\endtokencycle%
}
\begin{document}
\markletters{Hello World! äöü ÄÖÜ // ß}
\end{document}

enter image description here

SUPPLEMENT

Based on OP comments, here is a version (based on the first approach) that handles macros and groups. However, macros requiring arguments would have to be handled carefully, since the comma is being inserted after each token/group.

In this implementation, group content is merely echoed as a single entity. If one wanted group content, subdivided by token, tokcycle can do that, but it is not done in this implementation.

\documentclass{article}
\usepackage[T1]{fontenc}
\usepackage{tokcycle,listofitems}
\makeatletter
\stripgroupingtrue
\newcommand\markletters[1]{%
  \tokcycle{\addcytoks{,##1}}{\addcytoks{,}\groupedcytoks{##1}}%
    {\addcytoks{,##1}}{\addcytoks{,##1}}{#1}%
  \expandafter\expandafter\expandafter\def
  \expandafter\expandafter\expandafter\mystring
  \expandafter\expandafter\expandafter
  {\expandafter\@gobble\the\cytoks}%
}
\makeatother
\begin{document}
\markletters{Hello World! \today{} is a great day}

(\detokenize\expandafter{\mystring})

\readlist\mylist\mystring

\foreachitem\z\in\mylist{$<$\z$>$}

\mylist[7] is the 7th token

\detokenize\expandafter\expandafter\expandafter{\mylist[14]}
 is the 14th token
\end{document}

enter image description here

15
  • 1
    @Mico I think I was mistaken. If I compile under lualatex and use \markletters{Hello World! äöü ÄÖÜ // ß}, I get the correct result. Please see my edited answer. Commented Mar 20, 2022 at 21:55
  • 1
    @SylvainRigal Please see the SUPPLEMENT to my answer. Perhaps this gives you more of what you seek. Commented Mar 21, 2022 at 14:09
  • 1
    @SylvainRigal In that case, one needs to expand the macro first, as in \def\z{Hello World!}\expandafter\markletters\expandafter{\z}. Alternately, one could build the expansion into the definition of \markletters, with \newcommand\markletters[1]{\expandafter\tokcyclexpress\expandafter{#1}\expandafter\expandafter\expandafter\def \expandafter\expandafter\expandafter\mystring \expandafter\expandafter\expandafter {\expandafter\@gobble\the\cytoks}} Commented Mar 22, 2022 at 14:30
  • 1
    @SylvainRigal In the 2nd example, \newcommand\markletters[1]{\tokencycle{$<$##1$>$}{}{$<$##1$>$}{$<$##1$>$}#1\endtokencycle} will handle macros that don't require arguments. It will ignore anything in a group { }. Commented Mar 23, 2022 at 13:32
  • 1
    @SylvainRigal \documentclass{article} \usepackage{tokcycle} \newcommand\markletters[1]{% \tokencycle{\addcytoks{$<$##1$>$}}{\addcytoks{##1}} {\addcytoks{##1}}{\addcytoks{$<$##1$>$}}#1\endtokencycle% } \begin{document} \markletters{Hello World \today! äö\rule{1ex}{1ex}ü ÄÖÜ // ß} \end{document} will handle both macros and groups UNMODIFIED, that is, they will not add < > around them. Further, macro arguments must be placed in groups, or the argument will get messed up with additions of < >. See ctan.org/pkg/tokcycle for docs and examples. Commented Mar 23, 2022 at 13:37
6

Here's a \stringforeach command, with the same syntax as \foreach.

\documentclass{article}
\usepackage[T1]{fontenc}
\usepackage{geometry}
\usepackage{pgffor}

\makeatletter
\newcommand{\stringforeach}{}
\def\stringforeach#1#2in{%
  \def\@foreachvariable{#1}%
  \def\@foreachoptions{#2}%
  \@ifnextchar\bgroup{\@foreachexplicit}{\@foreachimplicit}%
}
\def\@foreachexplicit#1{\@stringforeachdo{#1}}
\def\@foreachimplicit#1{\expandafter\@stringforeachdo\expandafter{#1}}
\def\@stringforeachdo#1{%
  \@transformstringintoclist{#1}%
  \begingroup\edef\x{%
    \endgroup
    \noexpand\foreach
    \unexpanded\expandafter{\@foreachvariable}%
    \unexpanded\expandafter{\@foreachoptions}%
    in
    \noexpand\@transformedstring
  }\x
}
\makeatother

\ExplSyntaxOn

\cs_new_protected:cpn {@transformstringintoclist} #1
 {
  \tl_set:Nn \l_tmpa_tl { #1 }
  \tl_replace_all:Nnn \l_tmpa_tl { ~ } { \space }
  \seq_set_split:NnV \l_tmpa_seq { } \l_tmpa_tl
  \tl_set:cx { @transformedstring } { \seq_use:Nn \l_tmpa_seq { , } }
 }

\ExplSyntaxOff

\def\mystring{Hello World!}

\begin{document}

\stringforeach \char in \mystring {<\char>}

\stringforeach \char in {Hello World!} {<\char>}

\stringforeach \char [count=\x] in \mystring {<\x=\char>}

\stringforeach \char [count=\x] in {Hello World!} {<\x=\char>}

\end{document}

enter image description here

If you don't need all the \foreach features, it's simpler.

\documentclass{article}
\usepackage[T1]{fontenc}
\usepackage{geometry}

\ExplSyntaxOn

\NewDocumentCommand{\processtokenlist}{smm}
 {
  \IfBooleanTF{#1}
   {% we have a macro
    \sylvain_processtl:Vn #2 { #3 }
   }
   {% we have an explicit token list
    \sylvain_processtl:nn { #2 } { #3 }
   }
 }

\tl_new:N \l__sylvain_processtl_tl

\cs_new_protected:Nn \sylvain_processtl:nn
 {
  \tl_set:Nn \l__sylvain_processtl_tl { #1 }
  \tl_replace_all:Nnn \l__sylvain_processtl_tl { ~ } { \c_space_tl }
  \cs_set_protected:Nn \__sylvain_processtl_item:n { #2 }
  \tl_map_function:NN \l__sylvain_processtl_tl \__sylvain_processtl_item:n
 }
\cs_generate_variant:Nn \sylvain_processtl:nn { V }

\ExplSyntaxOff

\def\mystring{Hello World!}

\begin{document}

\processtokenlist*{\mystring}{<#1>}

\processtokenlist{Hello World!}{<#1>}

\end{document}

enter image description here

4
  • Your solutions seem to not well handle UTF-8 encoding (ß character seems to be not well printed). It was not requested in my original question but pointed out by @Mico in comment.
    – zetyty
    Commented Mar 22, 2022 at 20:02
  • @SylvainRigal No, currently that's not supported.
    – egreg
    Commented Mar 22, 2022 at 20:39
  • 1
    @SylvainRigal - The code shown in this answer can handle utf8-encoded characters just fine as long as the document is compiled under LuaLaTeX. (No need to load the fontenc package under LuaLaTeX, though.)
    – Mico
    Commented Mar 23, 2022 at 8:30
  • I first tried with fontenc AND LuaLaTeX engine but the "ß" was printed as "SS". Indeed, without fontenc it's OK (if compiled under LuaLaTeX). Thanks.
    – zetyty
    Commented Mar 24, 2022 at 9:40
2

Because we cannot absorb single space by unseparated macro parameter, we must pre-process the given macro by \replspaces macro first. Then you can use \insertcommas which inserts commas between token but not after the last one. Or \insertangles which replaces each token by <token>.

\def\afterfi#1#2\fi{\fi#1}
\def\replspaces#1{\edef#1{\expandafter\replspacesA#1 \end}}
\def\replspacesA #1 #2{#1\ifx\end#2\else { }\afterfi{\replspacesA#2}\fi}
\def\insertcommas#1{\edef#1{\expandafter\insertcommasA#1\end}}
\def\insertcommasA#1#2{#1\ifx\end#2\else,\afterfi{\insertcommasA{#2}}\fi}
\def\insertangles#1{\edef#1{\expandafter\insertanglesA#1\end}}
\def\insertanglesA#1{\ifx\end#1\else<#1>\expandafter\insertanglesA\fi}

% test:

\def\mystring{Hello World!}

\replspaces\mystring
\insertcommas\mystring
\meaning\mystring  % macro:-> H,e,l,l,o, ,W,o,r,l,d,!

\def\mystring{Hello World!}

\replspaces\mystring
\insertangles\mystring
\meaning\mystring  % macro:-> <H><e><l><l><o>< ><W><o><r><l><d><!>

\bye

Edit Your first ask was: how to create a macro H,e,l,l,o, ,w,o,r,d,! from the macro Hello World!. I answered correctly.

Now, your second ask is: how to use such macro in \foreach from Tikz. I don't know how to set effectively the expanded parameter to a somewhat impractical \foreach from Tikz (you can use \expanded{\unexpanded {\foreach ...}{\mystring}} but it seems horrible). It should be better not to use \foreach from Tikz at all and define your own \myforeach:

\def\replspaces#1{\edef#1{\expandafter\replspacesA#1 \end}}
\def\replspacesA #1 #2{#1\ifx\end#2\else { }\afterfi{\replspacesA#2}\fi}
\def\myforeach#1{\expandafter\myforeachA#1\end}
\def\myforeachA#1{\ifx\end#1\else\body{#1}\expandafter\myforeachA\fi}

\def\body#1{<#1>}
\def\mystring{Hello World!}

\replspaces\mystring 
\myforeach\mystring % prints: <H><e><l><l><o>< ><W><o><r><l><d><!>

\bye
6
  • If I do something like this \replspaces\mystring \insertcommas\mystring \foreach \char in {\mystring}{<\char>}, it doesn't loop over each character in the string, I just get this: <H,e,l,l,o, ,W,o,r,l,d,!>. Am I doing something wrong?
    – zetyty
    Commented Mar 22, 2022 at 8:04
  • My first ask was: How to create a macro H,e,l,l,o, ,w,o,r,d,! from the macro Hello World! which can be used as an \foreach argument? I'm sincerely sorry if it was not clear. I'm sure there are many other way to do it but the \foreach syntax is easy to use for me and I can acces easily to the loop integer or start it from 0. Anyway, your solution works fine thank you.
    – zetyty
    Commented Mar 23, 2022 at 10:07
  • @SylvainRigal The problem is that the parameter for Tikz's \foreach must be expanded before the \foreach is processed. But you cannot use single \expandafter because the parameter is due to the inappropriately \foreach syntax positioned at the very back. I don't want to study the 1400 pages manual of Tikz in order to know, how to do it using Tikz concept. It is more simple for me to write my own \foreach macro. I hoped that you know how to expand the parameter for \foreach from Tikz if you want to use it and that this is not the main problem asked here.
    – wipet
    Commented Mar 23, 2022 at 15:18
  • You're right is not the main problem here... But I disturb you again because I tried to used your myforeach with this: \newcounter{cnt}\setcounter{cnt}{-1}\def\body#1{\stepcounter{cnt}\pgfmathsetmacro{\myangle}{360*rnd}\pgfmathsetmacro{\mystep}{\thecnt}\pgfmathsetmacro{\myangle}{360*rnd}\node[rotate=\myangle] at (\hsp*\mystep,0){#1};}% but I got the following error: ! Undefined control sequence. \replspacesA ...->#1\ifx \end #2\else { }\afterfi{\replspacesA #2}\fi.
    – zetyty
    Commented Mar 24, 2022 at 10:28
  • Undefined control sequence \afterfi? Maybe there is not copied the definition of \afterfi from the firs line of my first example. Ii is my mistake that I didn't copied it from my first example to the second.
    – wipet
    Commented Mar 24, 2022 at 14:53
1

Since you're already using PGF/TikZ you might as well use the parser module that comes with it.

We'll define a parser char parser that does something (/char parser/do) with each letter, for this we need three actions:

  1. the one for the space (it's special),
  2. the one for every other character and
  3. for the end of the parsing.

I'm using a @-protected macro \charparser@stop to end the parsing since any other letter (like ;) could show up in the text that shall be parsed.

The action for every other character is done by defining an action for an unknown letter – unknown except for the space or for the ending. The parser would actually raise an error in that case but we can set char parser/silent=true.

And then it's just a question about what you want to do with every letter.

We can place a node for it:

\tikz[
  /char parser/do/.code={\node[rotate=360*rnd] at (\charparsercount,0) {#1};},
  /char parser/parse={Hello World! äöüßÄÖÜ}
]{}

Yes, the parser increments a macro (not a counter) \charparsercount. The key do will be used with the found character.

We can also make a comma-separated list and use it with \foreach:

\tikz[
  /char parser/list=\myList,
  /char parser/parse={Hello World! äöüßÄÖÜ}%
]
\foreach \s[count=\n from 0] in \myList
  \node[rotate=360*rnd] at (\n,0) {\s};

If you have the to-be-parsed text in a macro just use /.expand once:

\newcommand*\myText{Hello World! äöüßÄÖ}
/char parser/parse/.expand once=\myText

Code

% use lualatex
\documentclass[tikz]{standalone}
\newcommand*\charparserset{\pgfqkeys{/char parser}}
\usepgfmodule{parser}
\pgfparserdefunknown{char parser}{initial}{%
  \pgfkeysalso{/char parser/do/.expand once=\pgfparserletter,/char parser/next}}
\pgfparserdef{char parser}{initial}{blank space}{%
  \pgfkeysalso{/char parser/do=\space,/char parser/next}}
\pgfparserset{char parser/silent=true}
\makeatletter
\pgfparserdef{char parser}{initial}\charparser@stop{%
  \pgfkeysalso{/char parser/final/.try}\pgfparserswitch{final}}
\charparserset{parse/.code={%
    \charparserset{init}%
    \pgfparserparse{char parser}#1\charparser@stop}}
\makeatother
\charparserset{
  init/.code=\def\charparsercount{0},
  next/.code=\edef\charparsercount{\the\numexpr\charparsercount+1\relax},
  list/.style={
    /char parser/final/.append code=\pgfkeysgetvalue{/char parser/@list}#1,
    /char parser/@list/.initial=,
    /char parser/do/.style={
      /char parser/@list/.append={##1},
      /char parser/do/.style={
        /char parser/@list/.append={,####1}}}},
  do/.code={#1}}
\begin{document}
\tikz[
  x=5mm,
  /char parser/do/.code={\node[rotate=360*rnd] at (\charparsercount,0) {#1};},
  /char parser/parse={Hello World! äöüßÄÖÜ}
]{}
\tikz[
  x=5mm,
  /char parser/list=\myList,
  /char parser/parse={Hello World! äöüßÄÖÜ}%
]
\foreach \s[count=\n from 0] in \myList
  \node[rotate=360*rnd] at (\n,0) {\s};
\end{document}

Output

enter image description here

1

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .