3

I want to write a macro that will separate the characters in a given string with some condition. If I have a string abcdef<ghi>jkl<mn>opqrs I want to access each character as a string for further use in another command but ghi should be one string similarly mn should be one string. Could you please help me in achieving this?

Apologies for the following addition to the question, I thought the general answer would work in Xelatex with Devanagari characters but it seems it doesn't. Hence the edit.

my input string will be Devanagari characters and I will run Xelatex

e.g. my string will be of the form सा{परेग}नी{धम}पनी

Also, I am getting this string via operations on another string by using Xstring package, I am doing the following extraction

\StrBetween{(सा{परे}न{धम}पनी)[नी{धम}पनी]}{(}{)}[\firststring] now I want to use \firststring as input.

  • 3
    are your character a-z or accented or non latin characters as well and if so are you using pdftex or a unicode tex such as luatex, the handling of characters is completely different in such systems. Also are you forced to use <ghi> or could you use the syntax {ghi} (which would simplify things quite a bit) – David Carlisle May 6 '20 at 11:45
  • 2
    It is a bit unclear to me what your goal and use case is exactly. Could you provide a small example document with a call to a macro with some argument, for which you explain what the input is, what the output is, and the general procedure of converting the input into the output? – Marijn May 6 '20 at 11:53
  • Will there be unbalanced < or > also? What about nesting <..>, e.g., ab<<cde<fg>hi>jk>lm? – Ulrich Diez May 6 '20 at 17:49
  • @DavidCarlisle actually I would prefer {ghi} – Girish May 10 '20 at 17:19
  • @UlrichDiez There will be no nesting – Girish May 10 '20 at 17:20
7

Here, I use tokcycle to process the tokens of the original input and add a , after each character token, unless it is inside a <...> grouping. Then, I can use listofitems to read this comma-separated list, storing each of the items in an array \mystring. This array is accessible by the index of the item.

If you need to access the actual tokens of each array element, the call to \mystring[...] needs to be twice-expanded.

\documentclass{article}
\usepackage{listofitems,tokcycle}
\newcounter{nestlevel}
\Characterdirective{\ifx<#1\stepcounter{nestlevel}\else
  \ifx>#1\addtocounter{nestlevel}{-1}\addcytoks{,}\else
  \addcytoks{#1}\ifnum\thenestlevel=0\relax\addcytoks{,}\fi\fi\fi}
\ignoreemptyitems
\gdef\getmystring#1{%
  \setcounter{nestlevel}{0}%
  \tokcyclexpress{#1}%
  \expandafter\readlist\expandafter\mystring\expandafter{\the\cytoks}}
\begin{document}
\getmystring{abcdef<ghi>jkl<mn>opqrs}
The number of strings is \mystringlen.

\mystring[3], \mystring[6], \mystring[7], \mystring[11], \mystring[16]
\end{document}

enter image description here

If one were to allow brace grouping {...} of the input stream, rather than angle-bracket grouping <...>, the construction would be a bit simpler, to get the same result:

\documentclass{article}
\usepackage{listofitems,tokcycle}
\stripgroupingtrue
\ignoreemptyitems
\newcommand\getmystring[1]{%
  \tokcycle{\addcytoks{##1,}}{\addcytoks{##1,}}{}{}{#1}%
  \expandafter\readlist\expandafter\mystring\expandafter{\the\cytoks}}
\begin{document}
\getmystring{abcdef{ghi}jkl{mn}opqrs}
The number of strings is \mystringlen.

\mystring[3], \mystring[6], \mystring[7], \mystring[11], \mystring[16]
\end{document}

SUPPLEMENT

The OP comments that he would like to use this approach where the input is provided, not directly, but by way of a \defed string. Here, I provide \getmydefstring to do that:

\documentclass{article}
\usepackage{listofitems,tokcycle}
\stripgroupingtrue
\ignoreemptyitems
\newcommand\getmystring[1]{%
  \tokcycle{\addcytoks{##1,}}{\addcytoks{##1,}}{}{}{#1}%
  \expandafter\readlist\expandafter\mystring\expandafter{\the\cytoks}}
\newcommand\getmydefstring[1]{%
  \def\tmp{\tokcycle{\addcytoks{####1,}}{\addcytoks{####1,}}{}{}}
  \expandafter\tmp\expandafter{#1}%
  \expandafter\readlist\expandafter\mystring\expandafter{\the\cytoks}}
\begin{document}
\def\mystring{abcdef{ghi}jkl{mn}opqrs}
\getmydefstring{\mystring}
The number of strings is \mystringlen.

\mystring[3], \mystring[6], \mystring[7], \mystring[11], \mystring[16]
\end{document}
  • Thank you very much for your answer, I want to run this in xelatex with Devanagari character strings. I am raking input as a string extracted via xstring package, e.g. \StrBetween{(abcd{ef}gh)[pqrst]}{(}{)}[\firststring] now I want to use \firststring as input, again everything in Devanagari characters. How can I make it work in that case? Currently, this is not working in that scenario. – Girish May 10 '20 at 17:36
  • @Girish I'm sorry, but I very little experience with Unicode characters and also with xstring. I am likely of little help. – Steven B. Segletes May 10 '20 at 17:36
  • @Girish See my SUPPLEMENT, to operate on a string supplied in a \defed macro. – Steven B. Segletes May 11 '20 at 9:33
3

If you don't mind processing your string as a list of undelimited arguments or explicit space tokens, I can offer a macro \ExtractKthArgOrSpace.

Be aware that undelimited arguments

  • either consist of a single token which neither is an explicit space-token, nor is an explicit character-token of category code 1 (begin group), like {1, or 2 (end group), like }2,
  • or consist of a curly-brace-balanced set of tokens nested in curly braces.

So instead of < and > you use { and }.

When using this for processing utf8-encoded .tex-input by means of the inputenc-package while using a traditional TeX-engine where the internal character-encoding-scheme is 8-bit ASCII, then unicode-characters/sub-strings containing unicode-characters need to be nested in curly braces. This is because on such systems a single unicode-character in the .tex-input will be taken for a sequence of byte-encoded/8-bit-encoded characters yielding a sequence of several character-tokens, the first one being active and "looking at subsequent ones for deciding what glyphs to deliver to the output-file".

\makeatletter
%% Code for \ExtractKthArgOrSpace
%%=============================================================================
%% Paraphernalia:
%%    \UD@firstoftwo, \UD@secondoftwo, \UD@PassFirstToSecond, \UD@Exchange,
%%    \UD@CheckWhetherNull
%%=============================================================================
\newcommand\UD@firstoftwo[2]{#1}%
\newcommand\UD@secondoftwo[2]{#2}%
\newcommand\UD@PassFirstToSecond[2]{#2{#1}}%
\newcommand\UD@Exchange[2]{#2#1}%
%%-----------------------------------------------------------------------------
%% Check whether argument is empty:
%%.............................................................................
%% \UD@CheckWhetherNull{<Argument which is to be checked>}%
%%                     {<Tokens to be delivered in case that argument
%%                       which is to be checked is empty>}%
%%                     {<Tokens to be delivered in case that argument
%%                       which is to be checked is not empty>}%
%%
%% The gist of this macro comes from Robert R. Schneck's \ifempty-macro:
%% <https://groups.google.com/forum/#!original/comp.text.tex/kuOEIQIrElc/lUg37FmhA74J>
\newcommand\UD@CheckWhetherNull[1]{%
  \romannumeral0\expandafter\UD@secondoftwo\string{\expandafter
  \UD@secondoftwo\expandafter{\expandafter{\string#1}\expandafter
  \UD@secondoftwo\string}\expandafter\UD@firstoftwo\expandafter{\expandafter
  \UD@secondoftwo\string}\expandafter\expandafter\UD@firstoftwo{ }{}%
  \UD@secondoftwo}{\expandafter\expandafter\UD@firstoftwo{ }{}\UD@firstoftwo}%
}%
%%=============================================================================
%% Check whether brace-balanced argument's first token is a space-token
%%=============================================================================
%% \UD@CheckWhetherLeadingSpace{<Argument which is to be checked>}%
%%                             {<Tokens to be delivered in case <argument
%%                               which is to be checked>'s 1st token is a
%%                               space-token>}%
%%                             {<Tokens to be delivered in case <argument
%%                               which is to be checked>'s 1st token is not
%%                               a space-token>}%
\newcommand\UD@CheckWhetherLeadingSpace[1]{%
  \romannumeral0\UD@CheckWhetherNull{#1}%
  {\UD@firstoftwo\expandafter{} \UD@secondoftwo}%
  {\expandafter\UD@secondoftwo\string{\UD@@CheckWhetherLeadingSpace.#1 }{}}%
}%
\@ifdefinable\UD@@CheckWhetherLeadingSpace{%
  \long\def\UD@@CheckWhetherLeadingSpace#1 {%
    \expandafter\UD@CheckWhetherNull\expandafter{\UD@secondoftwo#1{}}%
    {\UD@Exchange{\UD@firstoftwo}}{\UD@Exchange{\UD@secondoftwo}}%
    {\UD@Exchange{ }{\expandafter\expandafter\expandafter\expandafter
     \expandafter\expandafter\expandafter}\expandafter\expandafter
     \expandafter}\expandafter\UD@secondoftwo\expandafter{\string}%
  }%
}%
%%=============================================================================
%% Remove a space-token
%%=============================================================================
\@ifdefinable\UD@gobblespace{\UD@firstoftwo{\def\UD@gobblespace}{} {}}%
%%=============================================================================
%% Extract K-th element of list of undelimited arguments or spaces:
%%
%% \ExtractKthArgOrSpace{<integer K>}{<list of undelimited args or spaces>} 
%% 
%% In case there is no K-th element in <list of undelimited args or spaces> : 
%%   Does not deliver any token.
%% In case there is a K-th element in <list of undelimited args or spaces> : 
%%   Does deliver that K-th element with one level of surrounding curly 
%%   braces removed if present.
%%
%% Examples:
%%
%%   \ExtractKthArgOrSpace{0}{ABCDE} yields: <nothing>
%%
%%   \ExtractKthArgOrSpace{3}{ABCDE} yields:  C
%%
%%   \ExtractKthArgOrSpace{3}{AB{CD}E} yields:  CD
%%
%%   \ExtractKthArgOrSpace{3}{AB DE} yields: <space token>
%%
%%   \ExtractKthArgOrSpace{2}{{AB} {DE}F} yields: <space token>
%%
%%   \ExtractKthArgOrSpace{4}{{001}{002}{003}{004}{005}} yields: 004
%%
%%   \ExtractKthArgOrSpace{6}{{001}{002}{003}} yields: <nothing> 
%%
%% Due to \romannumeral0-expansion the result is delivered after two expansion-
%% steps/after two "hits" by \expandafter.
%% 
%%=============================================================================
\newcommand\ExtractKthArgOrSpace[1]{%
  \romannumeral0%
  % #1: <integer number K>
  \expandafter\UD@ExtractKthArgOrSpaceCheck
  \expandafter{\romannumeral\number\number#1 000}%
}%
\newcommand\UD@ExtractKthArgOrSpaceCheck[2]{%
  \UD@CheckWhetherNull{#1}{ }{%
    \expandafter\UD@ExtractKthArgOrSpaceLoop\expandafter{\UD@firstoftwo{}#1}{#2}%
  }%
}%
\newcommand\UD@ExtractKthArgOrSpaceLoop[2]{%
  \UD@CheckWhetherNull{#2}{ }{%
    \UD@CheckWhetherNull{#1}{%
      \UD@CheckWhetherLeadingSpace{#2}{%
        \UD@ExtractFirstArgLoop{{ }#2\UD@SelDOm}%
      }{%
        \UD@ExtractFirstArgLoop{#2\UD@SelDOm}%
      }%
    }{%
      \UD@CheckWhetherLeadingSpace{#2}{%
        \expandafter\UD@PassFirstToSecond\expandafter{\UD@gobblespace#2}%
      }{%
        \expandafter\UD@PassFirstToSecond\expandafter{\UD@firstoftwo{}#2}%
      }%
      {\expandafter\UD@ExtractKthArgOrSpaceLoop\expandafter{\UD@firstoftwo{}#1}}%
    }%
  }%
}%
\newcommand\UD@RemoveTillUD@SelDOm{}%
\long\def\UD@RemoveTillUD@SelDOm#1#2\UD@SelDOm{{#1}}%
\newcommand\UD@ExtractFirstArgLoop[1]{%
  \expandafter\UD@CheckWhetherNull\expandafter{\UD@firstoftwo{}#1}%
  {\UD@firstoftwo{\expandafter}{} \UD@secondoftwo{}#1}%
  {\expandafter\UD@ExtractFirstArgLoop\expandafter{\UD@RemoveTillUD@SelDOm#1}}%
}%
%% End of code for \ExtractKthArgOrSpace.
\makeatother

\documentclass{article}
\begin{document}

\noindent The list of undelimited arguments or space-tokens is: \verb|abcdef{ghi}jkl{mn}opqr s|

\bigskip

\noindent
\verb|\ExtractKthArgOrSpace{1}{abcdef{ghi}jkl{mn}opqr s}|: \ExtractKthArgOrSpace{1}{abcdef{ghi}jkl{mn}opqr s}\\
\verb|\ExtractKthArgOrSpace{2}{abcdef{ghi}jkl{mn}opqr s}|: \ExtractKthArgOrSpace{2}{abcdef{ghi}jkl{mn}opqr s}\\
\verb|\ExtractKthArgOrSpace{3}{abcdef{ghi}jkl{mn}opqr s}|: \ExtractKthArgOrSpace{3}{abcdef{ghi}jkl{mn}opqr s}\\
\verb|\ExtractKthArgOrSpace{4}{abcdef{ghi}jkl{mn}opqr s}|: \ExtractKthArgOrSpace{4}{abcdef{ghi}jkl{mn}opqr s}\\
\verb|\ExtractKthArgOrSpace{5}{abcdef{ghi}jkl{mn}opqr s}|: \ExtractKthArgOrSpace{5}{abcdef{ghi}jkl{mn}opqr s}\\
\verb|\ExtractKthArgOrSpace{6}{abcdef{ghi}jkl{mn}opqr s}|: \ExtractKthArgOrSpace{6}{abcdef{ghi}jkl{mn}opqr s}\\
\verb|\ExtractKthArgOrSpace{7}{abcdef{ghi}jkl{mn}opqr s}|: \ExtractKthArgOrSpace{7}{abcdef{ghi}jkl{mn}opqr s}\\
\verb|\ExtractKthArgOrSpace{8}{abcdef{ghi}jkl{mn}opqr s}|: \ExtractKthArgOrSpace{8}{abcdef{ghi}jkl{mn}opqr s}\\
\verb|\ExtractKthArgOrSpace{9}{abcdef{ghi}jkl{mn}opqr s}|: \ExtractKthArgOrSpace{9}{abcdef{ghi}jkl{mn}opqr s}\\
\verb|\ExtractKthArgOrSpace{10}{abcdef{ghi}jkl{mn}opqr s}|: \ExtractKthArgOrSpace{10}{abcdef{ghi}jkl{mn}opqr s}\\
\verb|\ExtractKthArgOrSpace{11}{abcdef{ghi}jkl{mn}opqr s}|: \ExtractKthArgOrSpace{11}{abcdef{ghi}jkl{mn}opqr s}\\
\verb|\ExtractKthArgOrSpace{12}{abcdef{ghi}jkl{mn}opqr s}|: \ExtractKthArgOrSpace{12}{abcdef{ghi}jkl{mn}opqr s}\\
\verb|\ExtractKthArgOrSpace{13}{abcdef{ghi}jkl{mn}opqr s}|: \ExtractKthArgOrSpace{13}{abcdef{ghi}jkl{mn}opqr s}\\
\verb|\ExtractKthArgOrSpace{14}{abcdef{ghi}jkl{mn}opqr s}|: \ExtractKthArgOrSpace{14}{abcdef{ghi}jkl{mn}opqr s}\\
\verb|\ExtractKthArgOrSpace{15}{abcdef{ghi}jkl{mn}opqr s}|: \ExtractKthArgOrSpace{15}{abcdef{ghi}jkl{mn}opqr s}\\
Let's put the following into parentheses to make the space visible:\\
\verb|(\ExtractKthArgOrSpace{16}{abcdef{ghi}jkl{mn}opqr s})|: (\ExtractKthArgOrSpace{16}{abcdef{ghi}jkl{mn}opqr s})\\
\verb|\ExtractKthArgOrSpace{17}{abcdef{ghi}jkl{mn}opqr s}|: \ExtractKthArgOrSpace{17}{abcdef{ghi}jkl{mn}opqr s}

\bigskip

\noindent Space-tokens are taken into account, but be aware that with \LaTeX's tokenizer
consecutive spaces appearing in the .tex-input-file usually collapse into a single explicit space-token
if not skipped completely due to appearing behind something that got tokenized as control-word-token:

\bigskip

\noindent
\verb|(\ExtractKthArgOrSpace{1}{ })|: (\ExtractKthArgOrSpace{1}{ })\\
\verb|(\ExtractKthArgOrSpace{1}{ abc})|: (\ExtractKthArgOrSpace{1}{ abc})\\
\verb|(\ExtractKthArgOrSpace{1}{{ }abc})|: (\ExtractKthArgOrSpace{1}{{ }abc})\\
\verb|(\ExtractKthArgOrSpace{3}{ab c})|: (\ExtractKthArgOrSpace{3}{ab c})\\
\verb|(\ExtractKthArgOrSpace{3}{ab{ }c})|: (\ExtractKthArgOrSpace{3}{ab{ }c})

\bigskip

\noindent
Now let's define a macro after hitting \verb|\ExtractKthArgOrSpace| with \verb|\expandafter| twice:

\begin{verbatim}
\expandafter\expandafter
\expandafter            \newcommand
\expandafter\expandafter
\expandafter            \test
\expandafter\expandafter
\expandafter            {%
   \ExtractKthArgOrSpace{7}{abcdef{ghi}jkl{mn}opqr s}%
}%

\texttt{\meaning\test}
\end{verbatim}

\expandafter\expandafter
\expandafter            \newcommand
\expandafter\expandafter
\expandafter            \test
\expandafter\expandafter
\expandafter            {%
   \ExtractKthArgOrSpace{7}{abcdef{ghi}jkl{mn}opqr s}%
}%

\noindent
\texttt{\meaning\test}
\end{document}

enter image description here

2

In a comment you indicated that using {} rather than <> was OK.

That makes it trivial as latex has such a loop so you need no packages, just a command to define what to do on each iteration in this case, here I just surround with [] and end the paragraph each time.

enter image description here

\documentclass{article}

\makeatletter
\newcommand\myloop[1]{\@tfor\zzz:=#1\do{[\zzz]\par}}
\makeatother

\begin{document}
\myloop{abcdef{ghi}jkl{mn}opqrs}
\end{document}

Or if you want to save the items, just modify teh command in the loop

enter image description here

\documentclass{article}

\makeatletter
\newcount\mycount
\newcommand\myloop[1]{%
\mycount=0 %
\@tfor\zzz:= #1\do{%
\advance\mycount 1 %
\expandafter\edef\csname zzz\the\mycount\endcsname{\zzz}%
}}
\newcommand\myuse[1]{\csname zzz#1\endcsname}

\makeatother

\begin{document}
\myloop{abcdef{ghi}jkl{mn}opqrs}

7th item is \myuse{7}

11th item is \myuse{11}

13th item is \myuse{13}

\end{document}
  • thank you. How do I access each character group separately, also does this work for Unicode characters? – Girish May 10 '20 at 17:41
  • @Girish unicode if you use luatex or xetex not if you use pdftex. not sure what you mean by access separately the loop variable \zzz here is defined to be the character or group at each iteration. You can save them for future use if you need. or just use each one in the loop. – David Carlisle May 10 '20 at 17:44
  • Yes I am using xelatex. But this code produces no output. I want to access 3 character group or 2nd for example from the given string in further code. – Girish May 10 '20 at 17:47
  • @Girish I''l put a version that saves each iteration – David Carlisle May 10 '20 at 17:50
  • @Girish try now – David Carlisle May 10 '20 at 17:56
1

I show the scanner of tokens string based on the loop which use \isnextchar char{true}{false}. We are asking to < or to space token, then there is two \isnextchars nested. The \isnextchar is defined here first (borrowed from OPmac) and we can try this macro by pdftex:

\newcount\tmpnum

\long\def\isnextchar#1#2#3{\begingroup\toks0={\endgroup#2}\toks1={\endgroup#3}%
   \let\tmp= #1\futurelet\next\isnextcharA}
\def\isnextcharA{\the\toks\ifx\tmp\next0\else1\fi\space}

\def\readtext#1{\tmpnum=0 \readtextA #1\end}
\def\readtextA{\advance\tmpnum by1
   \isnextchar<{\readtextB}{\isnextchar{ }{\readtextC}{\readtextD}}}
\def\readtextB<#1>{\savetextitem{#1}\readtextA}
\def\readtextC{\savetextitem{ }\afterassignment\readtextA \let\next= }
\def\readtextD#1{\ifx\end#1\advance\tmpnum by-1 \else \afterfi \readtextB<#1>\fi}
\def\afterfi#1\fi{\fi#1}

\def\savetextitem#1{\expandafter\def \csname t:\the\tmpnum\endcsname{#1}}
\def\textitem[#1]{\csname t:#1\endcsname}

test: \readtext{ab cdef<ghi>jkl<mn>op qrs}

      \textitem[3], \textitem[6], \textitem[8], \textitem[11], \textitem[16].
1

It's a two-liner with expl3.

\documentclass{article}
\usepackage{xparse}

\ExplSyntaxOn

\NewExpandableDocumentCommand{\getstringitem}{mm}
 {
  \tl_item:en { #1 } { #2 }
 }
\cs_generate_variant:Nn \tl_item:nn { e }

\ExplSyntaxOff

\begin{document}

\getstringitem{abcdef{ghi}jkl{mn}opqrs}{1},
\getstringitem{abcdef{ghi}jkl{mn}opqrs}{2},
\getstringitem{abcdef{ghi}jkl{mn}opqrs}{3},
\getstringitem{abcdef{ghi}jkl{mn}opqrs}{4},
\getstringitem{abcdef{ghi}jkl{mn}opqrs}{5},
\getstringitem{abcdef{ghi}jkl{mn}opqrs}{6},
\getstringitem{abcdef{ghi}jkl{mn}opqrs}{7},
\getstringitem{abcdef{ghi}jkl{mn}opqrs}{8},
\getstringitem{abcdef{ghi}jkl{mn}opqrs}{9},
\getstringitem{abcdef{ghi}jkl{mn}opqrs}{10},
\getstringitem{abcdef{ghi}jkl{mn}opqrs}{11},
\getstringitem{abcdef{ghi}jkl{mn}opqrs}{12},
\getstringitem{abcdef{ghi}jkl{mn}opqrs}{13},
\getstringitem{abcdef{ghi}jkl{mn}opqrs}{14},
\getstringitem{abcdef{ghi}jkl{mn}opqrs}{15},
\getstringitem{abcdef{ghi}jkl{mn}opqrs}{16}

\getstringitem{\getstringitem{abcdef{ghi}jkl{mn}opqrs}{7}}{2} should be h

\newcommand{\mystring}{a{bcde}{klm}yz}

\getstringitem{\mystring}{1},
\getstringitem{\mystring}{2},
\getstringitem{\mystring}{3},
\getstringitem{\mystring}{4},
\getstringitem{\mystring}{5}

\end{document}

Note that you can even nest the calls and use strings stored in macros.

enter image description here

You can get the last element by

\getstringitem{\mystring}{-1}

(also an explicit string works, of course).

  • Thank you for your answer, I want to run this in xelatex with Devanagari character strings. How can I make it work in that case? Currently, this is not working in that scenario. – Girish May 10 '20 at 17:22
  • @Girish Without a real example of code I can't help more than this. Please, open a new question with some more details about the actual problem. – egreg May 10 '20 at 19:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.