How to get what is before the next '[' or ']'

Question

Given next LaTeX source code

\mymacro blablabla[ lorem ipsum
\mymacro blablabla[ lorem ipsum ] etc
\mymacro blablabla] lorem ipsum
\mymacro blablabla] lorem ipsum [ etc

How would I define \mymacro with 2 args such that #1 is blablabla, and #2 is either '[' in the former cases or ']' in the latter.

Notice that splitting in 2 different macros, one for '[' and another one for ']' is not an option. That would be too easy.

Nota Bene: Examples were updated to more closely reflect the problem stated in the title.

This question may be strange because of the unusual latex usage, the original problem is to import in a LaTeX document some text material created by a third party tool.

will you always have ..]..[.. or ..[..].. or could you have cases with just one? — David Carlisle, Nov 30 '20 at 22:33
If you are talking optional arguments, the xparse package supports having multiple and out of order optional arguments. The old school solution involves \ifnextchar[ — John Kormylo, Dec 1 '20 at 0:16
In TeX we almost always have to split things into multiple macros if we want to do token-by-token parsing, as TeX itself doesn't have any idea of optional arguments. Do you mean we are only allowed one interface macro or do you want only one macro definition (\def) statement? — Joseph Wright♦, Dec 1 '20 at 12:02
What about things like \mymacro bla{[bla bla]}bla[ lorem ipsum or \mymacro bla{]bla bla]}bla] lorem ipsum ? I.e., what about square brackets nested inside curly-brace-groups ? — Ulrich Diez, Dec 1 '20 at 15:18
I recommend to present the "original problem" as well. The problem presented here and the "original problem" are different problems. Different problems may have different sets of solutions. — Ulrich Diez, Dec 4 '20 at 23:41

Steven B. Segletes · Accepted Answer · 2020-12-08 19:58:27Z

The OP poses a very strange syntax, with ungrouped arguments. But here, I use a token cycle to achieve the desired output. The macro \mymacroaux is where you need to specify what to do with the arguments...here, I just echo them so that one can see they were digested properly.

\documentclass{article}
\usepackage{tokcycle,txfonts}
\makeatletter\let\gobble\@gobble\makeatother
\Characterdirective{%
  \aftertokcycle{\expandafter\mymacroaux\expandafter{\the\cytoks}{#1}}%
  \tctestifx{]#1}{\expandafter\endtokcycraw\gobble}{%
    \tctestifx{[#1}{\expandafter\endtokcycraw\gobble}{\addcytoks{#1}}}%
}
\def\mymacroaux#1#2{(\#1 is ``#1'' and \#2 is ``#2'')}
\let\mymacro\tokencyclexpress
\begin{document}
\mymacro blablabla[ lorem ipsum

\mymacro blablabla[ lorem ipsum ] etc

\mymacro blablabla] lorem ipsum

\mymacro blablabla] lorem ipsum [ etc
\end{document}

SUPPLEMENT

Here, I tried to make it even more user-friendly to write your own token intercept routines using tokcycle.

I have introduced a macro

\abortiftokenis{<test token>}{<command if not test token>}

which can be nested, to screen more than one token, such as [ and ]. One must have some familiarity with the tokcycle approach, in which tokens in the input stream are shunted off to one of four directives, Character, Group, Macro, or Space. Thus, trapping macros (commands) must occur in the \Macrodirective, spaces in the \Spacedirective, and normal characters in the \Characterdirective. The \Groupdirective is set up to pass through its content, as one cannot properly break out of the token cycle if submerged in a group.

The macro \aborttokcycle is separately defined, if one wishes to bail out of the tokencycle for reasons other than a matching token. So you will notice in this new version, the directives are much more streamlined as a result.

In this MWE, I will branch to the \mymacroaux handler upon finding a [, ], \today or a space in the top level content (but not within a group). Recall, this handler routine takes two arguments: the tokens leading up to the trapped token and the trapped token that caused the token-cycle exit.

\documentclass{article}
\usepackage[T1]{fontenc}
\usepackage{tokcycle,txfonts}
\makeatletter
\def\aborttokcycle{\expandafter\endtokcycraw\@gobble}
\def\abortiftokenis#1{%
  \aftertokcycle{\expandafter\mymacroaux\expandafter{\the\cytoks}{#1}}%
  \tctestifx{\tc@next#1}{\aborttokcycle}%
}
\makeatother
\Characterdirective{\abortiftokenis{]}{\abortiftokenis{[}{\addcytoks{#1}}}}
\Groupdirective{\addcytoks{#1}}
\Macrodirective{\abortiftokenis{\today}{\addcytoks{#1}}}
\Spacedirective{\abortiftokenis{ }{\addcytoks{#1}}}
\def\mymacroaux#1#2{(\#1 is ``\detokenize{#1}'' and \#2 is ``\detokenize{#2}'')}
\let\mymacro\tokencyclexpress
\begin{document}
\mymacro b{la b}labla[ lorem ipsum

\mymacro blab\relax labla] lorem ipsum ] etc

\mymacro blab\today labla] lorem ipsum [ etc

\mymacro blabla bla] lorem ipsum
\end{document}

egreg · Accepted Answer · 2020-12-08 13:19:17Z

With an up-to-date LaTeX system, you can use \peek_regex_replace_once:nn

\documentclass{article}

\ExplSyntaxOn
\NewDocumentCommand{\mymacro}{}
 {
  \peek_regex_replace_once:nn { ([^\[\]]*) (\[|\]) } { \c{innermymacro}\cB\{\1\cE\}\2 }
 }
\ExplSyntaxOff

\newcommand{\innermymacro}[2]{%
  First argument: ``#1''; second argument: ``#2''%
}


\begin{document}

\mymacro blablabla[ lorem ipsum

\mymacro bla bla bla[ lorem ipsum ] etc

\mymacro blablabla] lorem ipsum

\mymacro bla bla bla] lorem ipsum [ etc

\end{document}

Explanation:

the regular expression ([^\[\]]*) (\[|\]) looks for all tokens until finding either a [] or a ], saving the finding as \1 (the tokens up to the bracket) and \2 (the bracket);
the replacement text is \c{innermymacro}\cB\{\1\cE\}\2, which means "put \myinnermacro, a brace {, the tokens represented by \1, a brace } and the tokens represented by \2 (in this case a single token, either [ or ]);
next the processing will restart from \myinnermacro that can use its two given arguments.

Once again regular expressions allow a nice and clean answer. Since when xparse is no longer a required package ? — Jérôme LAURENS, Dec 8 '20 at 13:50
@JérômeLAURENS From release 2020-10-01. The addition of \peek_regex_replace_once:nn is very recent (a few days ago). — egreg, Dec 8 '20 at 15:10
@JérômeLAURENS Bruno is likely to add an xparse interface for the same concept: not yet fleshed out fully, so 'watch this space' — Joseph Wright♦, Dec 8 '20 at 17:20

user229669user229669 · Accepted Answer · 2020-11-30 22:46:47Z

2

Maybe I interpret the question correctly, maybe not, but at least this may help to clarify it.

\documentclass{article}
\makeatletter
\def\mymacro#1]{\edef\mymacro@tmp{\noexpand\in@{[}{#1}}%
\mymacro@tmp
\ifin@
\expandafter\mymacro@i#1%
\else
\#1=#1,\#2=]%
\fi}
\def\mymacro@i#1[#2]{\#1=#1,\#2=[}
\makeatother
\begin{document}
\mymacro blablabla[ lorem ipsum ] etc

\mymacro blablabla] lorem ipsum [ etc
\end{document}

answered Nov 30 '20 at 22:46

user229669

This solution does work for the originally given examples, but not for the more general problem stated in the title. I changed the examples to be more precise. – Jérôme LAURENS Dec 1 '20 at 11:51

Add a comment |

Ulrich Diez · Accepted Answer · 2020-12-04 23:26:11Z

I can offer a macro

\SplitAtSquareBracketAndPassToMacro{⟨macro A⟩}{⟨macro B⟩}{⟨tokens⟩}

which works as follows:

If ⟨tokens⟩ doesn't contain any square bracket of category code 12(other) which is not nested in curly braces, then

⟨macro B⟩{⟨tokens⟩}

is delivered.

If ⟨tokens⟩ does contain at least one square bracket of category code 12(other) which is not nested in curly braces, then

⟨macro A⟩{⟨tokens before first square bracket⟩}{⟨square bracket⟩}{⟨tokens behind first square bracket⟩}

is delivered.

First in a \romannumeral-expansion-driven tail-recursive loop it is tested whether the ⟨remaining tokens⟩-argument

is empty
or has a leading [₁₂ or ]₁₂
or has a leading explicit space token.

If it is empty, then ⟨macro B⟩{⟨tokens⟩} is delivered.

If it has a leading [₁₂ or ]₁₂, then a macro processing a [₁₂-delimited argument respective ]₁₂-delimited argument is called for splitting ⟨tokens⟩ accordingly.

If it has a leading explicit space token, then that is removed and another iteration of the loop is done.

If neither is the case, then a non-delimited argument is removed and another iteration of the loop is done.

As ⟨macro A⟩-argument you can pass \mymacro to \SplitAtSquareBracketAndPassToMacro.

\documentclass{article}
\makeatletter
%%=============================================================================
%% Paraphernalia:
%%    \UD@firstoftwo, \UD@secondoftwo,
%%    \UD@PassFirstToSecond, \UD@Exchange, \UD@removespace
%%    \UD@CheckWhetherNull, \UD@CheckWhetherLeadingTokens, 
%%    \UD@ExtractFirstArgLoop
%%=============================================================================
\newcommand\UD@firstoftwo[2]{#1}%
\newcommand\UD@secondoftwo[2]{#2}%
\newcommand\UD@PassFirstToSecond[2]{#2{#1}}%
\newcommand\UD@Exchange[2]{#2#1}%
\newcommand\UD@removespace{}\UD@firstoftwo{\def\UD@removespace}{} {}%
%%-----------------------------------------------------------------------------
%% Check whether argument is empty:
%%.............................................................................
%% \UD@CheckWhetherNull{<Argument which is to be checked>}%
%%                     {<Tokens to be delivered in case that argument
%%                       which is to be checked is empty>}%
%%                     {<Tokens to be delivered in case that argument
%%                       which is to be checked is not empty>}%
%%
%% The gist of this macro comes from Robert R. Schneck's \ifempty-macro:
%% <https://groups.google.com/forum/#!original/comp.text.tex/kuOEIQIrElc/lUg37FmhA74J>
\newcommand\UD@CheckWhetherNull[1]{%
  \romannumeral\expandafter\UD@secondoftwo\string{\expandafter
  \UD@secondoftwo\expandafter{\expandafter{\string#1}\expandafter
  \UD@secondoftwo\string}\expandafter\UD@firstoftwo\expandafter{\expandafter
  \UD@secondoftwo\string}\expandafter\z@\UD@secondoftwo}%
  {\expandafter\z@\UD@firstoftwo}%
}%
%%-----------------------------------------------------------------------------
%% Check whether argument's leading tokens form a specific 
%% token-sequence that does neither contain explicit character tokens of 
%% category code 1 or 2 nor contain tokens of category code 6:
%%.............................................................................
%% \UD@CheckWhetherLeadingTokens{<argument which is to be checked>}%
%%                              {<a <token sequence> without explicit 
%%                                character tokens of category code
%%                                1 or 2 and without tokens of
%%                                category code 6>}%
%%                              {<internal token-check-macro>}%
%%                              {<tokens to be delivered in case
%%                                <argument which is to be checked> has
%%                                <token sequence> as leading tokens>}%
%%                              {<tokens to be delivered in case 
%%                                <argument which is to be checked>
%%                                does not have <token sequence> as
%%                                leading tokens>}%
\newcommand\UD@CheckWhetherLeadingTokens[3]{%
  \romannumeral\UD@CheckWhetherNull{#1}{\expandafter\z@\UD@secondoftwo}{%
    \expandafter\UD@secondoftwo\string{\expandafter
    \UD@@CheckWhetherLeadingTokens#3{\relax}#1#2}{}}%
}%
\newcommand\UD@@CheckWhetherLeadingTokens[1]{%
  \expandafter\UD@CheckWhetherNull\expandafter{\UD@firstoftwo{}#1}%
  {\UD@Exchange{\UD@firstoftwo}}{\UD@Exchange{\UD@secondoftwo}}%
  {\expandafter\expandafter\expandafter\expandafter
   \expandafter\expandafter\expandafter\z@\expandafter\expandafter
   \expandafter}\expandafter\UD@secondoftwo\expandafter{\string}%
}%
%%-----------------------------------------------------------------------------
%% Extract first inner undelimited argument:
%%
%%   \romannumeral\UD@ExtractFirstArgLoop{ABCDE\UD@SelDOm} yields  {A}
%%
%%   \romannumeral\UD@ExtractFirstArgLoop{{AB}CDE\UD@SelDOm} yields  {AB}
%%.............................................................................
\@ifdefinable\UD@RemoveTillUD@SelDOm{%
  \long\def\UD@RemoveTillUD@SelDOm#1#2\UD@SelDOm{{#1}}%
}%
\newcommand\UD@ExtractFirstArgLoop[1]{%
  \expandafter\UD@CheckWhetherNull\expandafter{\UD@firstoftwo{}#1}%
  {\z@#1}%
  {\expandafter\UD@ExtractFirstArgLoop\expandafter{\UD@RemoveTillUD@SelDOm#1}}%
}%
%%-----------------------------------------------------------------------------
%% \UD@internaltokencheckdefiner{<internal token-check-macro>}%
%%                              {<token sequence>}%
%% Defines <internal token-check-macro> to snap everything 
%% until reaching <token sequence>-sequence and spit that out
%% nested in braces.
%%-----------------------------------------------------------------------------
\newcommand\UD@internaltokencheckdefiner[2]{%
  \@ifdefinable#1{\long\def#1##1#2{{##1}}}%
}%
%%=============================================================================
%% Supplementary macros for \SplitAtSquareBracketAndPassToMacro 
%% and \SplitAtSquareBracketAndPassToMacro 
%%=============================================================================
\UD@internaltokencheckdefiner{\UD@InternalExplicitSpaceCheckMacro}{ }%
\UD@internaltokencheckdefiner{\UD@InternalLeftSquaeBracketCheckMacro}{[}%
\UD@internaltokencheckdefiner{\UD@InternalRightSquaeBracketCheckMacro}{]}%
\@ifdefinable\UD@SplitAtLeftSquareBracket{%
  \long\def\UD@SplitAtLeftSquareBracket#1[{\expandafter\z@\expandafter{\UD@firstoftwo{}#1}{[}}%
}%
\@ifdefinable\UD@SplitAtRightSquareBracket{%
  \long\def\UD@SplitAtRightSquareBracket#1]{\expandafter\z@\expandafter{\UD@firstoftwo{}#1}{]}}%
}%
\newcommand\UD@SplitAtSquareBracket[3]{%
  \expandafter\UD@PassFirstToSecond\expandafter{%
     \romannumeral
     \expandafter\expandafter\expandafter\z@\expandafter\UD@firstoftwo\expandafter{\expandafter}%
     \romannumeral
     \expandafter\expandafter\expandafter\z@\expandafter\UD@firstoftwo\expandafter{\expandafter}%
     \romannumeral#3#1%
  }{%
    \expandafter\UD@PassFirstToSecond
    \romannumeral\expandafter\expandafter\expandafter\UD@ExtractFirstArgLoop
                 \expandafter\expandafter\expandafter{%
                 \expandafter\UD@firstoftwo\expandafter{\expandafter}%
                 \romannumeral#3#1\UD@SelDOm}{%
      \expandafter\UD@PassFirstToSecond
      \romannumeral\expandafter\UD@ExtractFirstArgLoop\expandafter{%
                   \romannumeral#3#1\UD@SelDOm}{%
        \z@#2% 
      }%
    }%
  }%
}%
\newcommand\SplitAtSquareBracketAndPassToMacro[3]{%
  \romannumeral\UD@SplitAtSquareBracketAndPassToMacroLoop{#3}{#3}{#1}{#2}%
}%
\newcommand\UD@SplitAtSquareBracketAndPassToMacroLoop[4]{%
  % #1 = <remaining tokens>
  % #2 = <tokens>
  % #3 = <macro A>
  % #4 = <macro B>
  \UD@CheckWhetherNull{#1}{\z@#4{#2}}{%
    \UD@CheckWhetherLeadingTokens{#1}{ }{\UD@InternalExplicitSpaceCheckMacro}{%
      \expandafter\UD@SplitAtSquareBracketAndPassToMacroLoop\expandafter{\UD@removespace#1}{#2}{#3}{#4}%
    }{%
      \UD@CheckWhetherLeadingTokens{#1}{[}{\UD@InternalLeftSquaeBracketCheckMacro}{%
         \UD@SplitAtSquareBracket{.#2}{#3}{\UD@SplitAtLeftSquareBracket}%
      }{%
        \UD@CheckWhetherLeadingTokens{#1}{]}{\UD@InternalRightSquaeBracketCheckMacro}{%
          \UD@SplitAtSquareBracket{.#2}{#3}{\UD@SplitAtRightSquareBracket}%
        }{%
          \expandafter\UD@SplitAtSquareBracketAndPassToMacroLoop\expandafter{\UD@firstoftwo{}#1}{#2}{#3}{#4}%
        }%
      }%
    }%
  }%
}%
\makeatother

%%=============================================================================
%% \mymacro{<tokens 1>}{<tokens 2>}{<tokens 3>}
%% Whan arguments are passed to  \mymacro from 
%% \SplitAtSquareBracketAndPassToMacro, then 
%% - <tokens 1> is the things before the first [ respective ] .
%% - <tokens 2> is either [ or ] .
%% - <tokens 3> is the things behind the first [ respective ] .
%%=============================================================================
\newcommand\mymacro[3]{%
  \noindent
  \scantokens\expandafter\expandafter\expandafter{%
             \expandafter\string\expandafter\verb\expandafter|\string\mymacro|:%
  }%
  Argument 1 is: \scantokens\expandafter{\string\verb|#1|.}%
  Argument 2 is: \scantokens\expandafter{\string\verb|#2|.}%
  Argument 3 is: \scantokens\expandafter{\string\verb|#3|.}%
 }%
%%=============================================================================
%% macro in case there was no square bracket
%%=============================================================================
\newcommand\nosquarebracketsmacro[1]{%
  \noindent
  \scantokens\expandafter\expandafter\expandafter{%
             \expandafter\string\expandafter\verb\expandafter|%
             \string\nosquarebracketsmacro|:%
  }%
  Argument is: \scantokens\expandafter{\string\verb|#1|.}%
}%

\parindent=0ex

\begin{document}

\verb|\SplitAtSquareBracketAndPassToMacro{\mymacro}{\nosquarebracketsmacro}{AB]CDE}|:\\
\SplitAtSquareBracketAndPassToMacro{\mymacro}{\nosquarebracketsmacro}{AB]CDE}

\vfill

\verb|\SplitAtSquareBracketAndPassToMacro{\mymacro}{\nosquarebracketsmacro}{AB[CDE}|:\\
\SplitAtSquareBracketAndPassToMacro{\mymacro}{\nosquarebracketsmacro}{AB[CDE}

\vfill

\verb|\SplitAtSquareBracketAndPassToMacro{\mymacro}{\nosquarebracketsmacro}{ABCDE}|:\\
\SplitAtSquareBracketAndPassToMacro{\mymacro}{\nosquarebracketsmacro}{ABCDE}

\vfill

\verb|\SplitAtSquareBracketAndPassToMacro{\mymacro}{\nosquarebracketsmacro}{{AB}]CDE}|:\\
\SplitAtSquareBracketAndPassToMacro{\mymacro}{\nosquarebracketsmacro}{{AB}]CDE}

\vfill

\verb|\SplitAtSquareBracketAndPassToMacro{\mymacro}{\nosquarebracketsmacro}{{AB}[CDE}|:\\
\SplitAtSquareBracketAndPassToMacro{\mymacro}{\nosquarebracketsmacro}{{AB}[CDE}

\vfill

\verb|\SplitAtSquareBracketAndPassToMacro{\mymacro}{\nosquarebracketsmacro}{{AB}CDE}|:\\
\SplitAtSquareBracketAndPassToMacro{\mymacro}{\nosquarebracketsmacro}{{AB}CDE}

\vfill

\verb|\SplitAtSquareBracketAndPassToMacro{\mymacro}{\nosquarebracketsmacro}{AB]{CDE}}|:\\
\SplitAtSquareBracketAndPassToMacro{\mymacro}{\nosquarebracketsmacro}{AB]{CDE}}

\vfill

\verb|\SplitAtSquareBracketAndPassToMacro{\mymacro}{\nosquarebracketsmacro}{AB[{CDE}}|:\\
\SplitAtSquareBracketAndPassToMacro{\mymacro}{\nosquarebracketsmacro}{AB[{CDE}}

\vfill

\verb|\SplitAtSquareBracketAndPassToMacro{\mymacro}{\nosquarebracketsmacro}{AB{CDE}}|:\\
\SplitAtSquareBracketAndPassToMacro{\mymacro}{\nosquarebracketsmacro}{AB{CDE}}

\vfill

\verb|\SplitAtSquareBracketAndPassToMacro{\mymacro}{\nosquarebracketsmacro}{{AB}]{CDE}}|:\\
\SplitAtSquareBracketAndPassToMacro{\mymacro}{\nosquarebracketsmacro}{{AB}]{CDE}}

\vfill

\verb|\SplitAtSquareBracketAndPassToMacro{\mymacro}{\nosquarebracketsmacro}{{AB}[{CDE}}|:\\
\SplitAtSquareBracketAndPassToMacro{\mymacro}{\nosquarebracketsmacro}{{AB}[{CDE}}

\vfill

\verb|\SplitAtSquareBracketAndPassToMacro{\mymacro}{\nosquarebracketsmacro}{{AB}{CDE}}|:\\
\SplitAtSquareBracketAndPassToMacro{\mymacro}{\nosquarebracketsmacro}{{AB}{CDE}}

\vfill

\verb|\SplitAtSquareBracketAndPassToMacro{\mymacro}{\nosquarebracketsmacro}{{AB]CDE}}|:\\
\SplitAtSquareBracketAndPassToMacro{\mymacro}{\nosquarebracketsmacro}{{AB]CDE}}

\vfill

\verb|\SplitAtSquareBracketAndPassToMacro{\mymacro}{\nosquarebracketsmacro}{{AB[CDE}}|:\\
\SplitAtSquareBracketAndPassToMacro{\mymacro}{\nosquarebracketsmacro}{{AB[CDE}}

\vfill

\verb|\SplitAtSquareBracketAndPassToMacro{\mymacro}{\nosquarebracketsmacro}{AB]CDE}|:\\
\SplitAtSquareBracketAndPassToMacro{\mymacro}{\nosquarebracketsmacro}{{ABCDE}}

\end{document}

As far as the task/macro-writing-challenge in the question is concerned, I admittedly cheated a bit:

By defining the macro \SplitAtSquareBracketAndPassToMacro in such a way that the tokens to be split up are passed to it as a macro argument, I have dodged the need to "fish" these tokens out of the token stream directly.

I did this because tokens can be "fished out" of the token stream in principle only as macro arguments.
For example, undelimited macro arguments consisting of several tokens are nested in a token pair consisting of an explicit character token of category code 1 (begin group) and an explicit character token of category code 2 (end group). This pair of tokens forms the so-called argument-group.
Usually, argument-groups are formed by the character-tokens {₁ and }₂, but it is not excluded that for some obscure reason the category code régime is different and other characters of these category-codes are in use.
I don't know of any reliable method to fish undelimited macro arguments out of the token stream in a way where the braces or category-code-1- and -2-character-tokens forming the argument-group in which the undelimited arguments are nested are preserved exactly instead of being replaced by some "hard-coded" pair of explicit character tokens of category code 1/2.
You can have TeX "look" via \futurelet and \ifcat for the category code of the next token in the token stream. But firstly a category-code-1-character-token denoting the begin of the argument-group is not fished out from just "looking", and secondly the category-code-2-character-token denoting the end of the argument-group is not the next token in the token stream...

Stack Exchange Network

current community

your communities

more stack exchange communities

How to get what is before the next '[' or ']'

4 Answers 4

Your Answer

Not the answer you're looking for? Browse other questions tagged macros delimiters arguments or ask your own question.

Linked

Hot Network Questions

How to get what is before the next '[' or ']'

4 Answers 4

Your Answer

Sign up or log in

Post as a guest

Not the answer you're looking for? Browse other questions tagged macros delimiters arguments or ask your own question.

Linked

Related

Hot Network Questions