# How does \path, and the like, parse its argument?

I'm really curious whether someone would care to take the time to explain how to create macros which work similar to some of the macros in the pgf/TiKz bundle.

What I'm particularly interested in is how to create a command that works as

\<csname> <content> [bare word directive] <more content>;


To make clear what I mean, I'm referring to how \path and \draw (among others) are parsed and can contain bare word commands such as node and edge as illustrated below:

\documentclass{article}
\usepackage{tikz}
\usetikzlibrary{calc}
\begin{document}

\begin{tikzpicture}
\path[blue] node (B) at (0,3) {CORNER} edge [->] (0,0) edge [->] (2,4);
\draw  (0,0) -- (2,4) node [midway] {HELLO} edge [->] (B);
\draw  (2,4) -- (4,2) edge [->] (B)         node [midway] {HELLO} ;
\end{tikzpicture}

\end{document}


What I'm not interested in is how node knows what to work with or how edge knows where to draw to and from.

Instead, what I'm curious is how does LaTeX wind up seeing the bare word node as, essentially, a further command, and likewise with edge. So this is a question about parsing arguments and interpreting the content of those arguments.

I've dug through quite a bit of the tikz.code.tex and other files trying to figure out how this parsing is achieved, but I very quickly get lost in the code.

I can't even get something so simple as the following to work:

\documentclass{article}
\makeatletter

\newcommand\hello{Hello World!}
\newcommand\ciao{ciao?}
\newcommand\secretmessage{\ae@message}
\def\ae@message(#1)#2;{%%
\def\ae@continue{}%%
\hspace{1em}\emph{#1}%%
\expandafter\ifx\expandafter\relax\detokenize\expandafter{#2}\relax
\else
\def\ae@continue{\ae@parse #2 ;}%%
\fi
\ae@continue
\def\ae@continue{}%%
}
\def\ae@parse#1 ;{\typeout{===>"#1"}\csname#1\endcsname}
\def\ae@trim#1{\ignorespaces#1\unskip}

\makeatother
\setlength\parskip{2ex}
\setlength\parindent{0pt}
\begin{document}

\secretmessage(This is my message.);

\secretmessage(This is my message.) hello;

\secretmessage(This is my message.) ciao;

\secretmessage(This is my message.);

The above should have been interpreted as equivalent to:

\secretmessage(This is my message.);

\secretmessage(This is my message.); \hello

\secretmessage(This is my message.); \ciao

\secretmessage(This is my message.);

\end{document}


I can't begin to figure out how to get the spaces out of the above without creating horrible cludges like

 \csname\space hello\endcsname


Though I would really like to be able to do something more complicated like:

\secretmessage(This is my message.) hello ciao;


which should effectively be interpreted to

\secretmessage(This is my message.); \hello\space \ciao


Though I strongly suspect that without the aid of further arguments to either hello or ciao such adjacent bare word directives are unlikely to get parsed correctly without an extreme amount of work.

• You can watch the parsing in slow motion by adding \tracingmacros=1 and looking in the log file. The key is using delimited arguments. Also, \path wants just to transform the code up to ; into lower level instructions and spaces are mostly discarded. – egreg Jul 8 '14 at 20:35
• Some brief introduction I gave some time ago tex.stackexchange.com/a/120443/3235 – percusse Jul 8 '14 at 20:39
• Can I suggest using my own (half-working) unravel package as an alternative to \tracingmacros=1? – Bruno Le Floch Jul 9 '14 at 0:21

Short answer: it requires indeed an extreme amount of work

Long answer: No, you don't want to know the details. I don't want to write them. Really... (Update: There is a good explanation by @percusse here)

Tikz does not use the TeX macro engine. It builds its own parser which basically processes all the characters after a given command (such as \tikz, \path) expecting some valid keywords, and checking letter by letter if that keyword is indeed present, until a ; (which marks the end of the input for this parser) is found. It also processes specially ( and so on. The meaning of most chars are context-dependent, so it has to call apropiate macros to deal with them depending on what was previously parsed.

To get an idea about the level of complexity we are talking about, here is a small snippet taken from tikz.code.tex which tries to determine if the following "token" (not in the TeX sense) is the word "coordinate". Tikz "calls" this macro where it expect a node as part of a path.

\def\tikz@parse@child@node{%
\pgfutil@ifnextchar n{\tikz@parse@child@node@n}%
{\pgfutil@ifnextchar c{\tikz@parse@child@node@c}%
{\tikz@parse@child@node@rest}}}
\def\tikz@parse@child@node@rest#1\pgf@stop{\def\tikz@child@node@rest{#1}}
\def\tikz@parse@child@node@c c{\pgfutil@ifnextchar o{\tikz@parse@child@node@co}{\tikz@parse@child@node@rest c}}
\def\tikz@parse@child@node@co o{\pgfutil@ifnextchar o{\tikz@parse@child@node@coordinate}{\tikz@parse@child@node@rest co}}
\def\tikz@parse@child@node@coordinate ordinate{%
\pgfutil@ifnextchar ({\tikz@@parse@child@node@coordinate}{%
\def\tikz@child@node@text{[shape=coordinate]{}}%
\tikz@parse@child@node@rest}}%}
\def\tikz@@parse@child@node@coordinate(#1){%
\pgfutil@ifnextchar a{\tikz@p@c@n@c@at(#1)}{%
\def\tikz@child@node@text{[shape=coordinate,name=#1]{}}%
\tikz@parse@child@node@rest}}
\def\tikz@p@c@n@c@at(#1)at#2({%
\def\tikz@child@node@text@pre{[shape=coordinate,name=#1]at}%
\tikz@scan@one@point\tikz@p@c@n@c@at@math(%
}


As you can see (can't you? :-)) it first check if the letter n is present. If it is, probably it is because the word node comes, so it uses the macro \tikz@parse@child@node@n to continue. If not, it checks if it is the letter c. If it is, then probably it is the beginning of the word coordinate, so it calls the macro \tikz@parse@child@node@c to continue, where it checks if the next char is o, and so on.

This is a \tiny part of a \Huge nightmare of \expandafter, \edef, \pgfutil@ifnextchar \ifx and \tikz@really@long@macro@name@s. I wonder how Till Tantau managed to keep his sanity after writing all of this, unless this code was generated somehow with a tool.

• Tikz does not use the TeX macro engine: well TikZ is written in TeX so it must :-) I guess you mean that everything is done token-by-token so the parsing is not dependent on a more 'classical' TeX delimited argument approach (e.g. \path #1 node #2 ; or similar). (Well, it's actually a bit more complex than that, as your exert shows: coordinate is picked up based on the first two tokens, so things will go very odd if you do something like \node coward!) – Joseph Wright Jul 9 '14 at 7:27