# pgfextra vs. graph: a paradox!

Consider the following LaTeX manuscript featuring a TikZ picture of two filled circles. The right circle is drawn inside a \pgfextra. The path's fill color is yellow!

\documentclass{standalone}
\usepackage{tikz}
\begin{document}
\begin{tikzpicture}
\path[fill=yellow] (-.5,0) circle(1)
\pgfextra{\fill (.5,0) circle(1);};
\end{tikzpicture}
\end{document}


It renders as

If we now replace the \pgfextra with a graph:

\documentclass{standalone}
\usepackage{tikz}
\usetikzlibrary{graphs}
\begin{document}
\begin{tikzpicture}
\path[fill=yellow] (-.5,0) circle(1)
graph{""[circle,at={(.5,0)},minimum size=2cm,fill]};
\end{tikzpicture}
\end{document}


we obtain the picture

In both examples the path has the same form:

\path[fill=yellow] (-.5,0) circle(1) ...;


One would expect this to be implemented in one of two ways: either the option is executed at the beginning:

\pgfsetfillcolor{yellow}
\pgfpathcircle{\pgfpoint{-.5cm}{0cm}}{1cm}
...
\pgfusepath{fill}


or just before the path is used:

\pgfpathcircle{\pgfpoint{-.5cm}{0cm}}{1cm}
...
\pgfsetfillcolor{yellow}
\pgfusepath{fill}


The first alternative must be rejected, since then the first example would be implemented as follows

\pgfsetfillcolor{yellow}
\pgfpathcircle{\pgfpoint{-.5cm}{0cm}}{1cm}

% \pgfextra{\fill (.5,0) circle(1);}
\pgfpathcircle{\pgfpoint{.5cm}{0cm}}{1cm}
\pgfusepath{fill}

\pgfusepath{fill}


yielding the picture

But the second alternative must also be rejected, because then the implementation of the second example would see the graph created before the fill color has been set:

\pgfpathcircle{\pgfpoint{-.5cm}{0cm}}{1cm}

% graph{""[circle,at={(.5,0)},minimum size=2cm,fill]}

\pgfsetfillcolor{yellow}
\pgfusepath{fill}


and since a graph is locked in a TeX box upon creation, and since the visual appearance of a TeX box is finalized upon creation, the graph node would be filled with black, and the picture would look something like this

We have arrived at a contradiction!

• You are expecting things to happen but they don't. As with your previous questions if you don't read the source code you are out of luck. Apparently the author had a different expectancy during iimplementation. – percusse Aug 2 '17 at 18:36
• @percusse: I don't think that a user of a package should need to read the source code in order to understand how it behaves, but more importantly, I don't think that "read the source code" is a legitimate answer on this website. The examples I gave are short and simple. I'm sure they can be explained in plain English. – Evan Aad Aug 2 '17 at 18:43
• Then you shouldn't fiddle with its internals because you are asking why is it implemented as such. Without TeX groups and many many details nobody can answer it. – percusse Aug 2 '17 at 18:47
• @percusse: I'm not asking why is it implemented as such? I'm asking: Why am I seeing this and not that?, hoping to find a systematic explanation. However, due to the nature of the primitives \pgfextra and graph I don't think this can be explained other than by delving to some degree into the implementation, but there's a difference between giving a thirsty man a glass of water and dropping him in the middle of the lake. – Evan Aad Aug 2 '17 at 18:52
• There is also a difference between I'll start learning this tool and oh I'm gonna dissect its lowest level macros without going through every other person went with manuals and lots and lots of trials and errors. – percusse Aug 2 '17 at 19:08

\pgfextra does not surround it's contents with a pgfinterruptpath environment, so in:

\path[fill=yellow] (-.5,0) circle [radius=1]


the "yellow" path is effectively continued by the path described by the path inside \pgfextra. This path is then used by "red" path and filled red. Note, that adding a path inside \pgfextra with or without the pgfinterruptpath environment is a really bad idea, the behaviour is essentially undefined. The intended use of \pgfextra is mostly for calculations or some simple TeX stuff, not arbitrary TikZ commands. In addition, with respect to \pgfextra the manual cautions (p.162, my emphasis):

...this operation should only be used by real experts and should only be used deep inside clever macros, not on normal paths.

By using

\path[fill=yellow] (-.5,0) circle [radius=1]
\pgfextra{
\pgfinterruptpath
\endpgfinterruptpath
};


A yellow and red circle will be drawn.

• Thanks. So I understand that the fill is set just before the path is used, correct? So in the first example it's not that the red fill overwrites the yellow one; rather, at the point when the red fill is set, the yellow fill has not yet been set, right? – Evan Aad Aug 2 '17 at 10:30
• In order to do that, I will have to go to tikz.code.tex and go through the code step by step. Which is something you can do yourself. – Mark Wibrow Aug 2 '17 at 12:30
• @EvanAad That claim is simply absurd. Because not everything can be explained without reading the source code (or the TeX Book or the documentation of expl3 or ...), it follows that everyone using the site would need to understand that stuff and, hence, the site would be without purpose? Or is it: because you think everything should be explicable in isolation from the rest, to everybody, it follows that people are just being perverse if they claim otherwise? – cfr Aug 3 '17 at 0:08
• @EvanAad Colour depends on literals. PGF/TikZ has to work with what is provided by the backend. There is no colour in TeX. Try \tikz{\node [text=blue] {blue \mbox{\textcolor{red}{red box}} and ??};} \tikz{\node [text=green] {green \textcolor{red}{red} and ??};} You don't have to read the source code to use TikZ because you don't have to have the kind of explanations whose existence you demand in order to use it. It is simply silly to claim otherwise. Pointers to relevant resources are a legitimate response to requests for explanations. How could it reasonably be otherwise? – cfr Aug 3 '17 at 0:12
• @EvanAad I am not being deliberately obtuse by referring you to the source code. You aspire to a deep level of understanding of the pgf/tikz internals and this is great! But as I cannot recall all the code in PGF (even the bits I wrote), to give the precise and detailed answer that you require, consultation of the source code is the only option. This is time consuming and I do not exist only to answer questions on this site. I think that pointing you to the source code where the necessary information can be found is a legitimate response. – Mark Wibrow Aug 3 '17 at 6:50

The TikZ "engine" parses a \path statement from left to write, executing the parts in their textual order. What "executing" means depends on the part, for instance

• nodes are typeset inside a TeX box which is saved to a register. In the beginning of a node's execution all path options except for font, text and behind path are temporarily cleared and restored at the end of the node's execution, after it has been typeset. Thus, a node is typeset based solely on the options explicitly applied to the node (allowing for the exceptions mentioned above).
• Path-construction operations such as circle(1) are converted to \pgf... commands that are appended to an internal list object called the current path (this is a gross over-simplification, but it will do for the present purposes). It is important to know that a \path statement is wrapped in an implicit TeX scope, however the current path list is global, so it transcends scopes.
• \pgfextra{<body>} temporarily returns control to the normal TikZ processor (as opposed to the path parser), passing it <body> as the text to be processed next. The current TeX scope is not exited, and no new scope is opened. The internal state of the engine at the time the \pgfextra was encountered is not altered, so all the internal data structures created so far while parsing the \path statement remain intact.

When TikZ finishes processing <body>, control returns to the path parser whose cursor is now set just to the right of the \pgfextra{<body>} expression. Again, the transition back to normal path parsing is not accompanied by any changes to the state of the engine and no scope is closed, since none was opened to begin with.

Options are executed by the \tikzset command. What this entails depends on the option. For instance

• The behind path option makes an internal change to the operation of the TikZ engine: it makes the engine typeset the nodes that are flagged with this option inside a different box, called the background box, than the one in which the other nodes are typeset.

• The fill=<color> option appends the command \pgfsetfillcolor{<color>} to a special list, the options list, consisting of similar basic-layer commands. This list has no effect on the operation of the TikZ engine. It is simply accumulated and eventually written to the dvi file as-is (after the \pgf... commands have been substituted by their \special implementation). Unlike the current path, the options list is local to the TeX scope that implicitly surrounds the \path command, and is set to the empty list in the beginning of this scope.

Thus, the execution of a path part usually means "translating" it into a sequence of low-level commands, and saving this sequence either as a new object, or by appending it to a pre-existing list.

The execution of the semicolon at the end of the TikZ path is special. It entails actually writing all the path parts to the dvi file (this is a gross over-simplification, but it will do for our purposes). Since by the time the semicolon is executed, the execution of the various parts has already resulted in their translation to low-level commands, the main job of the semicolon is to determine the order in which these lists will be dumped to the dvi file.

A simplified version of the semi-colon's execution is:

1. Write out nodes (including graphs, which are treated as glorified nodes) marked with behind path.

2. Write out the options list.

3. Write out the current path.

4. Use the path, i.e. draw it, fill it, etc. As a side effect, the current path object (which, recall, is global) is reset.

5. Write out the rest of the nodes (and graphs).

6. Close the TeX scope that implicitly surrounds the path statement, and restore the graphics state that prevails at the top-level of the TikZ environment. In particular, the default fill-color for the TikZ environment is restored at the end of each path, together with many other defaults of the TikZ environment.

Let's now apply all this theory to the examples in the original post.

First example

\path[fill=yellow] (-.5,0) circle(1) \pgfextra{\fill (.5,0) circle(1);};

1. The fill=yellow option is executed. This results in the instruction \pgfsetfillcolor{yellow} being appended to the initially empty options list.

2. (-.5,0) circle(1) is executed. This results in the instruction \pgfpathcircle{\pgfpoint{-.5cm}{0pt}}{1cm} being added to the initially empty current path.

3. \pgfextra{\fill (.5,0) circle(1);} is executed. This causes TikZ to temporarily "forget" that it was in the middle of parsing a path, and to start processing the code \fill (.5,0) circle(1); as if it was written at top-level. However the engine's internal state remains intact, so, in particular, the options list and the current path remain intact.

1. (.5,0) circle(1) is executed. This results in the instruction \pgfpathcircle{\pgfpoint{.5cm}{0pt}}{1cm} being appended to the current path object, which already contains a circle at (-.5,0).

2. The semicolon at the end of the path is encountered. This brings about the following sequence of actions.

3. Firstly, the behind path nodes are written to the dvi files. There are none, so this is a no-op.

4. Secondly, the options list is written to the dvi file. This is not the same list as the one mentioned above, since options lists are local to the implicit scope that surrounds a \path statement, and since they are set to the empty list in the beginning of this scope. Since no option has been specified for the path \fill (.5,0) circle(1);, the options list is empty, and so this step is a no-op.

5. Thirdly, the path-construction commands for the current path (consisting of two circles) are written to the dvi file.

6. Fourthly, the path is filled, i.e. the \specials implementing the command \pgfusepath{fill} are written to the dvi file. Since the top-level option fill=yellow has not yet been applied, the default color is used. In the present case, since no default color has been explicitly specified, either for the TikZ picture or for the document, black is the implicit default color (for this is the default color for all PDF documents). At the end of this step, the (global) current path object is reset, i.e. the current path becomes empty.

7. Fifthly, the rest of the nodes are written to the dvi file. There are none, so this is a no-op.

8. Sixthly, the TeX scope that implicitly surrounds the \fill ... statement is closed, and the default settings for the TikZ environment are restored.

4. The top-level semicolon is executed. However, since there are no nodes, and since the current path is empty, there's nothing left to do except writing the options list to the dvi file. However this has no overall effect: the options list changes the fill-color to yellow, but this change is then overturned when the default graphics state for the TikZ environment is restored as the last step in the semicolon's execution.

Second example

\path[fill=yellow] (-.5,0) circle(1)
graph{""[circle,at={(.5,0)},minimum size=2cm,fill]};

1. As in the first example.

2. As in the first example.

3. The graph is executed. A graph's execution is similar to a node's. Therefore, the graph is typeset inside a TeX box that is saved to a register. When the graph's execution begins, the path options (fill=yellow) are temporarily cleared, so the fill-color does not get set inside the box. For this reason, whatever the fill-color will be at the time the box's contents are spilled to the dvi file will be the one used to render the graph.

4. The semicolon is executed:

1. behind path nodes are written to the dvi file. There are none, so this is a no-op.

2. The options list, i.e. \pgfsetcolor{yellow}, is written to the dvi file.

3. The path-construction commands for the current path, consisting of a circle centered at (-.5,0), are written to the dvi file.

4. The path is used: the command \pgfusepath{fill} is written to the dvi file. When the dvi file will be rendered on the screen, a single yellow circle will appear at this point. At the end of this step, the (global) current path object is reset, i.e. the current path becomes empty.

5. The rest of the nodes (and graphs) are written to the dvi file. So now the contents of the graph's box get written out to the dvi file. Since the fill color is not set inside the box, the fill color that will be used is the last one that has been set, namely yellow.

6. The TeX scope that implicitly surrounds the \path ... statement is closed, and the default settings for the TikZ environment are restored. In particular, the fill-color is reset to the default black.

Third example (Mark Wibrow's example)

\path[fill=yellow] (-.5,0) circle [radius=1]
\pgfextra{
\pgfinterruptpath
\endpgfinterruptpath
};

1. As in the first example.

2. As in the first example.

3. The \pgfextra block is executed. This causes TikZ to temporarily "forget" that it was in the middle of parsing a path, and to start processing the \pgfextra body as if it was written at top-level. However the engine's internal state remains intact, so, in particular, the options list and the current path remain intact.

1. \pgfinterrupt starts a new TeX block, and saves the current path (consisting of a circle centered at (-.5,0)) in a local "private variable", and resets the (global) current path to the empty path.

2. The option red is executed. This effectively adds the instructions \pgfsetfillcolor{red} and \pgfsetstrokecolor{red} to the options list. This is not the same list as the one mentioned above, since options lists are local to the implicit scope that surrounds a \path statement, and since they are set to the empty list in the beginning of this scope. So at the end of this step, the visible options list consists of only two commands, and these commands set the fill- and stroke color to red.

3. (.5,0) circle [radius=1] is executed. This results in the instruction \pgfpathcircle{\pgfpoint{.5cm}{0pt}}{1cm} being appended to the empty current path object.

4. The semicolon at the end of the path is encountered. This causes the current path, consisting of a single circle centered at (.5,0) to be written to the dvi file followed by an instruction to fill this circle with red. After this, the current path is set to the empty path. When the dvi file will be rendered on the screen, a single red circle will appear at this point.

5. The implicit TeX block that surrounds the \fill [red] (.5,0) circle [radius=1]; command is closed. Since options lists are local to the implicit scope that surrounds a \path statement, the options list is replaced by the previous one consisting of the single command to set the fill color to yellow.

6. \endpgfinterruptpath ends the TeX block opened by \pgfinterruptpath and sets the global current path object back to the one saved by the \pgfinterruptpath command, namely the one consisting of a circle centered at (-.5,0).

4. The top-level semicolon is executed:

1. behind path nodes are written to the dvi file. There are none, so this is a no-op.

2. The options list, i.e. \pgfsetcolor{yellow}, is written to the dvi file.

3. The path-construction commands for the current path, consisting of a circle centered at (-.5,0), are written to the dvi file.

4. The path is used: the command \pgfusepath{fill} is written to the dvi file. When the dvi file will be rendered on the screen, a yellow circle will appear at this point. It will be drawn on top of and slightly to the left of the red circle that was drawn before. At the end of this step, the (global) current path object is reset, i.e. the current path becomes empty.

5. The rest of the nodes (and graphs) are written to the dvi file. There are none, so this is a no-op.

6. The TeX scope that implicitly surrounds the \path ... statement is closed, and the default settings for the TikZ environment are restored. In particular, the fill-color is reset to the default black.

The resulting picture is