# In what sense does PGF not keep track of canvas transformations?

In section 103.3 'Canvas Transformations' the TikZ & PGF manual for version 3.0.1a points out a distinction between coordinate transformations and canvas transformations (p. 1051):

Unlike coordinate transformations, PGF does not “keep track” of canvas transformations. In particular, it will not be able to correctly save the coordinates of shapes or nodes when a canvas transformation is used.

However when you look at the source code of PGF commands, e.g. \pgfshadepath you see that it uses system-layer transformations (a.k.a. canvas transformations). Isn't this dangerous, as it will disable PGF's ability to correctly save coordinates of shapes and nodes?

I also tried to write code that combines PGF-layer commands with system-layer transformations, and the resulting picture didn't become garbled. Rather, it looked exactly as expected. Take, for instance this:

\documentclass{article}
\usepackage{pgf}
\begin{document}
\begin{pgfpicture}
\makeatletter
% A system-layer rotation by 90 degrees counter-clockwise
\pgfsys@transformcm{0}{1}{-1}{0}{0cm}{0cm}
\makeatother
% A PGF-layer rectangle that is thin and tall.
\pgfpathrectangle{\pgfpointorigin}{\pgfpoint{1cm}{2cm}}
\pgfusepath{stroke}
\end{pgfpicture}
\end{document}


which renders as follows:

So in what sense does PGF not keep track of canvas transformations, and is unable to correctly save the coordinates of shapes or nodes when such a transformation is used?

• write more stuff after the picture – percusse Jul 27 '17 at 15:57
• @percusse: Like what? – Evan Aad Jul 27 '17 at 15:59
• Like anything that is not in the picture – percusse Jul 27 '17 at 16:00
• @percusse: Do you mean to write something after the \end{pgfpicture}? – Evan Aad Jul 27 '17 at 16:02
• Yes. Or rather, that is what I would mean and I'm 99% certain that is what is meant. – cfr Jul 27 '17 at 16:06

PGF/TikZ loses track of coordinates and, in particular, of the bounding box when a canvas transformation is applied.

Consider the following code:

\documentclass{article}
\usepackage{tikz}
\begin{document}
\begin{tikzpicture}
\makeatletter
\pgfsys@transformcm{0}{1}{-1}{0}{0cm}{0cm}
\makeatother
\filldraw [blue] (0,0) rectangle (1,2);
\draw (current bounding box.south east) rectangle (current bounding box.north west);
\end{tikzpicture}
As any dedicated reader can clearly see, the Ideal of practical reason is a representation of, as far as I know, the things in themselves; as I have shown elsewhere, the phenomena should only be used as a canon for our understanding.

\begin{tikzpicture}
\begin{scope}
\makeatletter
\pgfsys@transformcm{0}{1}{-1}{0}{0cm}{0cm}
\makeatother
\filldraw [blue] (0,0) rectangle (1,2);
\end{scope}
\draw (current bounding box.south east) rectangle (current bounding box.north west);
\end{tikzpicture}
As any dedicated reader can clearly see, the Ideal of practical reason is a representation of, as far as I know, the things in themselves; as I have shown elsewhere, the phenomena should only be used as a canon for our understanding.

\begin{tikzpicture}
\filldraw [rotate=90, blue] (0,0) rectangle (1,2);
\draw (current bounding box.south east) rectangle (current bounding box.north west);
\end{tikzpicture}
As any dedicated reader can clearly see, the Ideal of practical reason is a representation of, as far as I know, the things in themselves; as I have shown elsewhere, the phenomena should only be used as a canon for our understanding.

\begin{tikzpicture}
\begin{scope}
\filldraw [rotate=90, blue] (0,0) rectangle (1,2);
\end{scope}
\draw (current bounding box.south east) rectangle (current bounding box.north west);
\end{tikzpicture}
As any dedicated reader can clearly see, the Ideal of practical reason is a representation of, as far as I know, the things in themselves; as I have shown elsewhere, the phenomena should only be used as a canon for our understanding.
\end{document}


Sometimes, it doesn't matter if you lose track. Shading a path is typically such a case. You need to keep track of the path. You don't necessarily need to keep track of the shading. Since TikZ clips the transformed shading to the untransformed path, only the path matters when it comes to calculating the bounding box.

PGF transformations are not related to the end picture. What PGF is trying to do is to report the correct box sizes to TeX. Otherwise PGF is, as usual, converting the path soft path directives to driver-specific specials.

Here, canvas transformation means PGF does not take into account the resulting object sizes and what it calls dangerous is that the result can spill out the box size that it reports to TeX including the object locations because the transformation will be performed by the driver and PGF will mind its own business. If the result is still in the bounding box fine, if not tough luck.

Independent from all these steps, the transformation is a PostScript/PDF special. It has nothing to do with TeX side.

The following is meant to complement cfr's answer by providing more detailed explanations.

Where I describe the workings of TikZ/PGF "under the hood", it is not meant to be understood as a rigorous guide to the source code, but as an informal, but hopefully helpful and illuminating explanation of the behavior perceived by a user. To paraphrase The Textbook, this post doesn’t always tell the truth, but the author feels that this technique of deliberate lying will actually make it easier for you to understand the ideas.

One particular inaccuracy that I should mention is the use of the word "engine". When I talk about "the PGF-layer engine" or "the system-layer engine", I mean it conceptually rather than technically, somewhat in the spirit of objects in an object-oriented program. In reality there is only one engine that is at work typesetting a TeX document, and that is pdftex, or one of its siblings.

A) The layered design of the PGF/TikZ bundle

The TikZ/PGF bundle is structured in hierarchical layers: 1. system, 2. PGF, 3. TikZ. Each layer is independent of the following layers. Thus, the system layer knows nothing of PGF, whereas PGF uses the system layer, but knows nothing of TikZ. Each layer is implemented in terms of the layer directly below it: the system layer is the only one that "talks" to the rendering driver (e.g. the PDF driver, when the LaTeX engine is pdftex/pdflatex), the PGF layer is implemented in terms of the system layer, and TikZ is implemented in terms of the PGF layer and never access the system layer directly. The layers can coexist: a PGF picture can include system-layer commands, and a TikZ picture can include both PGF and system-layer commands. When command from different layers are intermixed, each command will be uniquely processed by the "engine" that deals with the corresponding layer. We will now restrict our attention to the two lower layers: the system layer, and the PGF layer.

Changes made in one layer do not propagate automatically to the other layer, and there is no event mechanism that enables one layer to "listen to" the other layer in real time. Moreover, the layers never query the state of the other layer; each layers keeps its own state and this is the only state it ever accesses.

Thus, system-layer commands that are written inside a PGF picture are invisible to the PGF engine. When a system-layer command is executed in the context of a PGF picture, its effects will take place (for instance, a transformation will be installed in the system layer engine), but the PGF engine will not know that this command has been executed, nor will it be aware of the effects this command has had on the system-layer engine. The PGF engine sees only PGF-layer commands and only its own state.

This answers OP's question: "In what sense does PGF not keep track of canvas transformations?"

We will next look at an example, however in order to be able to explain it properly, we need first to describe how PGF pictures are rendered on the page.

B) How a PGF picture is rendered on a PDF page

The system layer commands that are generated during the life of a PGF picture, be it those specified directly by the user, or those generated by a PGF-layer command, are not immediately sent to the PDF driver for rendering, rather they are recorded - or protocoled, to use PGF's jargon - and saved to be replayed once the picture is complete. The reason for this is that the positioning of the picture on the page depends on the final size and PGF-coordinates of the bounding box, and these are not known for sure until all of the picture's commands have been executed.

The coordinate system used by the PGF layer is abstract and does not reference physical locations. On the other hand, the coordinate system used by the system layer (or, more precisely, by the PDF engine, but the distinction can be safely ignored here) references the current physical page on which the picture is to be rendered in the following way: the coordinate system's origin is at the page's lower left corner, the x-axis increases along the page's lower edge, and the y-axis increases along the page's left edge. These are the system-layer's initial settings, however they can be changed by executing system-layer transformations.

When a PGF picture has completed and is ready to be rendered, the system-layer coordinates (x,y) of the physical location on the page where the lower-left corner of the bounding box is to be are established, and then a system-layer transformation is executed to shift the origin of the system-layer coordinate system to (x,y). Finally, the protocoled sequence of system-layer commands is executed to render the picture on the page.

Once the picture has been rendered, normal TeX processing resumes. The next letter will be positioned a distance equivalent to the length of the lower edge of the picture's bounding box to the right of the location that the bounding box's lower-left corner is mapped to on the page.

C) An example

Now, let's look at an example. Consider the following LaTeX manuscript.

\documentclass{article}
\usepackage{pgf}
\usepackage{lipsum}
\begin{document}
% 1st paragraph: Only text, no picture.
\lipsum[1]

% 2nd paragraph: Text preceded by a PGF picture. No transformations.
\begin{pgfpicture}
% A rectangle whose lower-left corner is at the origin,
% and upper-rigth corder is at (1cm,2cm).
% The rectangle is filled, but its circumference is not stroked.
\pgfpathrectangle{\pgfpointorigin}{\pgfpoint{1cm}{2cm}}
\pgfsetfillcolor{blue!20}
\pgfusepath{fill}

% The picture's bounding box is stroked.
\pgfpathrectangle{\pgfpointanchor{current bounding box}{south west}}
{\pgfpointanchor{current bounding box}{north east}}
\pgfusepath{stroke}
\end{pgfpicture}%
\lipsum[1]

% 3rd paragraph: Text preceded by a PGF picture.
% The picture is rotated 90 degrees counter-clockwise
% using a canvas transformation.
% The rectangle is filled, but its circumference is not stroked.
% An attempt is then made to stroke the picture's bounding box
% using PGF-layer commands, while the canvas transformation
% is still in effect.
\begin{pgfpicture}
% Allow the character '@' to be a part of a macro's name.
\makeatletter
% Install the canvas transformation matrix
%  0 1
% -1 0
% which represents a rotation by 90 degrees counter-clockwise.
\pgfsys@transformcm{0}{1}{-1}{0}{0pt}{0pt}
% Restore original '@' settings.
\makeatother

% A rectangle is painted as in paragraph #2 above.
\pgfpathrectangle{\pgfpointorigin}{\pgfpoint{1cm}{2cm}}
\pgfsetfillcolor{blue!20}
\pgfusepath{fill}

% An attempt is made to stroke the picture's bounding box
% using the same command as the one used in paragraph #2 above.
\pgfpathrectangle{\pgfpointanchor{current bounding box}{south west}}
{\pgfpointanchor{current bounding box}{north east}}
\pgfusepath{stroke}
\end{pgfpicture}%
\lipsum[1]
\end{document}


The resulting PDF document looks like this (the document's margin's have been clipped off):

D) The example explained

The first paragraph has no PGF picture embedded in it, so we can see where a paragraph's first line naturally starts.

The second paragraph starts with a PGF picture of a rectangle whose lower left corner is at the origin and whose upper right corner is at (1cm,2cm). This picture was coded in pure PGF, and no transformations were applied. The rectangle is filled, but its circumference is not outlined. The stroked boundary that is seen is the picture's bounding box, which was explicitly stroked. A picture's bounding box is a PGF concept. The system-layer has no such notion. The PGF engine keeps track of the bounding box in its state.

The third paragraph, too, starts with a PGF picture. The picture's code is almost identical to the one used in the picture of the 2nd paragraph, except that a system-layer rotation by 90 degrees counter-clockwise has been inserted at the beginning. When this command executes, a corresponding transformation matrix is installed in the system layer engine. However, as mentioned in section A, the following PGF code knows nothing about this. As far as the PFG engine is concerned, it's as if this line was not there.

Now PGF paints a rectangle. As far as PGF is concerned, the painted rectangle is the same as the one drawn in paragraph 2; in particular, it is not rotated in any way, it's coordinates being (0,0) for the lower left corner, and (1cm,2cm) for the upper right corner. However, when the PGF engine sends the rectangle's coordinates to the system-layer engine, these coordinates are interpreted by the system-layer engine in light of the installed rotation. Thus the rectangle will be rendered "lying on its side", but as far as the PGF-engine is concerned, the rectangle stands "upright".

Next the bounding box is stroked. As mentioned in section A, a bounding box is a PGF concept, not a system-layer concept, and the PGF engine calculates it based solely on its own state. This state is the same at this point as it was during the execution of the picture in paragraph 2 when the bounding box was calculated. Therefore, the resulting bounding box is the same as was calculated there, namely running along the rectangle's circumference. Thus, PGF now sends commands to the system-layer engine to draw a rectangle with the same coordinates as the previously painted rectangle. As before, the system-layer interprets these coordinates in light of the installed rotation, and therefore the resulting rectangle-outline will likewise be rendered "on its side". As before, as far as the PGF engine is concerned, the picture's bounding box is upright: its lower-left corner is at the origin, and its upper-right corner is at (1cm,2cm).

Now the picture's scope has reached its end, and the picture is ready for rendering. As explained in section B, a system-layer transformation is executed to shift the origin of the system-layer coordinate system, which is initially at the lower-left corner of the current physical page, to the correct point on the page, given the PGF-coordinates of the bounding box and the physical location on the page where the bounding box's lower-left corner is. Since the PGF engine's state is the same as it was at the end of the paragraph 2 picture, the origin of the system-layer coordinate system will be shifted to coincide with the point where the 'Lorem ipsum' text would have been positioned in the absence of the present picture.

The picture is now rendered, and normal TeX processing resumes, and the next letter is rendered a distance equivalent to the length of the lower edge of the picture's bounding box to the right of the physical location on the page where the bounding box's lower left vertex is. Since the current PGF engine's state is the same as it was at the end of the paragraph 2 picture, this length will be the same as it was there.

• It just isn't true that the TikZ layer is implemented only in terms of the PGF layer and not the system layer. Try grepping for @sys in tikz.code.tex. These commands are not restricted to code forming the PGF layer! – cfr Jul 28 '17 at 22:01
• If the layers were as distinct as you claim, it would be very hard to see how one could be implemented in the other. Moreover, it is really misleading to talk of three different 'engines', none of which are the engine doing the compilation. If your explanation was right, PGF/TikZ would have no idea about anything happening at the system layer. Canvas transformations would be nothing special in this respect. – cfr Jul 28 '17 at 22:13
• @cfr: I'll quote a comment that you wrote to me in a different thread when I complained about the accuracy of an algorithm described in the TikZ & PGF manual: "You are treating as an 'algorithm' what is a rough description." My description may not be 100% accurate, but it's good enough for the present purposes, and it is in the spirit of the TikZ/PGF design, if not in the letter of the source code. – Evan Aad Jul 29 '17 at 3:46
• @cfr: The idea of TikZ/PGF being structured in hierarchical layers is not mine, it is Till Tantau's as he describes in the manual (section 1.1), and the fact that, in theory if not in practice, TikZ is implemented strictly in terms of PGF is again his: "In theory, TikZ itself is just one of several possible “frontends,” which are sets of commands or a special syntax that makes using the basic layer easier." – Evan Aad Jul 29 '17 at 3:46
• @cfr: I will end with a quote from Knuth's The TeXbook: "Another noteworthy characteristic of this manual is that it doesn’t always tell the truth. When certain concepts of TEX are introduced informally, general rules will be stated; afterwards you will find that the rules aren’t strictly true. [...] The author feels that this technique of deliberate lying will actually make it easier for you to learn the ideas. Once you understand a simple but false rule, it will not be hard to supplement that rule with its exceptions." – Evan Aad Jul 29 '17 at 3:47