The following is meant to complement cfr's answer by providing more detailed explanations.
Where I describe the workings of TikZ/PGF "under the hood", it is not meant to be understood as a rigorous guide to the source code, but as an informal, but hopefully helpful and illuminating explanation of the behavior perceived by a user. To paraphrase The Textbook, this post doesn’t always tell the truth, but the author feels that this technique of deliberate lying will actually make it easier for you to understand the ideas.
One particular inaccuracy that I should mention is the use of the word "engine". When I talk about "the PGF-layer engine" or "the system-layer engine", I mean it conceptually rather than technically, somewhat in the spirit of objects in an object-oriented program. In reality there is only one engine that is at work typesetting a TeX document, and that is pdftex, or one of its siblings.
A) The layered design of the PGF/TikZ bundle
The TikZ/PGF bundle is structured in hierarchical layers: 1. system, 2. PGF, 3. TikZ. Each layer is independent of the following layers. Thus, the system layer knows nothing of PGF, whereas PGF uses the system layer, but knows nothing of TikZ. Each layer is implemented in terms of the layer directly below it: the system layer is the only one that "talks" to the rendering driver (e.g. the PDF driver, when the LaTeX engine is pdftex/pdflatex), the PGF layer is implemented in terms of the system layer, and TikZ is implemented in terms of the PGF layer and never access the system layer directly. The layers can coexist: a PGF picture can include system-layer commands, and a TikZ picture can include both PGF and system-layer commands. When command from different layers are intermixed, each command will be uniquely processed by the "engine" that deals with the corresponding layer. We will now restrict our attention to the two lower layers: the system layer, and the PGF layer.
Changes made in one layer do not propagate automatically to the other layer, and there is no event mechanism that enables one layer to "listen to" the other layer in real time. Moreover, the layers never query the state of the other layer; each layers keeps its own state and this is the only state it ever accesses.
Thus, system-layer commands that are written inside a PGF picture are invisible to the PGF engine. When a system-layer command is executed in the context of a PGF picture, its effects will take place (for instance, a transformation will be installed in the system layer engine), but the PGF engine will not know that this command has been executed, nor will it be aware of the effects this command has had on the system-layer engine. The PGF engine sees only PGF-layer commands and only its own state.
This answers OP's question: "In what sense does PGF not keep track of canvas transformations?"
We will next look at an example, however in order to be able to explain it properly, we need first to describe how PGF pictures are rendered on the page.
B) How a PGF picture is rendered on a PDF page
The system layer commands that are generated during the life of a PGF picture, be it those specified directly by the user, or those generated by a PGF-layer command, are not immediately sent to the PDF driver for rendering, rather they are recorded - or protocoled, to use PGF's jargon - and saved to be replayed once the picture is complete. The reason for this is that the positioning of the picture on the page depends on the final size and PGF-coordinates of the bounding box, and these are not known for sure until all of the picture's commands have been executed.
The coordinate system used by the PGF layer is abstract and does not reference physical locations. On the other hand, the coordinate system used by the system layer (or, more precisely, by the PDF engine, but the distinction can be safely ignored here) references the current physical page on which the picture is to be rendered in the following way: the coordinate system's origin is at the page's lower left corner, the x-axis increases along the page's lower edge, and the y-axis increases along the page's left edge. These are the system-layer's initial settings, however they can be changed by executing system-layer transformations.
When a PGF picture has completed and is ready to be rendered, the system-layer coordinates (x,y) of the physical location on the page where the lower-left corner of the bounding box is to be are established, and then a system-layer transformation is executed to shift the origin of the system-layer coordinate system to (x,y). Finally, the protocoled sequence of system-layer commands is executed to render the picture on the page.
Once the picture has been rendered, normal TeX processing resumes. The next letter will be positioned a distance equivalent to the length of the lower edge of the picture's bounding box to the right of the location that the bounding box's lower-left corner is mapped to on the page.
C) An example
Now, let's look at an example. Consider the following LaTeX manuscript.
\documentclass{article}
\usepackage{pgf}
\usepackage{lipsum}
\begin{document}
% 1st paragraph: Only text, no picture.
\lipsum[1]
% 2nd paragraph: Text preceded by a PGF picture. No transformations.
\begin{pgfpicture}
% A rectangle whose lower-left corner is at the origin,
% and upper-rigth corder is at (1cm,2cm).
% The rectangle is filled, but its circumference is not stroked.
\pgfpathrectangle{\pgfpointorigin}{\pgfpoint{1cm}{2cm}}
\pgfsetfillcolor{blue!20}
\pgfusepath{fill}
% The picture's bounding box is stroked.
\pgfpathrectangle{\pgfpointanchor{current bounding box}{south west}}
{\pgfpointanchor{current bounding box}{north east}}
\pgfusepath{stroke}
\end{pgfpicture}%
\lipsum[1]
% 3rd paragraph: Text preceded by a PGF picture.
% The picture is rotated 90 degrees counter-clockwise
% using a canvas transformation.
% The rectangle is filled, but its circumference is not stroked.
% An attempt is then made to stroke the picture's bounding box
% using PGF-layer commands, while the canvas transformation
% is still in effect.
\begin{pgfpicture}
% Allow the character '@' to be a part of a macro's name.
\makeatletter
% Install the canvas transformation matrix
% 0 1
% -1 0
% which represents a rotation by 90 degrees counter-clockwise.
\pgfsys@transformcm{0}{1}{-1}{0}{0pt}{0pt}
% Restore original '@' settings.
\makeatother
% A rectangle is painted as in paragraph #2 above.
\pgfpathrectangle{\pgfpointorigin}{\pgfpoint{1cm}{2cm}}
\pgfsetfillcolor{blue!20}
\pgfusepath{fill}
% An attempt is made to stroke the picture's bounding box
% using the same command as the one used in paragraph #2 above.
\pgfpathrectangle{\pgfpointanchor{current bounding box}{south west}}
{\pgfpointanchor{current bounding box}{north east}}
\pgfusepath{stroke}
\end{pgfpicture}%
\lipsum[1]
\end{document}
The resulting PDF document looks like this (the document's margin's have been clipped off):
D) The example explained
The first paragraph has no PGF picture embedded in it, so we can see where a paragraph's first line naturally starts.
The second paragraph starts with a PGF picture of a rectangle whose lower left corner is at the origin and whose upper right corner is at (1cm,2cm). This picture was coded in pure PGF, and no transformations were applied. The rectangle is filled, but its circumference is not outlined. The stroked boundary that is seen is the picture's bounding box, which was explicitly stroked. A picture's bounding box is a PGF concept. The system-layer has no such notion. The PGF engine keeps track of the bounding box in its state.
The third paragraph, too, starts with a PGF picture. The picture's code is almost identical to the one used in the picture of the 2nd paragraph, except that a system-layer rotation by 90 degrees counter-clockwise has been inserted at the beginning. When this command executes, a corresponding transformation matrix is installed in the system layer engine. However, as mentioned in section A, the following PGF code knows nothing about this. As far as the PFG engine is concerned, it's as if this line was not there.
Now PGF paints a rectangle. As far as PGF is concerned, the painted rectangle is the same as the one drawn in paragraph 2; in particular, it is not rotated in any way, it's coordinates being (0,0) for the lower left corner, and (1cm,2cm) for the upper right corner. However, when the PGF engine sends the rectangle's coordinates to the system-layer engine, these coordinates are interpreted by the system-layer engine in light of the installed rotation. Thus the rectangle will be rendered "lying on its side", but as far as the PGF-engine is concerned, the rectangle stands "upright".
Next the bounding box is stroked. As mentioned in section A, a bounding box is a PGF concept, not a system-layer concept, and the PGF engine calculates it based solely on its own state. This state is the same at this point as it was during the execution of the picture in paragraph 2 when the bounding box was calculated. Therefore, the resulting bounding box is the same as was calculated there, namely running along the rectangle's circumference. Thus, PGF now sends commands to the system-layer engine to draw a rectangle with the same coordinates as the previously painted rectangle. As before, the system-layer interprets these coordinates in light of the installed rotation, and therefore the resulting rectangle-outline will likewise be rendered "on its side". As before, as far as the PGF engine is concerned, the picture's bounding box is upright: its lower-left corner is at the origin, and its upper-right corner is at (1cm,2cm).
Now the picture's scope has reached its end, and the picture is ready for rendering. As explained in section B, a system-layer transformation is executed to shift the origin of the system-layer coordinate system, which is initially at the lower-left corner of the current physical page, to the correct point on the page, given the PGF-coordinates of the bounding box and the physical location on the page where the bounding box's lower-left corner is. Since the PGF engine's state is the same as it was at the end of the paragraph 2 picture, the origin of the system-layer coordinate system will be shifted to coincide with the point where the 'Lorem ipsum' text would have been positioned in the absence of the present picture.
The picture is now rendered, and normal TeX processing resumes, and the next letter is rendered a distance equivalent to the length of the lower edge of the picture's bounding box to the right of the physical location on the page where the bounding box's lower left vertex is. Since the current PGF engine's state is the same as it was at the end of the paragraph 2 picture, this length will be the same as it was there.
\end{pgfpicture}?