How to label arrows in tikz-cd diagrams with matrices?

I'm using tikz-cd for some commutative diagrams, but if I try to label an arrow with a matrix, & appears to be "undefined control sequence". So all the entries in a row get concatenated, and it looks like a column vector. The following is a (minimal) working example.

\documentclass{article}
\usepackage{amsmath}
\usepackage{tikz-cd}
\begin{document}

$\begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}$

\begin{tikzcd}
A \arrow{r}{ \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} } & B
\end{tikzcd}

\end{document}

-

This is because the tikz-cd environment is a wrapper around a TikZ matrix and in a TikZ matrix then & doesn't mean what it usually means. Inside a TikZ matrix then & is a command which does a fair bit extra than the usual "hop to the next cell". However, inside an AMS matrix, you want it to be just the "hop to the next cell" that it usually is. So the two occurrences of & in your code have different meanings but you are using the same symbol for them. There are a variety of ways to fix this:

1. Keep the normal meaning of & so that it works inside the AMS matrix, in which case you need to use something else for the TikZ matrix. Fortunately, TikZ provides a way to do this: you write [ampersand replacement=\&] (or whatever) after the \begin{tikzcd} and then use \& (or whatever) for the TikZ cell separator.

2. Keep TikZ's meaning of & so that it works for the TikZ matrix, in which case you need to use something else for the AMS matrix. This is also quite straightforward. You put \let\amsamp=& in your preamble and then use \amsamp in your code (unfortunately, it would appear that \& can't be used here).

3. Rejig AMS's matrix environment to reset & to its usual meaning whilst inside its environs. This isn't quite as easy as it looks due to the fact that when the environment is read then the catcode of & is already frozen (as it is in the argument to a macro, in this case \arrow). So we have to do a bit of \begingroup ... \endgroup trickery when defining the new type of matrix environment. (Obviously, to do this right one should just redefine the basic AMS matrix environment that all the others are built on top of). Note that this also works outside tikzcd environments because what it actually does is say "If & is a command, let it be what & usually is." (okay, it's slightly more complicated but that'll do for now) and in the normal circumstances & isn't a command so this is (almost) a NO-OP.

\documentclass{article}
%\url{http://tex.stackexchange.com/q/81618/86}
\let\amsamp=&

\usepackage{amsmath}
\usepackage{tikz-cd}

\begingroup
\catcode\&=13
\gdef\pampmatrix{%
\begingroup
\let&=\amsamp
\begin{pmatrix}%
}
\gdef\endpampmatrix{\end{pmatrix}\endgroup}
\endgroup

\begin{document}

$\begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}$

\begin{tikzcd}[ampersand replacement=\&]
A \arrow{r}{ \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} } \& B
\end{tikzcd}

\begin{tikzcd}
A \arrow{r}{ \begin{pmatrix} 1 \amsamp 0 \\ 0 \amsamp 1 \end{pmatrix}
} & B
\end{tikzcd}

\begin{tikzcd}
A \arrow{r}{ \begin{pampmatrix}
1 & 0 \\ 0 & 1 \end{pampmatrix} } & B
\end{tikzcd}

$\begin{pampmatrix} 1 & 0 \\ 0 & 1 \end{pampmatrix}$
\end{document}
`

As a postscript (or PDF), I would like to point out that this is a lovely example of the distinction between fudge, Finagle, and diddle factors as explained in an article in THE IRE STUDENT QUARTERLY, SEPTEMBER 1958 (also reproduced in the wonderful book A Random Walk in Science, or it might be in More Random Walks in Science). The following is the relevant part:

However, John W. Campbell feels there is a different basic structure behind the Finagle, fudge and diddle factors. The Finagle factor, he claims, is characterized by changing the universe to fit an equation. The fudge factor, on the other hand, changes the equation to fit the universe. And finally, the diddle factor changes things so that the universe and the equation appear to fit, without making any real change in either.

For example, the planet Uranus was introduced to the universe when Newtonian laws couldn't be to match known planetary motions. This is a beautiful example of the application of the Finagle factor.

Einstein's work leading to relativity was strongly influenced by the observed facts about the orbit of Mercury. Obviously a fudge factor was introduced.

The photographer's use of a "soft-focus" lens when taking portraits of women over 35 is an example of the diddle factor. By blurring the results, photographs are made to appear to match the facts in a far more satisfactory manner.

Thus the first solution, which changes TikZ to fit AMS, is clearly an application of the Finagle factor. The second, which changes AMS to fit TikZ, is obviously a fudge. The last, in which everything suddenly appears to work with no outward change, is a definite diddle.

-