Redefining \^ and \~ in math mode?

\^ and \~ apply accents in text mode, but each gives an error when used in math mode. So I consider redefining them for my own use, perhaps like this:

\let\old@hat\^
\def\^{\ifmmode\mbox{\textasciicircum}\else\old@hat\fi}


(Or we could replace \mbox with \text if we've loaded amsmath.)

Besides the slight hit to efficiency, and perhaps confusing some others who read the source of my documents, are there any other downsides to this that I'm not recognizing?

-
what's wrong with \hat and \tilde? Keep the short commands for text and the long ones for math. – yo' Aug 20 '14 at 13:05
Those existing commands provide accents on the token that follows. I wanted to escape the ^ and ~ to get the raw characters (with bigger, lower shapes than one gets by doing \hat{} or \tilde{}). – dubiousjim Aug 20 '14 at 13:15
like \wedge and \sim ? – yo' Aug 20 '14 at 13:18
Bigger than $\hat{}$ but smaller than $\wedge$. It's not so important. For this question, I was interested in what the downsides might be to giving \^ a definition in math mode, regardless of what one defines it to expand to. – dubiousjim Aug 20 '14 at 20:04

Unless you take more precautions the \ifmmode will take the wrong choice at the start of any array or alignment cell, that could be fixed using a suitable \protected\def or \DeclareRobustCommand or simply starting the definition with \relax but even then there are few if any cases where the text character \mbox{\textasciicircum} is useful in math, it will not use the math fonts, and will not size correctly in superscripts. Compared to the math accent \hat or the math symbol \wedge.