In math mode, typing '
is equivalent to typing ^{\prime}
, ''
to ^{\prime\prime}
and so on.
If you need “real” quotes in math mode, you have to define a math character for them. Typing `
is not guaranteed to give the desired result.
You can define
\DeclareMathSymbol{\mlq}{\mathord}{operators}{``}
\DeclareMathSymbol{\mrq}{\mathord}{operators}{`'}
Complete example:
\documentclass{article}
\DeclareMathSymbol{\mlq}{\mathord}{operators}{``}
\DeclareMathSymbol{\mrq}{\mathord}{operators}{`'}
\DeclareMathSymbol{\mlqq}{\mathord}{operators}{"5C}
\DeclareMathSymbol{\mrqq}{\mathord}{operators}{`"}
\begin{document}
$\mlq a\mrq$
$\mlqq a\mrqq$
\end{document}

See https://tex.stackexchange.com/a/161253/4427 for a similar setup, but in that case relations were concerned.
`abc'
?\usepackage{amsmath}
and$abc{\text{'}}$
.