With your choice of packages, the simple
\documentclass[12pt]{article}
\usepackage[utf8]{inputenc}
\begin{document}
ā, ä, ā, ä, ǟ, ḑ, ē, ī
\end{document}
would raise the same errors, since ā ǟ ḑ ē ī
are not supported by the utf8
option, even if \usepackage[T1]{fontenc}
is loaded. So it's no surprise they don't work in an mpost
environment. On the other hand
\documentclass[12pt]{article}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage[shellescape,latex]{gmp}
\gmpoptions{everymp={input expressg;}}
\usempxpackage[T1]{fontenc}
\usempxpackage[utf8]{inputenc}
\begin{document}
\begin{figure}[htp]
\centering
\begin{mpost}
z0 = origin;
drawroundedbox(0, 2.5cm, 3cm, 5mm)(
\btex {\begin{tabular}{@{}c@{}} ä, ă, ö \end{tabular}} etex
);
\end{mpost}
\end{figure}
\end{document}
works flawlessly, producing

This works, by teaching LaTeX how it should deal with the unknown characters:
\documentclass[12pt]{article}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage[shellescape,latex]{gmp}
\gmpoptions{everymp={input expressg;}}
\usempxpackage[T1]{fontenc}
\usempxpackage[utf8]{inputenc}
\usempxpackage{newunicodechar}
\mpxcommands{%
\newunicodechar{ā}{\=a}%
\newunicodechar{ē}{\=e}%
\newunicodechar{ī}{\={\i}}%
\newunicodechar{ḑ}{\d{d}}%
\newunicodechar{ǟ}{\={ä}}%
}
\begin{document}
\begin{figure}[htp]
\centering
\begin{mpost}
z0 = origin;
drawroundedbox(0, 2.5cm, 3cm, 5mm)(
\btex {\begin{tabular}{@{}c@{}} ā, ä, ā, ä, ǟ, ḑ, ē, ī \end{tabular}} etex
);
\end{mpost}
\end{figure}
\end{document}

Of course, the same declarations should be made for the main document, if you want those characters also in normal text.
national character
?$\alpha$
is a mathematical symbol. If you wanted to typeset Greek, you wouldn't do it this way. And\alpha
doesn't seem to have much to do with typesetting letters with a diaeresis. Is the point that you want to input these as unicode characters? Or...?