How to capture the current math style?

Question

\mathchoice helps to distinguish between math styles:

\mathchoice
  {<something in \displaystyle>}
  {<something in \textstyle>}
  {<something in \scriptstyle>}
  {<something in \scriptscriptstyle>}

How would one use \mathchoice to capture the current math style? The following does not work:

\documentclass{article}
\makeatletter
\newcommand{\getmathstyle}{%
  \mathchoice
    {\global\def\curmathstyle{\displaystyle}}
    {\global\def\curmathstyle{\textstyle}}
    {\global\def\curmathstyle{\scriptstyle}}
    {\global\def\curmathstyle{\scriptscriptstyle}}
}
\makeatother
\begin{document}
\ttfamily
$\getmathstyle$ \meaning\curmathstyle\par
$\displaystyle\getmathstyle$ \meaning\curmathstyle\par
$\scriptstyle\getmathstyle$ \meaning\curmathstyle\par
$\scriptscriptstyle\getmathstyle$ \meaning\curmathstyle
\end{document}

Answer 1

The mathstyle package provides a means to tap into the current math style and redefines \mathchoice as a switch. As such, the following is a possible solution:

\documentclass{article}
\usepackage{mathstyle}% http://ctan.org/pkg/mathstyle
\makeatletter
\newcommand{\getmathstyle}{
  \global\edef\curmathstyle{\expandafter\@gobble\mathchoice{\@@displaystyle}{\@@textstyle}{\@@scriptstyle}{\@@scriptscriptstyle}}
}
\makeatother
\begin{document}
\ttfamily
\verb|\textstyle|: $xyz\getmathstyle$ \meaning\curmathstyle \par
\verb|\displaystyle|: $\displaystyle xyz\getmathstyle$ \meaning\curmathstyle \par
\verb|\scriptstyle|: $\scriptstyle xyz\getmathstyle$ \meaning\curmathstyle \par
\verb|\scriptscriptstyle|: $\scriptscriptstyle xyz\getmathstyle$ \meaning\curmathstyle
\end{document}

\mathpalette acts like a server for \mathchoice (see The mysteries of \mathpalette) and can be used to extract the math style in a slightly more elegant way:

%...
\usepackage{mathstyle}% http://ctan.org/pkg/mathstyle
\newcommand{\getmathstyle}{%
  \mathpalette{\global\let\curmathstyle}{\relax}%
}
%...

Answer 2

LuaTeX provides a \mathstyle primitive which takes the following values for different math styles:

• 0 = display
• 1 = crampeddisplay
• 2 = text
• 3 = crampedtext
• 4 = script
• 5 = crampedscript
• 6 = scriptscript
• 7 = crampedscriptscript

Here is an example usage:

\starttext
\startlines
$\displaystyle                  \sum_{k=0}^n (a_k + b_k): \mathstyle$
$\crampeddisplaystyle           \sum_{k=0}^n (a_k + b_k): \mathstyle$
$\textstyle                     \sum_{k=0}^n (a_k + b_k): \mathstyle$
$\crampedtextstyle              \sum_{k=0}^n (a_k + b_k): \mathstyle$
$\scriptstyle                   \sum_{k=0}^n (a_k + b_k): \mathstyle$
$\crampedscriptstyle            \sum_{k=0}^n (a_k + b_k): \mathstyle$
$\scriptscriptstyle             \sum_{k=0}^n (a_k + b_k): \mathstyle$
$\crampedscriptscriptstyle      \sum_{k=0}^n (a_k + b_k): \mathstyle$
\stoplines
\stoptext

which gives

You can do something based on the current mathstyle by using

\ifcase\mathstyle 
 ...
 \or
 ...
 \or
 ... % etc.
\fi

An interesting usage is

$\mathstyle_{\mathstyle_{\mathstyle}}$

which gives showing that cramped style is active in subscripts and subsubscripts.

Answer 3

I probably should have posted this answer (Proper use of \mathchoice) here, but I will just refer you to it. In the MWE at that post, I introduce the syntax

\MS{...\SavedMathStyle...}

which can also be nested as

\MS{...\SavedMathStyle...\MS{...\SavedMathStyle...}...}

The invocation of \MS saves the current math style, which can later be recalled via \SavedMathStyle. A final noteworthy point is that this approach uses the TeX primitive \mathchoice, which will not suffer the many compatibility issues that others have noted with the mathstyle package.