Wrong parentheses size in \binom with xelatex and unicode-math in displaystyle

Question

I am writing thesis, using XeLaTeX with unicode-math, documentclass is modified memoir. I need to typeset equations with binomial coefficient, however I get wrong sizes of parentheses around them.

And unicode-math seems to be ruining the output of \binom, \choose, \genfrac, etc.

When I do:

$$\binom{n}{k}\qquad \tbinom{n}{k} \qquad \dbinom{n}{k}$$
$$\binom{5}{6}\qquad \tbinom{5}{6} \qquad \dbinom{5}{6}$$

I get binomial coefficient with too small parentheses around it:

I’ve tried renewcommanding binom by:

\renewcommand{\binom}[2]{\genfrac{(}{)}{0pt}{}{#1}{#2}}

with no success, however placing it between \left( and \right) gives correct bigger parentheses.

I have set non-standard fonts (see below), but disabling them doesn’t change this.

Using {n \choose k} gives the very same output… How to overcome this problem?

Minimal code generating this is:

\documentclass[a4paper,oneside]{memoir}
\usepackage{amsmath,unicode-math,xltxtra}

% Those are needed to use same fonts as I do.
% Parentheses are strange even without them
%\setromanfont[Mapping=tex-text]{Linux Libertine O}
%\setsansfont[Mapping=tex-text]{Ubuntu}
%\setmonofont{Ubuntu Mono}
%\setmathfont{Latin Modern Math}
%\setmathfont[range=\mathit/{latin,Latin,num,Greek,greek}]{Linux Libertine O Italic}
%\setmathfont[range=\mathup/{latin,Latin,num,Greek,greek}]{Linux Libertine O}
%\setmathfont[range=\mathbfup/{latin,Latin,num,Greek,greek}]{Linux Libertine O Bold}
%\setmathfont[range={"221E}]{Linux Libertine O}% "0221E = \infty

\begin{document}

$$\binom{n}{k}\qquad \tbinom{n}{k} \qquad \dbinom{n}{k}$$
$$\binom{5}{6}\qquad \tbinom{5}{6} \qquad \dbinom{5}{6}$$

\end{document}

EDIT: Simple working workaround

Redefining \binom this way makes it look acceptably, at least for my set of fonts:

\renewcommand{\dbinom}[2]{\left(\!\!\begin{array}{c}{#1} \\ {#2} \end{array}\!\!\right)}
\let\oldbinom\binom
\renewcommand{\binom}[2]{\mathchoice{\dbinom{#1}{#2}}{\oldbinom{#1}{#2}}{\oldbinom{#1}{#2}}{\oldbinom{#1}{#2}}}

Answer 1

The unicode-math and stix/xits fonts are natively OpenType fonts. Setting of math is accomplished by means of parameters provided by the OTF math table.

The OpenType mechanism was a creation of Microsoft. The math table, although it is based largely on the mechanism used by TeX, as described in appendix G of the TeXbook, lacks two of the font parameters required by TeX, sigma20 and sigma21, whose use is defined in rule 15e on p.445. The primitives \overwithdelims, \atopwithdelims and \abovewithdelims require appropriate settings of these parameters in order to work properly.

Ulrik Vieth reported on the absence of these two parameters in his talk, "OpenType Math Illuminated", presented at BachoTeX 2009; the reference is on p.19 of the slides.

These primitives wrap delimiters (usually a matched pair of parentheses, brackets or braces, but there are other possibilities) around a "generalized fraction", a vertically aligned pair of symbols with or without a fraction line. They are the structure on which the amsmath commands \binom and others using \genfrac are based. There is no other "native" mechanism for setting such delimiters with any TeX engine.

Although the absence of these parameters has been known for some time, only relatively recently has the extremely serious nature of the implications of their absence become widely obvious, with near-final testing of the stix fonts.

This is not an easy problem to solve or work around. The two "obvious" approaches are (1) change the TeX program (not going to happen), or (2) upgrade the OTF math table. There are serious "political" difficulties involved in implementing the second approach. So, for the moment, proper correction of this flaw is on hold.

Answer 2

Update

The implementation of \genfrac (and so \binom) has been altered in Unicode TeXs in the 2016-11-05 amsmath.dtx 2.16a release of amsmath and the example in the question now produces

\documentclass{article}

\usepackage{amsmath}

\begin{document}

$$\binom{n}{k}\qquad \tbinom{n}{k} \qquad \dbinom{n}{k}$$
$$\binom{5}{6}\qquad \tbinom{5}{6} \qquad \dbinom{5}{6}$$
\end{document}

The code in amsmath is basically following the outline sketched in the answer below.

---

Original Answer

An updated version more closely matching the tighter spacing and larger minimum size of generalised fraction brackets is given by the following, which may be run with pdftex or xetex, to compare using lm fonts.

\documentclass[fleqn,a4paper]{article}

\addtolength\textwidth{40pt}
\usepackage{amsmath}
\ifx\Umathchar\undefined
\usepackage{lmodern}
\else
\usepackage{unicode-math}
\fi


\makeatletter
\ifx\Umathcode\@undefined\else

\DeclareRobustCommand{\genfrac}[6]{%
  \def\@tempa{#1#2}%
  \edef\@tempb{\@nx\@genfrac\@mathstyle{#4}%
    % just \over or \above never withdelims versions
    \ifx @#3@\@@over\else\@@above\fi 
    }%
\ifx\@tempa\@empty \else
\bgroup % so mathord not mathinner
\left\ifx\relax#1\relax.\else#1\fi % assumes . is null delimiter 
                                   % otherwise add specified delimiter
\kern-\nulldelimiterspace % fractions add extra nulldelimiter space
\fi
  \@tempb{#3}{\mathstrut{#5}}{\mathstrut{#6}}%
\ifx\@tempa\@empty \else
\kern-\nulldelimiterspace
\right\ifx\relax#2\relax.\else#2\fi
\egroup
\fi
}
\fi
\makeatother
\begin{document}

\[\binom{n}{k}\qquad \tbinom{n}{k} \qquad \dbinom{n}{k}\]
\[\binom{5}{6}\qquad \tbinom{5}{6} \qquad \dbinom{5}{6}\]
\[ 2^k-\binom{k}{1}2^{k-1}+\binom{k}{2}2^{k-2}\]
\[\sum_i\left[\binom{n_1}{i}
    \binom{n_2}{n_3-n_1+i}\binom{n_3}{n_3-n_2+i} + \binom{n_1-1}{i}
    \binom{n_2-1}{n_3-n_1+i} \binom{n_3-1}{n_3-n_2+i}\right]\]
\end{document}

---

Original answer:

If you want to use \left/\right I don't think you need an array and \mathchoice just use \atop or \over without the delimiters, then wrap in left/right pair.

\documentclass[a4paper,oneside]{memoir}


\usepackage{amsmath,unicode-math,xltxtra}

% Those are needed to use same fonts as I do.
% Parentheses are strange even without them
%\setromanfont[Mapping=tex-text]{Linux Libertine O}
%\setsansfont[Mapping=tex-text]{Ubuntu}
%\setmonofont{Ubuntu Mono}
%\setmathfont{Latin Modern Math}
%\setmathfont[range=\mathit/{latin,Latin,num,Greek,greek}]{Linux Libertine O Italic}
%\setmathfont[range=\mathup/{latin,Latin,num,Greek,greek}]{Linux Libertine O}
%\setmathfont[range=\mathbfup/{latin,Latin,num,Greek,greek}]{Linux Libertine O Bold}
%\setmathfont[range={"221E}]{Linux Libertine O}% "0221E = \infty

\makeatletter
\DeclareRobustCommand{\genfrac}[6]{%
  \def\@tempa{#1#2}%
  \edef\@tempb{\@nx\@genfrac\@mathstyle{#4}%
    \csname @@\ifx @#3@over\else above\fi
    \endcsname
    }%
\ifx\@tempa\@empty \else
\left\ifx\relax#1\relax.\else#1\fi
\fi
  \@tempb{#3}{#5}{#6}%
\ifx\@tempa\@empty \else
\right\ifx\relax#2\relax.\else#2\fi
\fi
}
\makeatother
\begin{document}

$$\binom{n}{k}\qquad \tbinom{n}{k} \qquad \dbinom{n}{k}$$
$$\binom{5}{6}\qquad \tbinom{5}{6} \qquad \dbinom{5}{6}$$

\end{document}

Answer 3

With lualatex this is not an unsolvable problem. It tries to give the two missing parameters sensible values (and it if fails one can adapt them):

\documentclass[a4paper,oneside]{memoir}
\usepackage{amsmath,unicode-math}

\begin{document}

\[\binom{n}{k}\qquad \tbinom{n}{k} \qquad \dbinom{n}{k}\]

\Umathfractiondelsize\displaystyle=40pt
\Umathfractiondelsize\textstyle=40pt

\[\binom{n}{k}\qquad \tbinom{n}{k} \qquad \dbinom{n}{k}\]

\end{document}

You could make a feature request for xelatex that it add something similar too.