# Horizontal braces with XeTeX take too much space

After some experiments and tests, I managed to create the following code for \upbracefill and \downbracefill in Plain XeTeX, which it seems is that used for \underbrace and \overbrace:

\def\midshift#1{
\lower.5\dimen0\box0 }

\def\rotatebrace#1{%
\leavevmode\setbox0=\hbox{#1}\rlap{%
\raise.5\dimen0\hbox{\special{x:gsave}\special{x:rotate 90}}}%
\box0\special{x:grestore}}

[...]

\XeTeXmathchardef\bracelu = 0 3 \⎧
\XeTeXmathchardef\bracemu = 0 3 \⎨
\XeTeXmathchardef\braceru = 0 3 \⎩
\XeTeXmathchardef\bracebar = 0 3 \⎪
\XeTeXmathchardef\braceld = 0 3 \⎫
\XeTeXmathchardef\bracemd = 0 3 \⎬
\XeTeXmathchardef\bracerd = 0 3 \⎭

\def\upbracefill{\rotatebrace{\midshift{$\bracelu$}}%
\cleaders\hbox{\rotatebrace{\midshift{$\bracebar$}}}\hfill
\rotatebrace{\midshift{$\bracemu$}}%
\cleaders\hbox{\rotatebrace{\midshift{$\bracebar$}}}\hfill
\rotatebrace{\midshift{$\braceru$}}}%
\def\downbracefill{\rotatebrace{\midshift{$\braceld$}}%
\cleaders\hbox{\rotatebrace{\midshift{$\bracebar$}}}\hfill
\rotatebrace{\midshift{$\bracemd$}}%
\cleaders\hbox{\rotatebrace{\midshift{$\bracebar$}}}\hfill
\rotatebrace{\midshift{$\bracerd$}}}%


Where Cambria Math is loaded for the 3. family.
However, the height of these braces are bigger because of the middle part of the brace, and I can't find a way to shrink/limit the height of the box, without which the brace would fall apart.

I originally used the :vertical parameter for the font, which I figured has no dimension in horizontal text.

So how can I make this horizontal brace height smaller? (preferably to the character width, which is now the height.)

-
I'm really going to have to remember this for unicode-math... –  Will Robertson Sep 26 '10 at 9:51
Again, not a real answer, but if I were you, I'd save myself the trouble and use LuaTeX instead which already have \Umathaccent, \Umathbotaccent, \Umathaccents and more, check ntg.nl/maps/38/04.pdf. You can use luaotfload package with plain TeX, and most of your code can be reused, but you get better quality OpenType math. –  Khaled Hosny Sep 26 '10 at 18:06
I understand and accept that as an option, but generally if I meet a problem, I prefer to solve it, if that is possible. (Not very efficient though.) –  Adam L. S. Sep 27 '10 at 23:18
@Adam, I do so all the time, but emulating TeX's math engine in macros is no fun. –  Khaled Hosny Oct 5 '10 at 8:24
@Khaled: On the contrary, I find it can be lots of “fun” :) –  Will Robertson Oct 5 '10 at 10:48

After I saw that \upbracefill is used for \underbrace, I felt the need to correct this problem, so I take some time to it and look at it to see what choices do I have. Soon I realized that the only good solution is if I shrink the bounding box of each character, with the help of \setbox. (The only problem was to do this in the characters right state.) Since the most biggest character is the middle one, I arrange their height to that. (The rest of that characters height is aligned to that, however 0pt height is also ok.)
So here is the result using the above \rotatebrace and \midshift macros:

\def\upbracefill{%
\setbox0=\hbox{\lower.64ex\hbox{\rotatebrace{\midshift{$\bracemu$}}}}\ht0=.1\wd0\dp0=0pt%
\setbox1=\hbox{\lower.64ex\hbox{\rotatebrace{\midshift{$\bracelu$}}\kern-.2em}}\ht1=.1\wd0\dp1=0pt%
\setbox2=\hbox{\lower.64ex\hbox{\rotatebrace{\midshift{$\bracebar$}}}}\ht2=.1\wd0\dp2=0pt%
\setbox3=\hbox{\lower.64ex\hbox{\kern-.2em\rotatebrace{\midshift{$\braceru$}}}}\ht3=.1\wd0\dp3=0pt%

\def\downbracefill{%
\setbox0=\hbox{\lower.64ex\hbox{\rotatebrace{\midshift{$\bracemd$}}}}\ht0=.1\wd0\dp0=0pt%
\setbox1=\hbox{\lower.64ex\hbox{\rotatebrace{\midshift{$\braceld$}}\kern-.2em}}\ht1=.1\wd0\dp1=0pt%
\setbox2=\hbox{\lower.64ex\hbox{\rotatebrace{\midshift{$\bracebar$}}}}\ht2=.1\wd0\dp2=0pt%
\setbox3=\hbox{\lower.64ex\hbox{\kern-.2em\rotatebrace{\midshift{$\bracerd$}}}}\ht3=.1\wd0\dp3=0pt%

I'd certainly be happily willing to accept a patch to unicode-math` if you were interested in taking a look into it. –  Will Robertson Oct 10 '10 at 16:54