\underbrace{} choppy

Question

Why are my underbraces so ugly? I've tried a few different fonts for the document and it appears it is font that determines the underbraces appearance. I however don't want to change my font. How can I make my underbraces not look so ugly?

I use the following packages (Edit by Caramdir: I removed all non-relevant packages from the list).

\usepackage{amssymb}
\usepackage{amsmath}

Answer 1

I can answer the first part of the question: Why are the underbraces choppy? The reason is that latex constructs the underbrace (also overbraces) from glyphs from a font (for the curly parts) and TeX rules (for the straight parts). The TeX rules translate into PDF filled rectangles, so you are correct that it is not a piece of type. Due to some tricky aspects of how the PDF format works, it's not really possible to ensure that these different parts meet seamlessly in all PDF viewers at all zoom levels.

Large vertical delimiters in TeX avoid this problem by using repeated glyphs also for the straight parts, so that the whole brace is constructed from only glyphs (there can be some slight glitches at the joins, but it is usually much less noticeable).

As for your second question: There are a few ways to deal with this, but none are really satisfactory.

1. Use a different PDF viewer. Some of them use different drawing algorithms that make these glitches less obvious;
2. Use different setting for your current PDF viewer. For example in Adobe reader go to the "Page Display" preferences page and under "Rendering" set "Smooth line art"=ON and "Enhance thin lines"=OFF. (This will have other effects that you may or may not appreciate);

3. Program a different way to generate horizontal braces, more similar to how vertical braces are constructed. Maybe take a look at how latex builds horizontal arrows, for inspiration. This is also how MnSymbol makes underbraces:

   

4. Use a font that provides large horizontal braces (I know you didn't want to change fonts, but...). For example the (commercial) MathTime Professional 2 fonts have individually designed glyphs for braces up to 4inches wide. For example (taken from this webpage):

   

5. Don't worry about it: it will look absolutely fine when printed -- the problems with making the pieces match only apply to antialiased rendering at low resolution such as on a monitor (which PDF was not really designed for, originally). On the printed page (which PDF was designed for) they will match perfectly.
6. Construct the braces using something like TikZ. I'm sure this is easy enough, but I'm not a TikZ expert so I'll let someone else write this.

Answer 2

You may want to read what the mathtools manual has to say about LaTeX's implementation of \underbrace. Try \usepackage{mathtools} and see if you think the result is better.

Answer 3

If you only want the \underbrace command from the MnSymbol package but use the amsmath symbols otherwise, you can edit MnSymbol.sty to that effect. Copy the code below and paste it into a new text file. Save the file as MyMnSymbol.sty into the folder where your TeX document lives. In the preamble of your document, type \usepackage{MyMnSymbol} which then only redefines the \underbrace command.

\ProvidesPackage{MyMnSymbol}[2007/01/21 v1.4 Underbraces from MnSymbol]

\DeclareFontFamily{OMX}{MnSymbolE}{}
\DeclareSymbolFont{largesymbolsX}{OMX}{MnSymbolE}{m}{n}
\DeclareFontShape{OMX}{MnSymbolE}{m}{n}{
    <-6>  MnSymbolE5
   <6-7>  MnSymbolE6
   <7-8>  MnSymbolE7
   <8-9>  MnSymbolE8
   <9-10> MnSymbolE9
  <10-12> MnSymbolE10
  <12->   MnSymbolE12}{}

\DeclareMathSymbol{\upbrace}      {\mathord}{largesymbolsX}{'256}
\DeclareMathSymbol{\upbraceg}     {\mathord}{largesymbolsX}{'257}
\DeclareMathSymbol{\upbracegg}    {\mathord}{largesymbolsX}{'260}
\DeclareMathSymbol{\upbraceggg}   {\mathord}{largesymbolsX}{'261}
\DeclareMathSymbol{\upbracegggg}  {\mathord}{largesymbolsX}{'262}
\DeclareMathSymbol{\bracelu}      {\mathord}{largesymbolsX}{'264}
\DeclareMathSymbol{\braceru}      {\mathord}{largesymbolsX}{'266}
\DeclareMathSymbol{\bracemu}      {\mathord}{largesymbolsX}{'270}
\DeclareMathSymbol{\bracemid}     {\mathord}{largesymbolsX}{'271}

\def\horiz@expandable#1#2#3#4#5#6#7#8{%
  \@mathmeasure\z@#7{#8}%
  \@tempdima=\wd\z@
  \@mathmeasure\z@#7{#1}%
  \ifdim\noexpand\wd\z@>\@tempdima
    $\m@th#7#1$%
  \else
    \@mathmeasure\z@#7{#2}%
    \ifdim\noexpand\wd\z@>\@tempdima
      $\m@th#7#2$%
    \else
      \@mathmeasure\z@#7{#3}%
      \ifdim\noexpand\wd\z@>\@tempdima
        $\m@th#7#3$%
      \else
        \@mathmeasure\z@#7{#4}%
        \ifdim\noexpand\wd\z@>\@tempdima
          $\m@th#7#4$%
        \else
          \@mathmeasure\z@#7{#5}%
          \ifdim\noexpand\wd\z@>\@tempdima
            $\m@th#7#5$%
          \else
           #6#7%
          \fi
        \fi
      \fi
    \fi
  \fi}

\def\underbrace@expandable#1#2#3{\vtop{\m@th\ialign{##\crcr
  $\m@th\hfil#2#3\hfil$\crcr
  \noalign{\kern2\p@\nointerlineskip}%
  #1#2{#3}\crcr}}}

\def\underbrace@#1#2#3{\vtop{\m@th\ialign{##\crcr
  $\m@th\hfil#2#3\hfil$\crcr
  \noalign{\kern2\p@\nointerlineskip}%
  #1#2\crcr}}}

\def\bracefill@#1#2#3#4#5{$\m@th#5#1\leaders\hbox{$#4$}\hfill#2\leaders\hbox{$#4$}\hfill#3$}
\def\upbracefill@{\bracefill@\bracelu\bracemu\braceru\bracemid}

\def\upbrace@expandable{%
  \horiz@expandable
    \upbrace
    \upbraceg
    \upbracegg
    \upbraceggg
    \upbracegggg
    \upbracefill@}

\DeclareRobustCommand{\underbrace}[1]{\mathop{\mathpalette{\underbrace@expandable\upbrace@expandable}{#1}}\limits}