Change the shape of a fraction

Is it possible to change the bar in a fraction to a different symbol? ie.

\frac{x}{y}


But instead of a straight line, I'd like to choose the delimiter to say \overbrace. I know I can get rid of the delimiter completely using \genfrac

\genfrac{}{}{0pt}{}{x}{y}


And the closest I've come is:

\genfrac{}{}{0pt}{}{x}{\overbrace{y}}


While this extends if y is a large expression, it does not extend the delimiter if x is a large expression. Any thoughts?

EDIT: Changed to \overbrace from \hat as the hat does not compile. Question remains the same.

I'm not sure what you're doing to get \hat to extend—it doesn't for me—but the following sort of works. I'm not totally happy with the spacing. Note that I used \overbrace instead of \hat since it actually does extend.

\newcommand*\hatfrac[2]{
\genfrac{}{}{0pt}{}{#1}{\mathpalette\makehathelper{{#1}{#2}}}
}
\def\makehathelper#1#2{\makehat#1#2}
\def\makehat#1#2#3{
\begingroup
\setbox0\hbox{$\mathsurround0pt #1#2$}
\setbox2\hbox{$\mathsurround0pt #1#3$}
\ifdim\wd0<\wd2
\overbrace{\box2}
\else
\overbrace{\hbox to\wd0{\hfil\box2\hfil}}
\fi
\endgroup
}


The braces don't seem to line up in the two cases. If you make the vinculum (the fraction bar) larger than 0pt, you get a line, of course, but then the braces line up. Try

$\hatfrac{abcdefghijkl}{x} \hatfrac{x}{abcdefghijkl} \hatfrac{x}{y}$


to see what I mean.

Update
I'm leaving my old solution because I suspect that with some modifications, someone can make it work. Here's my current solution.

\newcommand*\hatfrac[2]{
\mathchoice{\makehat\textstyle{#1}{#2}}
{\makehat\scriptstyle{#1}{#2}}
{\makehat\scriptscriptstyle{#1}{#2}}
{\makehat\scriptscriptstyle{#1}{#2}}
}
\def\makehat#1#2#3{
\mathinner{
\hskip\nulldelimiterspace
\setbox0\hbox{\strut$\mathsurround0pt#1#2$}
\setbox2\hbox{\strut\cramped[#1]{#3}}
\dimen0 \wd\ifdim\wd0<\wd2 2\else0\fi
\setbox4\hbox{$\mathsurround0pt\overbrace{\hbox to\dimen0{\hfil}}$}
\dimen0 \wd4
\dimen2 \ht2
\setbox6\vbox{\hbox to\dimen0{\hfil\unhbox0\hfil}
\nointerlineskip
\box4
\nointerlineskip
\hbox to\dimen0{\hfil\unhbox2\hfil}}
\lower.5\dimen2\box6
\hskip\nulldelimiterspace
}
}


It has one problem, which I hope is minor enough that you can use it. The first is that the numerator and denominator are typeset with a \strut for spacing. What should happen instead is the numerator (resp. denominator) should be moved up (resp. down) some amount dependent on the font and its height+depth. In principle, this is doable, but I'd need to spend a lot more time reading Appendix G of the TeXbook.

EDIT: There was an additional problem regarding cramped math styles that has been resolved thanks to this answer. You need the mathtools package for the \cramped macro. Actually, a similar issue exists with the numerator. The denominator should always be cramped, so using \cramped we can solve it. The numerator should be cramped only when the entire fraction is cramped. I don't know how to test if the current style is cramped or not.

I'm in a TikZ-mood.

\documentclass{article}

\usepackage{tikz}
\usetikzlibrary{decorations.pathreplacing}

\newcommand{\intfunnyfrac}[1]{%
\tikz[baseline]
\draw
node[anchor=mid] (frac) {$$#1$$}
[draw,decorate,decoration=brace]
(frac.west) -- (frac.east)
;%
}

\newcommand{\funnyfrac}[2]{%
\mathchoice{%
\intfunnyfrac{\displaystyle{ #1 \atop #2}}%
}{%
\intfunnyfrac{\textstyle{ #1 \atop #2}}%
}{%
\intfunnyfrac{\scriptstyle {#1 \atop #2}}%
}{%
\intfunnyfrac{\scriptscriptstyle {#1 \atop #2}}%
}%
}
\begin{document}
$X \frac{A}{B} \funnyfrac{A}{B} \frac{A}{B} Y$

$X \funnyfrac{A B C D}{B} \funnyfrac{A}{A B C D} \funnyfrac{A}{B} Y$

\end{document}


It could do with a bit of adjustment on spacing, and with bumpy replacements for the horizontal line then the numerator and denominator could do with a bit more separation, but it's intended more as a "proof of concept" than some code you could cut-and-paste.

This is a slightly modified version of TH's solution that does not use \genfrac. Instead, I use the TeX primitive \atop.

\def\genbraced#1#2#3#4%accent style num den
{\begingroup
\setbox0\hbox{$\mathsurround0pt #2#3$}%
\setbox2\hbox{$\mathsurround0pt #2#4$}%
\dimen0\wd0
\ifdim\dimen0<\wd2 \dimen0\wd2 \fi
{\box0 \atop #1{\hbox to \dimen0{\hss\box2\hss}}}
\endgroup}

\def\braced{\genbraced\overbrace\displaystyle}

%\showboxes %Really useful for visual debugging

\starttext

\startformula
\braced{abcdefghijkl}{x}
\braced{x}{abcdefghijkl}
\braced{x}{y}
\stopformula

\stoptext

• This seems to have the same spacing problems my original solution has. – TH. Sep 1 '10 at 6:10

Another possibility is to build the "fractions" from scratch, from boxes. Something like this seems to be working for me:

\newcommand*\hatfrac[2]{
\mathchoice{\makehat\displaystyle{#1}{#2}}
{\makehat\textstyle{#1}{#2}}
{\makehat\scriptstyle{#1}{#2}}
{\makehat\scriptscriptstyle{#1}{#2}}
}

\def\makehat#1#2#3{
\begingroup
\setbox0\hbox{$\mathsurround0pt #1#2$}
\setbox2\hbox{$\mathsurround0pt #1#3$}
\dimen0 = \ht0
\dimen2 = \ht2
\ifdim\dimen0<\dimen2
\setbox4\vbox to\dimen2{\vfil\copy0\vskip0pt}
\setbox6\vbox to\dimen2{\copy2\vskip0pt}
\else
\setbox4\vbox to\dimen0{\copy0\vskip0pt}
\setbox6\vbox to\dimen0{\copy2\vfil\vskip0pt}
\fi
\ifdim\wd0<\wd2
\dimen4=\wd2
\else
\dimen4=\wd0
\fi
\setbox8\hbox{\strut$\overbrace{\hbox to\dimen4{\hfil}}$}
\dimen4=\wd8
\;\vcenter{\hbox{\vbox{\hbox to\dimen4{\hfil\box4\hfil}%
\box8\hbox to\dimen4{\hfil\box6\hfil}}}}\;
\endgroup
}

• Looks like we came up with pretty similar solutions! – TH. Sep 1 '10 at 6:14