Nested right cases

I was reading a script for maths and came across this "picture" and was wondering how he typeset it in LaTeX, so I wanted to try myself.

First I thought of cases, but that did not work as intended. Then I tried arrays but while trying to align the arrays with the top line but not getting larger curly braces than wanted I could not succeed.

I would like to get the centred look, or at least no manual spacing adding.

This is what the sources looks like:

My attempt:

$$\begin{rcases} \begin{rcases} \begin{rcases} \text{ganz rational, Polynom:} \\[-1ex] y = f(x) = x \\ \text{gebrochen rational:} \\[-1ex] y = f(x) = \frac1x \end{rcases} \quad \text{rational} \quad \\ \mskip220mu\text{irrational:} \\[-1ex] \mskip220mu y = f(x) = \sqrt{x} \end{rcases} \quad\text{algebraisch} \quad \\ \mskip360mu\text{transzendent:} \\[-1ex] \mskip360mu y = f(x) = \sin x \end{rcases}\quad \text{elementar}$$


Here is what you can do with the package cascade.

\documentclass{article}
\usepackage{geometry}

\begin{document}

{}
{
{}
{
{\begin{tabular}{c}ganz rational, Polynom:\\ $y=(x)=x$\end{tabular}}
{\begin{tabular}{c}gebrochen rational:\\ $y=f(x)=\frac{1}{x}$\end{tabular}}
rational
}
{}
{\begin{tabular}{c}irrational\\ $y=f(x)=\sqrt{x}$\end{tabular}}
algebraisch
}
{}
{\begin{tabular}{c}tranzendent\\ $y=f(x)=\sin x$\end{tabular}}
elemantar

\bigskip

{
{
{\begin{tabular}{c}ganz rational, Polynom:\\ $y=(x)=x$\end{tabular}}
{\begin{tabular}{c}gebrochen rational:\\ $y=f(x)=\frac{1}{x}$\end{tabular}}
rational
}
{}
{}
{\begin{tabular}{c}irrational\\ $y=f(x)=\sqrt{x}$\end{tabular}}
algebraisch
}
{}
{}
{\begin{tabular}{c}tranzendent\\ $y=f(x)=\sin x$\end{tabular}}
elemantar

\vspace{1cm}

{}
{
{}
{
{\begin{tabular}{c}ganz rational, Polynom:\\ $y=(x)=x$\end{tabular}}
{\begin{tabular}{c}gebrochen rational:\\ $y=f(x)=\frac{1}{x}$\end{tabular}}
\makebox[\widthof{$y=f(x)=\sqrt{x}$}]{rational}
}
{}
{\begin{tabular}{c}irrational\\ $y=f(x)=\sqrt{x}$\end{tabular}}
\makebox[\widthof{$y=f(x)=\sin(x)$}]{algebraisch}
}
{}
{\begin{tabular}{c}tranzendent\\ $y=f(x)=\sin x$\end{tabular}}
elemantar

\bigskip

{
{
{\begin{tabular}{c}ganz rational, Polynom:\\ $y=(x)=x$\end{tabular}}
{\begin{tabular}{c}gebrochen rational:\\ $y=f(x)=\frac{1}{x}$\end{tabular}}
\makebox[\widthof{$y=f(x)=\sqrt{x}$}]{rational}
}
{}
{}
{\begin{tabular}{c}irrational\\ $y=f(x)=\sqrt{x}$\end{tabular}}
\makebox[\widthof{$y=f(x)=\sin(x)$}]{algebraisch}
}
{}
{}
{\begin{tabular}{c}tranzendent\\ $y=f(x)=\sin x$\end{tabular}}
elemantar

\end{document}


A first cut at this would be, using the array package:

\newcolumntype{M}{>{$}c<{$}}
\newcolumntype{C}[1]{>{\centering\arraybackslash}p{#1}}

$$\begin{tabular}{M c} % ❶ \left. \begin{array}{c C{1.0in}} % ❷ \left. \begin{array}{c C{1.0in}} % ❸ \left. \begin{tabular}{c} % ❹ ganz rational. Polynom:\\ y=f(x)=x\\[.4ex] gebrochen rational:\\ y=f(x)=\frac{1}{x} \end{tabular} \right\} & rational\\ & irrational:\break y=f(x)=\sqrt{x} \end{array} \right\} & alebraisch\\ & transcendent:\break y=f(x)=\sin x \end{array} \right\} & elementar \end{tabular}$$


The secret sauce here is mixing tabular and array based on whether we want to have a text c-column or not (you can use tabular anywhere you use array. The converse is not true) and defining a new column type C which gives us a centered p-style column.

But we miss out on how the text tucks under the brace and the \arraycolsep is a bit much. I was originally thinking that I would do some fancy footwork with new column types to add negative space here and take care of the latter by changing \arraycolsep and then I realized 🤦🏻‍♂️ that I could do both with the same command by setting \arraycolsep to a negative value since there's a large right brace at every column separation.

Putting

\setlength{\arraycolsep}{-5pt}


after ¹ almost solves it. The problem is that we get \arraycolsep at the sides of the tabular or array environment as well as between columns.² But remember that @ in a column specification tells LaTeX not to insert the \arraycolsep so we can change some column specifications to be (use the circled numbers for reference of where to change):

    \begin{array}{c C{1.0in}@{}} % ❷
\begin{array}{c C{1.0in}@{}} % ❸
\begin{tabular}{c@{}} % ❹


There's doubtless fine-tuning that can be done, but this should be enough to enable further exploration.

1. I put it here so that it won't affect any other tabular or array environments in the document.
2. The amsmath matrix environment does manual spacing around its inner array to get rid of this. We take a different approach here.
• Thanks a lot @Don. I am always amazed a how much one can do with the tabular and array options in LaTeX. I appreciate you taking the time to answer my question. Commented Sep 11, 2021 at 18:31

Using the rcases machinery of the mathtools package may amount to code overkill. A couple of nested tabular environments, along with suitably-placed \left. and \right\} statements, would appear to get the job oone.

\documentclass{article}
$$\setlength\tabcolsep{3pt} % default: 6pt \left. \begin{tabular}{@{}cc} \left. \begin{tabular}{@{}cc} \left. \begin{tabular}{@{}c} ganz rational, Polynom:\\ f(x)=x \\[12pt] gebrochen rational:\\ f(x)=1/x \end{tabular} \right\} & rational \\ \addlinespace & irrational: \\ & f(x)=\sqrt{x} \end{tabular} \right\} & algebraisch \\ \addlinespace & transzendent:\\ & f(x)=\sin x \end{tabular} \right\} \mbox{ elementar}$$

• Thanks a lot @Mico. That is exactly what I was trying to do, but I had problems with the \left. and \right\} and the arrays / tabulars. Commented Sep 11, 2021 at 18:07