You're not making good use of the rcases*
machinery; I would ditch it in favor a simple \left. ... \right\}
wrapper. To format the material to left of the tall curly brace, I suggest you employ an alignedat{3}
environment. This will allow you to get rid of all \displaystyle
and \limits
directives.
Optionally, you could use \mathclap
directives to get rid of the whitespace around the \lim
terms. The solutions without and with the \mathclap
idea are shown in the following screenshot.
\documentclass[a4paper]{article}
\usepackage{%%amsmath, %% amsmath is loaded automatically by mathtools
amssymb}
%%\usepackage{amsfonts} %% amsfonts is loaded automatically by amssymb
\usepackage{mathtools} % for \mathclap macro
\begin{document}
\noindent
Basic solution:
\[
\left.
\begin{alignedat}{3}
&\lim_{x\to2^+}f(x)&&=\lim_{x\to2^+}\sqrt{5x-1}&&=3\\
&\lim_{x\to2^-}f(x)&&=\lim_{x\to2^-}(3x^2-4x-1)&&=3\\
&f(2) &&=\sqrt{5\cdot2-1} &&=3
\end{alignedat}
\,\right\}
\Leftrightarrow \lim_{x\to2^-}f(x)=f(2)=\lim_{x\to2^+}f(x)
\]
\bigskip\noindent
With \verb+\mathclap+:
\[
\left.
\begin{alignedat}{3}
&\lim_{\mathclap{x\to2^+}}f(x)&&=\lim_{\mathclap{x\to2^+}}\sqrt{5x-1}&&=3\\
&\lim_{\mathclap{x\to2^-}}f(x)&&=\lim_{\mathclap{x\to2^-}}(3x^2-4x-1)&&=3\\
&f(2) &&=\sqrt{5\cdot2-1} &&=3
\end{alignedat}
\,\right\}
\Leftrightarrow \lim_{\mathclap{x\to2^-}}f(x)=f(2)=\lim_{\mathclap{x\to2^+}}f(x)
\]
\end{document}