In this instance you're most likely interested in adjusting the number widths of all sectional units in the ToC, since they are usually set hierarchically. As such, a simple redefinition of the \numberline macro suffices:
\usepackage{tocloft}% http://ctan.org/pkg/tocloft
\makeatletter
\renewcommand{\numberline}[1]{%
\@cftbsnum #1\@cftasnum~\@cftasnumb%
}
\makeatother
The above removes the former \hb@xt@\@tempdima{...} construction, which sets the number inside a fixed-width box, left-aligned. In fact, the original definition made by tocloft is:
\renewcommand{\numberline}[1]{%
\hb@xt@\@tempdima{\@cftbsnum #1\@cftasnum\hfil}\@cftasnumb}
Here's a minimal working example showing the result:

\documentclass{report}
\usepackage{tocloft}% http://ctan.org/pkg/tocloft
\makeatletter
\renewcommand{\numberline}[1]{%
\@cftbsnum #1\@cftasnum~\@cftasnumb%
}
\makeatother
\begin{document}
\tableofcontents
\chapter{A chapter}\section{A section}
\setcounter{chapter}{11}
\chapter{A chapter}\section{A section}
\setcounter{chapter}{122}
\chapter{A chapter}\section{A section}
\setcounter{chapter}{1233}
\chapter{A chapter}\section{A section}
\setcounter{chapter}{12344}
\chapter{A chapter}\section{A section}
\end{document}
Since the original definition left a "dynamic" gap between the number and the title, I've forced it to be ~ in my example. You can modify that to whatever length you're interested in (\quad, \hspace*{2em}, ...).
Note that this modification has to be inserted before calling \tableofcontents.