The following code is based on this posting to comp.text.tex in May 2011. No rule will be drawn if the section header is at the top of a page, including the very first page. This method works with both numbered and unnumbered section headers.
The amount of space between the end of a section and the subsequent rule and between the rule and the next section header, is controlled by the commands \vspace{2\bigskipamount} and \vspace{\smallskipamount}; change the arguments of the two \vspace commands to suit your preferences. The width of the rule by default is 0.4pt. (0.4pt seems to be something of a standard width for many lines in LaTeX...) You can change this setting too, of course.
Note that the command \separatorrule can be used on its own anywhere in the text, not just right before a section heading. Its two main distinguishing properties are (i) no rule will be drawn at the top of a page and (ii) no rule will be drawn later on in a page if the command is followed by an implicit or explicit \clearpage command, or similar-effect command such as \cleardoublepage.
\documentclass{article}
\newcommand{\separatorrule}{%
\par\vspace{2\bigskipamount}
\leaders\vrule width \textwidth\vskip 0.4pt
\vspace{\smallskipamount}\nointerlineskip}
\let\origsection\section
\renewcommand\section{\separatorrule \origsection}
\usepackage{lipsum} % for filler text
\begin{document}
\section*{A} \lipsum[1]
\section{B} \lipsum[2]
\section*{C} \lipsum[3]
\clearpage
\section*{D} \lipsum[4]
\section{E} \lipsum[5]
\end{document}
Finally, I have a strong hunch that the provider of the code for the separator rule in the comp.text.tex group, "Enrico Gregorio", is the very same person as egreg in the TeX Stack Exchange group. At any rate, the originality of my contribution to this posting is clearly minimal.
\newpageto flush the page? – Werner Nov 30 '11 at 20:34