No line here to end inside custom environment

Question

I wrote an "advanced" environment, but I am having problems if I put another environment inside:

\documentclass{article}

\usepackage{amsmath,amsfonts,amssymb,amsthm} % For math equations, theorems, symbols, etc
\usepackage{thmtools}

\newif\ifadvanced % advanced environment
\advancedtrue % comment out to hide advanced environments


\newcommand{\separator}{\noindent\makebox[\linewidth]{\rule{\linewidth}{0.4pt}}}
\newenvironment{advanced}{\ifadvanced\par\separator \small \fi}{\ifadvanced\\\separator\par\fi}

\declaretheoremstyle[%
  spaceabove=-6pt,%
  spacebelow=6pt,%
  headfont=\normalfont\itshape,%
  postheadspace=1em,%
  qed=\qedsymbol%
]{myproofstyle} 
\declaretheorem[name={Proof},style=myproofstyle,unnumbered,
]{advancedproof}

\begin{document}

\begin{advanced}
Inside advanced environment.
\begin{advancedproof}
inside proof
\end{advancedproof}
\end{advanced}
\end{document}

I could fix this problem by adding text between \end{advanced} and \end{advancedproof} but there must be a better way to solve this problem. I can't seem to find the answer, as I tried to add newlines inside the environments without luck.

UPDATE

The proposed code messes up the behavior where there is no inner environment:

\documentclass{article}

\usepackage{amsmath,amsfonts,amssymb,amsthm} % For math equations, theorems, symbols, etc
\usepackage{thmtools}

\newif\ifadvanced % advanced environment
\advancedtrue % comment out to hide advanced environments


\newif\ifadvanced % advanced environment
\advancedtrue % comment out to hide advanced environments


\newcommand{\separator}{\noindent\makebox[\linewidth]{\rule{\linewidth}{0.4pt}}}
\newenvironment{advanced}{\ifadvanced\par\separator \small \fi}{\ifadvanced\vspace*{-1em}\separator\par\fi}

\declaretheoremstyle[%
  spaceabove=-6pt,%
  spacebelow=6pt,%
  headfont=\normalfont\itshape,%
  postheadspace=1em,%
  qed=\qedsymbol%
]{myproofstyle} 
\declaretheorem[name={Proof},style=myproofstyle,unnumbered,
]{advancedproof}

\begin{document}

\begin{advanced}
Inside advanced environment.
\begin{advancedproof}
inside proof
\end{advancedproof}
\end{advanced}


\begin{advanced}
Inside advanced environment. But without any other internal environment
\end{advanced}

\end{document}

Answer 1

You could replace your code with the following. This just translates \\ to \vspace*{0pt}.

\documentclass{article}

\usepackage{amsmath,amsfonts,amssymb,amsthm} % For math equations, theorems, symbols, etc
\usepackage{thmtools}

\newif\ifadvanced % advanced environment
\advancedtrue % comment out to hide advanced environments


\newcommand{\separator}{\noindent\makebox[\linewidth]{\rule{\linewidth}{0.4pt}}}
\newenvironment{advanced}{\ifadvanced\par\separator \small \fi}{\ifadvanced\vspace*{0pt}\separator\par\fi}

\declaretheoremstyle[%
  spaceabove=-6pt,%
  spacebelow=6pt,%
  headfont=\normalfont\itshape,%
  postheadspace=1em,%
  qed=\qedsymbol%
]{myproofstyle} 
\declaretheorem[name={Proof},style=myproofstyle,unnumbered,
]{advancedproof}

\begin{document}

\begin{advanced}
Inside advanced environment.
\begin{advancedproof}
inside proof
\end{advancedproof}
\end{advanced}
\end{document}

Update: Your problem should be solved using

\newenvironment{advanced}{\ifadvanced\par\separator \small \fi}{\ifadvanced\vspace*{-.75em}\par\separator\par\fi}