Komascript fix vertical position of chapter heading with \flushbottom

Question

I adjusted the format of chapter headings like in the original question (scrbook class). I'm using the \flushbottom option. When for instance the list of figures spans more than one page, the vertical position of the chapter heading is changed a little bit. I tried to fix the rubber length of \parskip based on this question. This did not work as intended (see image). Is there a way to fix the vertical position without using \raggedbottom? Thanks!

MWE:

\documentclass[
    a4paper,
    11pt,
    listof=totoc,
    bibliography=totoc,
    parskip=half,
    openany,
]{scrbook}

\usepackage{etoolbox}
\usepackage[no-math]{fontspec}
\PassOptionsToPackage{no-math}{fontspec}
\usepackage[showframe,twoside]{geometry}
\usepackage{blindtext}
\usepackage[pdfpagelayout=TwoColumnLeft]{hyperref}

\KOMAoptions{chapterprefix=true}
\renewcommand{\raggedchapter}{\raggedleft}
\renewcommand{\chapapp}{}
\RedeclareSectionCommand[
    prefixfont = \bfseries\fontsize{50}{52}\selectfont,
    font = \Huge,
    beforeskip = 40pt,
    innerskip = -5pt,
    afterskip = 40pt,
]{chapter}

\makeatletter
\renewcommand*\chapterlineswithprefixformat[3]{%
    \Ifstr{#1}{chapter}
    {\IfArgIsEmpty{#2}
        {{\usekomafont{#1prefix}{%
                    \mbox{}%
                    \setlength{\@tempskipa}{\csname scr@#1@innerskip\endcsname}\chapterheadmidvskip
            }}#3%
        }% Unnumbered chapter headings
        {#2#3}% Numbered chapter headings
    }
    {#2#3}% Other headings using style=chapter
}
\makeatother

% Fix position of chapter heading (not working)
\makeatletter
\patchcmd{\@makechapterhead}
{\parindent}
{\parskip=\z@skip\parindent}
{}{}
\makeatother

\begin{document}
\tableofcontents
\listoffigures
\blindtext[4]
\begin{figure}[htbp]
    \caption{Test figure}
\end{figure}
\end{document}