fontsize does affect only part of the text

I use scrbookand want to set the font for some part of the text with fontsize. The effect is that the text in the normal font is in the normal font size and only special fonts are scaled by fontsize.

\documentclass{scrbook}
\KOMAoptions{ fontsize=11pt }

\usepackage{anyfontsize}

\begin{document}
some normal text \\
{
\fontsize{4pt}{9.5pt}
%       \footnotesize
Teile dieses Buches wurde  Programmen in Haskell erstellt. Der Text wurde
mit  {\LaTeX}, {Lua\LaTeX} and {\KOMAScript} gesetzt.
}
\end{document}


The result shows only the KOMAScript part is smaller:

The fontsizes are selected for contrast and not what is ultimately used!

• \fontsize{4pt}{9.5pt}\selectfont – moewe Jun 10 '19 at 10:13

\fontsize{4pt}{9.5pt} only sets up the internal parameters for the next font selection, it does not actually select a font, you need \selectfont to do that, also if you want the 9.5pt baseline to be used you need a blank line or \par before the closing } as the baseline at the end of the paragraph is used for the whole paragraph.