You can "hack" the macros from tikz-3dplot-circleofsphere. Here I "hack" \tdplotCsDrawLatCircle by storing the foreground arc in a macro called \pathFG:
\tdplotCsDrawLatCircle[tdplotCsFront/.style={draw=none,save path=\pathFG},
tdplotCsBack/.style={draw=none}]{R}{Angle}
This gives us access to this stretch for clipping utilizing reuse path=\pathFG. (Note that use path does not do what we want here.) Then one can clip and protect the relevant areas.
\documentclass[border=2mm,12pt,tikz]{standalone}
\usepackage{tikz-3dplot-circleofsphere}
\usepackage{fouriernc}
\makeatletter
\tikzset{
reuse path/.code={\pgfsyssoftpath@setcurrentpath{#1}}
}
\tikzset{even odd clip/.code={\pgfseteorule},
protect/.code={
\clip[overlay,even odd clip,reuse path=#1]
(-6383.99999pt,-6383.99999pt) rectangle (6383.99999pt,6383.99999pt);
}}
\makeatother
\begin{document}
\tdplotsetmaincoords{70}{80}
\begin{tikzpicture}[tdplot_main_coords,scale=1,line join = round,
line cap = round,
declare function={R=5;r=4;h=sqrt(R^2 - r^2);%
myx= 2; myy=sqrt(R*R-h*h- myx*myx); k=-1; Angle=k*acos(r/R);}]
\path
(0,0,0) coordinate (O)
(0,0,k*h) coordinate (H)
(myx,myy,k*h) coordinate (M)
;
\tdplotCsDrawLatCircle[tdplotCsFront/.style={draw=none,save path=\pathFG},
tdplotCsBack/.style={draw=none}]{R}{Angle}
\begin{scope}
\path[save path=\sphere,thick,tdplot_screen_coords] (O) circle[radius=R];
\end{scope}
\begin{scope} [canvas is xy plane at z=k*h]
\path[save path=\rectA] (-R,-R) rectangle (R,R);
\begin{scope}
\clip[reuse path=\pathFG,save path=\pathFGB] -- (R,R) -- (R,-R) -- cycle;
\draw[dashed,use path=\sphere];
\end{scope}
\begin{scope}
\clip[use path=\sphere];
\draw[dashed,use path=\rectA];
\end{scope}
\begin{scope}
\tikzset{protect=\pathFGB}
\draw[thick,use path=\sphere];
\end{scope}
\draw[thick] (R,R) -- (R,-R);
\tikzset{protect=\sphere}
\draw[thick,use path=\rectA];
\end{scope}
\tdplotCsDrawLatCircle[blue, thick]{R}{Angle}
\foreach \p in {H,M,O}
{\draw[fill=black] (\p) circle (1.5pt);}
\foreach \p/\g in {M/90,O/-135,H/30}
{\path (\p)+(\g:3mm) node{$\p$}; }
\draw[dashed] (O) -- (H) -- (M) --cycle;
\end{tikzpicture}
\end{document}
Note that this trick is not limited to tikz-3dplot-circleofsphere. Whenever a package draws a path with a macro we can access and use it this way. For instance, one can hack the tikzlings package to provide our friends with cloths.
\documentclass[tikz,border=3.14mm]{standalone}
\usepackage{tikzlings}
\newcounter{savedpath}
\makeatletter
\tikzset{reuse path/.code={\pgfsyssoftpath@setcurrentpath{#1}},
save paths/.code={\setcounter{savedpath}{0}%
\edef\tikz@path@name{#1}%
\tikzset{every path/.append style={autosave path}}},
autosave path/.code={\stepcounter{savedpath}%
\edef\temp{\noexpand\tikzset{save path=\csname\tikz@path@name\roman{savedpath}\endcsname}}%
\temp
}
}
\makeatother
\begin{document}
\begin{tikzpicture}
\begin{scope}[save paths=mpath]
\marmot
\end{scope}
\begin{scope}
\clip[reuse path=\mpathvii];
\fill[blue] (-1,1.4) to[bend right=10] (1,1.4) -- (1,0.5) to[bend left=10] (-1,0.5) --
cycle;
\fill[brown!30!black,reuse path=\mpathix];
\fill[brown!30!black,reuse path=\mpathx];
\end{scope}
\end{tikzpicture}
\end{document}
As can be seen, these reused paths can be used for clipping and filling. They can also be used for drawing. For some reason they cannot be used for shading, though, even though you can use them to clip some shading which yields the equivalent result. One can also combine them. It is conceivable that these are some steps towards solving this issue.
ADDENDUM: Some animation. No, this does not work for arbitrary view angles but for some nontrivial subset of them.
\documentclass[border=2mm,12pt,tikz]{standalone}
\usepackage{tikz-3dplot-circleofsphere}
\usepackage{fouriernc}
\makeatletter
\tikzset{
reuse path/.code={\pgfsyssoftpath@setcurrentpath{#1}}
}
\tikzset{even odd clip/.code={\pgfseteorule},
protect/.code={
\clip[overlay,even odd clip,reuse path=#1]
(-6383.99999pt,-6383.99999pt) rectangle (6383.99999pt,6383.99999pt);
}}
\makeatother
\begin{document}
\foreach \Angle in {5,15,...,355}
{\tdplotsetmaincoords{70}{\Angle}
\begin{tikzpicture}[tdplot_main_coords,scale=1,line join = round,
line cap = round,
declare function={R=5;L=5.5;r=4;h=sqrt(R^2 - r^2);%
myx= 2; myy=sqrt(R*R-h*h- myx*myx); k=-1; Angle=k*acos(r/R);}]
\path[tdplot_screen_coords,use as bounding box] (-9,-9) rectangle (9,9);
\path
(0,0,0) coordinate (O)
(0,0,k*h) coordinate (H)
(myx,myy,k*h) coordinate (M)
;
\pgfmathtruncatemacro{\itest}{(abs(sin(\tdplotmainphi)*cos(\tdplotmainphi))<0.3 ? 0 : 1)}
\tdplotCsDrawLatCircle[tdplotCsFront/.style={draw=none,save path=\pathFG},
tdplotCsBack/.style={draw=none}]{R}{Angle}
\begin{scope}
\path[save path=\sphere,thick,tdplot_screen_coords] (O) circle[radius=R];
\end{scope}
\begin{scope} [canvas is xy plane at z=k*h]
\path[save path=\rectA] (-L,-L) rectangle (L,L);
\begin{scope}
\path ({(cos(\tdplotmainphi)<0 ? -1 : 1)*L},
{(sin(\tdplotmainphi)<0 ? -1 : 1)*L}) coordinate (p1)
({(cos(\tdplotmainphi-90+0)<0 ? -1 : 1)*L},
{(sin(\tdplotmainphi-90+0)<0 ? -1 : 1)*L}) coordinate (p2)
({(cos(\tdplotmainphi-180+0)<0 ? -1 : 1)*L},
{(sin(\tdplotmainphi-180+0)<0 ? -1 : 1)*L})
coordinate (p3);
\clip[overlay,reuse path=\pathFG,save path=\pathFGB]
--(p1) -- (p2) -- (p3) -- cycle;
\draw[dashed,use path=\sphere];
\end{scope}
\begin{scope}
\clip[use path=\sphere];
\draw[dashed,use path=\rectA];
\end{scope}
\begin{scope}
\tikzset{protect=\pathFGB}
\draw[thick,use path=\sphere];
\end{scope}
\draw[thick] (p1) -- (p2) -- (p3);
\tikzset{protect=\sphere}
\draw[thick,use path=\rectA];
\end{scope}
\tdplotCsDrawLatCircle[blue, thick]{R}{Angle}
\foreach \p in {H,M,O}
{\draw[fill=black] (\p) circle (1.5pt);}
\foreach \p/\g in {M/90,O/-135,H/30}
{\path (\p)+(\g:3mm) node{$\p$}; }
\draw[dashed] (O) -- (H) -- (M) --cycle;
\end{tikzpicture}}
\end{document}
And here is another animation.
\documentclass[border=2mm,12pt,tikz]{standalone}
\usepackage{tikz-3dplot-circleofsphere}
\usepackage{fouriernc}
\makeatletter
\tikzset{
reuse path/.code={\pgfsyssoftpath@setcurrentpath{#1}}
}
\tikzset{even odd clip/.code={\pgfseteorule},
protect/.code={
\clip[overlay,even odd clip,reuse path=#1]
(-6383.99999pt,-6383.99999pt) rectangle (6383.99999pt,6383.99999pt);
}}
\makeatother
\begin{document}
\foreach \Z in {4,3,...,-4,-3,-2,...,3}
{\tdplotsetmaincoords{70}{80}
\begin{tikzpicture}[tdplot_main_coords,scale=1,line join = round,
line cap = round,
declare function={R=5;L=5.5;h=abs(\Z);r=sqrt(R*R-\Z*\Z);%
myx= 2; myy=sqrt(R*R-h*h- myx*myx); k=sign(\Z); Angle=k*acos(r/R);}]
\path[tdplot_screen_coords,use as bounding box] (-9,-9) rectangle (9,9);
\path
(0,0,0) coordinate (O)
(0,0,k*h) coordinate (H)
(myx,myy,k*h) coordinate (M)
;
\pgfmathtruncatemacro{\itest}{(abs(sin(\tdplotmainphi)*cos(\tdplotmainphi))<0.3 ? 0 : 1)}
\tdplotCsDrawLatCircle[tdplotCsFront/.style={draw=none,save path=\pathFG},
tdplotCsBack/.style={draw=none}]{R}{Angle}
\begin{scope}
\path[save path=\sphere,thick,tdplot_screen_coords] (O) circle[radius=R];
\end{scope}
\begin{scope} [canvas is xy plane at z=k*h]
\path[save path=\rectA] (-L,-L) rectangle (L,L);
\begin{scope}
\path ({(cos(\tdplotmainphi)<0 ? -1 : 1)*L},
{(sin(\tdplotmainphi)<0 ? -1 : 1)*L}) coordinate (p1)
({(cos(\tdplotmainphi-90+0)<0 ? -1 : 1)*L},
{(sin(\tdplotmainphi-90+0)<0 ? -1 : 1)*L}) coordinate (p2)
({(cos(\tdplotmainphi-180+0)<0 ? -1 : 1)*L},
{(sin(\tdplotmainphi-180+0)<0 ? -1 : 1)*L})
coordinate (p3);
\clip[overlay,reuse path=\pathFG,save path=\pathFGB]
--(p1) -- (p2) -- (p3) -- cycle;
\draw[dashed,use path=\sphere];
\end{scope}
\begin{scope}
\clip[use path=\sphere];
\draw[dashed,use path=\rectA];
\end{scope}
\begin{scope}
\tikzset{protect=\pathFGB}
\draw[thick,use path=\sphere];
\end{scope}
\draw[thick] (p1) -- (p2) -- (p3);
\tikzset{protect=\sphere}
\draw[thick,use path=\rectA];
\end{scope}
\tdplotCsDrawLatCircle[blue, thick]{R}{Angle}
\foreach \p in {H,M,O}
{\draw[fill=black] (\p) circle (1.5pt);}
\foreach \p/\g in {M/90,O/-135,H/30}
{\path (\p)+(\g:3mm) node{$\p$}; }
\draw[dashed] (O) -- (H) -- (M) --cycle;
\end{tikzpicture}}
\end{document}
