I not sure about what without manually calculating the coordinates of R means.
May be something like
\node[outer sep=0pt,circle, draw,inner sep=1.5pt, label={[fill=white]right:$R$}] (R)
at ($(P)!1cm*sqrt(5)!-90:(Q)$){};
In any case, TikZ
draws everything in the same order as code is written. Therefore, as you draw
\draw[green!20!white] (Q) -- (P) -- (R) -- (Q) -- cycle;
after
\coordinate (R) at ($(P)!1cm*sqrt(5)!-90:(Q)$);
\node[outer sep=0pt,circle, fill,inner sep=1.5pt, label={[fill=white]right:$R$}] at(R) {};
the triangle is drawn over R
. But why over R
and not over P
and Q
? Because R
, as you can see from previous lines, is a not drawn coordinate (you draw the dot with un unnamed node over R
) while P
and Q
has been defined as nodes
with a certain size. And
\draw[green!20!white] (Q) -- (P) -- (R) -- (Q) -- cycle;
draws lines between node borders, borders which don't only exist in case of R
because it's a point.
See what happens with following code. Nodes P
and Q
are only drawn and remember that circle around R
is an unnamed node which is not referenced later on.
\documentclass[10pt]{amsart}
\usepackage{tikz}
\usetikzlibrary{calc}
\begin{document}
\begin{tikzpicture}
\node[outer sep=0pt,circle, draw ,inner sep=1.5pt,label={[fill=white]left:$P$}] (P) at (-1,-1) {};
\node[outer sep=0pt,circle, draw,inner sep=1.5pt, label={[fill=white]right:$Q$}] (Q) at (2,1) {};
\coordinate (R) at ($(P)!1cm*sqrt(5)!-90:(Q)$);
\node[outer sep=0pt,circle, draw,inner sep=1.5pt, label={[fill=white]right:$R$}] at(R) {};
\draw[red, fill=green] (Q) -- (P) -- (R) --(Q)-- cycle;
\end{tikzpicture}
\end{document}

While
\documentclass[10pt]{amsart}
\usepackage{tikz}
\usetikzlibrary{calc}
\begin{document}
\begin{tikzpicture}
\node[outer sep=0pt,circle, draw ,inner sep=1.5pt,label={[fill=white]left:$P$}] (P) at (-1,-1) {};
\node[outer sep=0pt,circle, draw,inner sep=1.5pt, label={[fill=white]right:$Q$}] (Q) at (2,1) {};
\draw[green!20!white] (P) -- (Q);
%\coordinate (R) at ($(P)!1cm*sqrt(5)!-90:(Q)$);
\node[outer sep=0pt,circle, draw,inner sep=1.5pt, label={[fill=white]right:$R$}] (R) at ($(P)!1cm*sqrt(5)!-90:(Q)$){};
\draw[red, fill=green] (Q) -- (P) -- (R) --(Q)-- cycle;
\end{tikzpicture}
\end{document}
produces

As you can see from previous figure, in this case is not possible to close an area to be filled because R
has a real size and the path is disjoint (is it correct?).
How to solve both problems: filling the triangle and doing it below nodes?
With backgrounds
tikzlibrary and closing the path.
Following code shows a possible solution. Grid and axes are drawn on the background
layer which is declared by backgrounds
library. And the white triangle is also drawn on background layer, but over grid because everything is drawn in order.
\begin{scope}[on background layer]
...
\end{scope}
But triangle vertex and corner are drawn on foreground layer.
The triangle can be filled referencing nodes centers instead of only their names
\draw[green!20!white, fill=white] (Q.center) -- (P.center) -- (R.center) -- cycle;
The complete code:
\documentclass[tikz,10pt,border=2mm]{standalone}
\usepackage{tikz}
\usetikzlibrary{calc,backgrounds}
\begin{document}
\begin{tikzpicture}[dot/.style={circle, fill, inner sep=1.5pt, outer sep=0pt}]
\begin{scope}[on background layer]
\draw[yellow, line width=0.1pt] (-1.75,-3.25) grid[xstep=0.5, ystep=0.5] (2.75,1.75);
\draw[draw=gray!30,latex-latex] (0,1.75) +(0,0.25cm) node[above right] {$y$} -- (0,-3.25) -- +(0,-0.25cm);
\draw[draw=gray!30,latex-latex] (-1.75,0) +(-0.25cm,0) -- (2.75,0) -- +(0.25cm,0) node[below right] {$x$};
\end{scope}
\coordinate[dot, label={[fill=white]left:$P$}] (P) at (-1,-1) {};
\coordinate[dot, label={[fill=white]right:$Q$}] (Q) at (2,1) {};
\coordinate[dot, label={[fill=white]below right:$R$}] (R) at ($(P)!1cm*sqrt(5)!-90:(Q)$) {};
%
%%\draw[green!20!white] (P) -- (Q);
%
%%\coordinate (R) at ($(P)!1cm*sqrt(5)!-90:(Q)$);
%
\begin{scope}[on background layer]
\draw[green!20!white, fill=white] (Q.center) -- (P.center) -- (R.center) -- cycle;
\end{scope}
%
\coordinate (a) at ($ (P)!5mm! -45:(Q) $);
\draw[green!20!white] ($(P)!(a)!(Q)$)--(a)--($(P)!(a)!(R)$);
\end{tikzpicture}
\end{document}
