This question is very close to what I want, yet not quite.

The accepted answer offers linkcolor = black, but I do not want to hide all links.

I like my in-line figure numbers and section references in red.

My document goes roughly like so:

...
\usepackage[]{hyperref}
...
\begin{document}
\tableofcontents
\chapter*{...}
\chapter{...}
...
\end{document}


You can change the settings locally using

{
\tableofcontents
}


or

{
\tableofcontents
}


Code:

\documentclass{report}
\usepackage{blindtext}% dummy text
\usepackage{amsmath}
\usepackage{hyperref}
\hypersetup{
}
\begin{document}
{
\tableofcontents
}
\chapter{First hapter}\label{ch1}
See chapter \ref{ch2}
\chapter{Second chapter}\label{ch2}
See chapter \ref{ch1}
\end{document}


If you want to remove the links from TOC complettly, you can use option linktoc=none:

\documentclass{report}
\usepackage{blindtext}% dummy text
\usepackage{amsmath}
\usepackage{hyperref}
\hypersetup{