As already mentioned, the hyphen is interpreted as a minus in math mode.
To correct the issue you could define a macro \mhyphen
that stands for the hyphen in math mode:
\mathchardef\mhyphen="2D
\newcommand{\nuSwap}{\nu\mhyphen\textsf{swap}}
this can of course be combined with \ensuremath
to get a macro that works in both modes.
I also recommend using \texorpdfstring
(from hyperref
) if you intend to use it in headers and produce a pdf:
\documentclass{article}
\usepackage{hyperref}
\mathchardef\mhyphen="2D
\newcommand{\nuSwap}{\texorpdfstring{\ensuremath{\nu\mhyphen\textsf{swap}}}{nu-swap}}
\begin{document}
\section{Definition of \nuSwap}
Here we define \nuSwap:
\[
\nuSwap = \dots
\]
\end{document}
if you inspect the generated PDF you'll find "nu-swap" is used in the index.