\documentclass{article}
\usepackage{unicode-math}
\usepackage{listings}
\usepackage{fancyvrb}
\begin{document}
\lstset{fancyvrb}
\begin{Verbatim}[commandchars=\\\{\}]
\pm
\end{Verbatim}
\end{document}
This is the minimal code. And I get the error "Improper alphabetic constant".
PS: I need \lstset{fancyvrb}
.