Closing parenthesis as delimiter not matched when breaklines=true

Question

I want code in parentheses and in brackets to appear in different colors. In the following document, the words brackets again appear in green as intended, and outside again is black as intended, but stretches into the margin of the page.

If I uncomment the line ,breaklines=true, the line is broken between brackets and again, however, these words and the following are red, i.e. as if a (-)-delimited section was detected but then the closing ) was ignored.

What is going on here?

Addition: The problem occurs also if there is no line break present, e.g. if I shorten all words in the text to their first letter. The problem goes away in any case if I pick any other kind of delimiter instead of (), e.g. {}.

\documentclass{article}
\usepackage{color}
\usepackage{listings}

\lstdefinelanguage{new}{
    moredelim=*[s][\color{red}]{(}{)},
    moredelim=*[s][\color{green}]{[}{]},
  }
\lstset{
  language=new
  %,breaklines=true
  }

\begin{document}

\begin{figure}
\begin{lstlisting}
outside(in paren) [in brackets (paren+brackets) brackets again] outside again
\end{lstlisting}
\end{figure}

\end{document}

Answer 1

You might have come across a bug arising from the special treatment of the right parenthesis by the line-breaking algorithm of the listings package. In fact, when the SelectCharTable hook used by the listings package is executed and line breaking is enabled ) is defined as a macro, namely by

\lst@Def{`)}{\lst@breakProcessOther)}

where

\lst@Def=macro:
#1->\lccode `\~=#1\lowercase {\def ~}.

Unfortunately, this means that ) is not recognized as a delimiter any more, which eventually leads to your observation.

I’ve put some effort in finding a fix for this bug, but haven’t succeeded so far. (My TeX programming skills are still limited.) So, the only thing I can provide right now is a workaround in the form of a patch that simply cancels the special treatment of ) by the line-breaking algorithm:

\documentclass{article}

\usepackage{color}

\usepackage{listings}

\lstdefinelanguage{new}{
  moredelim=*[s][\color{red}]{(}{)},
  moredelim=*[s][\color{green}]{[}{]},
}

\lstset{
  language=new,
  breaklines=true
}

\usepackage{etoolbox}

\makeatletter

\patchcmd{\lsthk@SelectCharTable}{%
  \lst@ifbreaklines\lst@Def{`)}{\lst@breakProcessOther)}\fi
}{%
}{
}{
}

\makeatother

\begin{document}

\begin{lstlisting}
outside (in paren) [in brackets (paren+brackets) brackets again] outside again
\end{lstlisting}

\end{document}

The resulting output is:

Answer 2

A simple workaround would be, instead of using closing parenthesis ')', one can use another character e.g ']' and replace it with closing parenthesis ')'.

\begin{lstlisting[literate={(}{{\textbf{(}}}1                          
                           {]}{{\textbf{)}}}1]
(In text I'm closing using]
\end{lstlisting}