Can I use “moredelim” in package “listings” to fetch a regular expression?

Question

I would like to know if it is possible to use the listings package to highlight regular expressions in order to make them look like a normal string object. Note: The syntax of a regexp is not context free so I cannot expect a solution that recognizes 100% of the cases. Anyhow, in e.g. Ruby I would like to write

(/[a-z]+/)

and tell listings to recognize the regexp /[a-z]+/ by using the left parenthesis (as an anchor point (to make it apart from arithmetic division):

\lstset{moredelim=[s][\color{red}]{(/}{/}}

But, this makes the left parenthesis colored - not only the regexp itself. In order to better understand the parser I modified the statement to:

 \lstset{moredelim=[s][\color{red}\textcolor{black}{:macro:}]{(/}{/}}

To my surprise, however, ":macro:" is applied to each of the tokens: (/, [, a, -, z, ], and /, spotted by moredelim. I wrongly expected that ":macro:" would have been applied to the entire matched expression (/[a-z]+/. Which, by the way, is the situation if I change the statement to its double-starred version:

 \lstset{moredelim=**[s][\color{red}\textcolor{black}{:macro:}]{(/}{/}}

But, in this case the regexp will be parsed by listings - coloring keywords inside the expression. This is not what I want. I want to use ( only as an anchor to find the regexp, afterwards I want to treat the anchor separately from the regexp itself (giving it a different color).

To be specific: I want to know if I can pick up only the first token in moredelim - containing the (/ - and treat this separately from the other tokens.

An MWE for examining the issue is provided here:

\documentclass{article}  
\usepackage[english]{babel}   
\usepackage{listings}  
\usepackage{xcolor} 

\lstnewenvironment{lstRuby}{  
 \lstset{  
  language={},  
  moredelim=[s][\color{red}\textcolor{black}{:macro:}]{(/}{/}  
 }  
}{}  

\begin{document}  
\setlength{\parindent}{0pt}  

\ttfamily  
This is the code:  

(/[a-z]+/)

This is what I want to achieve:

(\textcolor{red}{/[a-z]+/})

This is what I get using listings:

\begin{lstRuby}  
(/[a-z]+/)  
\end{lstRuby}  

\end{document}

Answer 1

This answer perhaps isn't relevant to the asker anymore. But because this problem was quite harder to solve then it really should have been and may be relevant to others, I'll post the answer anyway.

As mentioned in the question and comments, the moredelim=**[s] variant treats the whole delimited text as one unit/group, which allows arbitrary text to be inserted at the start of the delimited group and also at the end (through \aftergroup). The problem with that is, other styles are applied on the text between the delimiters, too. On the other hand, moredelim=[s] doesn't apply other styles but in turn applies the delimiter group style to each chunk of characters of the same class (letter, other etc.). This prevents text to be inserted only right before and after the delimiters.

There doesn't seem to be an easy way to achieve the desired effect, so we have to hook into some internals here. The \lst@DelimOpen and \lst@DelimClose macros control the actions for when a new delimiter pair is found in the listing. So we redefine them to install two hooks \@delim@open@hook and \@delim@close@hook. In these we can check what style is currently active (by comparing against \lst@currstyle) and based on that choose the proper actions. With this approach, several such fancy delimiters can be used in parallel. Note that \@delim@close@hook is executed before the last chunk of characters is output, so we again need to use the \aftergroup trick to move the final action after the last characters in the delimited group.

The final implementation uses moredelim=[is][\regexstyle]{(/}{/)} to define the style for the regexes, where the i field removes the original delimiters from the output. \regexstyle is the actual style that is applied to all the inner text of the regex. Make sure to use an unambiguously named wrapper macro here, otherwise the test against \lst@currstyle may lead to wrong results. \regexstyle@start and \regexstyle@end are the macros that insert the code in place of the original start and end delimiter, respectively.

Here's the full example:

\documentclass{article}
\usepackage{listings}
\usepackage{xcolor}

\lstnewenvironment{lstRuby}{%
    \lstset{
        language={},
        moredelim=[is][\color{green}]{*}{*},
        moredelim=[is][\regexstyle]{(/}{/)},
        emphstyle=\color{blue},
        emph={foo}
    }%
}{}

\makeatletter

\let\orig@lst@DelimOpen=\lst@DelimOpen
\def\lst@DelimOpen#1#2#3#4#5#6\@empty{%
    \orig@lst@DelimOpen{#1}{#2}{#3}{#4}{#5}{#6}\@empty
    \@delim@open@hook
}
\let\orig@lst@DelimClose=\lst@DelimClose
\def\lst@DelimClose{%
    \@delim@close@hook
    \orig@lst@DelimClose
}

\def\@delim@open@hook{%
    \def\@temp{\regexstyle}%
    \ifx\lst@currstyle\@temp
        \regexstyle@start
    \fi
}
\def\@delim@close@hook{%
    \def\@temp{\regexstyle}%
    \ifx\lst@currstyle\@temp
        \aftergroup\regexstyle@end
    \fi
}

\def\regexstyle{\color{red}}
\def\regexstyle@start{({\regexstyle /}}
\def\regexstyle@end{{\regexstyle /})}

\makeatother

\begin{document}
\setlength{\parindent}{0pt}

\ttfamily
This is the code:

(/[a-z]+/)

This is what I want to achieve:

(\textcolor{red}{/[a-z]+/})

This is what I get using listings:

\begin{lstRuby}
text (/[a-z]+/) /[a-z]+/ *foo*
text foo (/foo|\/|foo/) *bar*
\end{lstRuby}

\end{document}