Unexpected behavior when using words as delimiters in listings

Question

I am trying to customize a listings style to automatically highlight my code exactly (or sufficiently close to) how my installation of Eclipse/PyDev does. I would like to be able to use "class" and ":" as a set of delimiters, such that anything found between "class" and ":" will be formatted in a certain way. Specifically, I want "class" to have the "keyword" color, everything between "class" and ":" to be either the "classc" color or to inherit its formatting from other definitions ("Base" should be "classc"-colored and "(object)" should be "code"-colored).

I have attempted to achieve this by implementing a macro that reformats the text between the delimiters while also reprinting the delimiters with different formatting.

The first line of the given output is almost exactly what I desire, except that for some reason the ":" is appearing between Base and (object) instead of after (object). That is, the coloring is correct, but the line should be "class Base(object):".

The indention in the second line of output seems to be causing another problem. Again, the coloring is mostly correct here, but I expect the output to be "def new_function(self):". Technically, I also want "self" to have the "keyword" coloring, but I assume that's a different issue.

\documentclass{article}
\usepackage{listings, color}

\definecolor{keyword}{RGB}{221,40,103}
\definecolor{classc}{RGB}{18,144,195}
\definecolor{code}{RGB}{217,232,247}
\definecolor{bg}{RGB}{35,35,35}
\definecolor{function}{RGB}{167,236,33}

\newcommand{\classHighlight}[1]{{\ttfamily\color{keyword}class\ }{\bfseries\color{classc}#1}{:}}

\newcommand{\functionHighlight}[1]{{\ttfamily\color{keyword}def\ }{\bfseries\color{function}#1}{:}}

\lstdefinestyle{myPython}{
    language = Python,
    basicstyle = \color{code}\ttfamily,
    morekeywords=[1]{self,None,True,False,class,def},
    moredelim={*[is][\classHighlight]{class\ }{:}},
    moredelim={*[is][\functionHighlight]{def\ }{:}},
    moredelim={*[s][\color{code}]{(}{)}},
    backgroundcolor = \color{bg},
}
\lstset{style=myPython}

\begin{document}
    \begin{lstlisting}
class Base(object):
    def new_function(self):
        function contents
    \end{lstlisting}
\end{document}

When I remove the "i" tags and run with the following lines, I see output that I find even more confusing:

moredelim={*[s][\classHighlight]{class\ }{:}},
moredelim={*[s][\functionHighlight]{def\ }{:}},

Answer 1

So the problem is that listings doesn't think that the delimiter styler command is going to print anything other than it's argument, so it thinks it's safe to call it repeatedly to typeset different pieces of the code. You've told the styler to actually print text, so if it gets called multiple times, the text is printed multiple times.

To be precise it gets called either two or three times. Consider a line like:

    class Base(object):

\classHighlight gets called three times. The first time it's called, its argument is a blank space, and it's supposed to make the indent (I'm not sure why it needs to call it on the blank space -- it seems to me that the indent should be styled consistently independent of what's on the line, so this seems like a bug to me).

The second time it's called, it gets the argument Base, so this is when we want to print class and Base with the appropriate colors. If you set up the delimiters right, \classHighlight will be called a third time with argument :.

So how are we going to deal with this? We can't have the macro print "class" every time, or else "class" will appear in three places. What we're going to do is test whether we've been passed a blank space command, whether we've been passed a :, or just something else. If we've been passed a space or a colon, just output it as-is. If we've been passed something else, color it as desired.

The only other important detail is that we need to make the closing delimiter ): rather than just : for some reason. Note also that this fails horribly without moredelim={*[s][\color{code}]{(}{)}}.

Here's the full code:

\documentclass{article}
\usepackage{listings, color}


\makeatletter
% This macro makes the style commands
% #1 -- the name of the command, 
% #2 -- the styling code
\def\makeHighlightCommand#1#2{%
    \gdef#1##1{%
        \testkern{##1}% Check if ##1 starts with \kern, if so just output it
        {% else do this code
            \edef\testa{\the\lst@token}% The actual thing to be typeset is in \lst@token
            \def\testb{:}%
            \ifx\testa\testb % Test whether lst@token is just : 
                :% if so print the colon
                 % we're done now but for some reason \lst@currstyle (which is what 
                 % contains this function) doesn't go out of scope correctly.
                \global\let\lst@currstyle\relax % So we explicitly set it to \relax
            \else
                #2% Use the styling code
            \fi
        }%
    }%  
}  

% Tests if the first argument starts with \kern, 
% if so, just outputs its first argument, otherwise just outputs its second argument    
\def\testkern#1{\testkern@#1\relax\nil}
\def\testkern@#1#2\nil{\ifx#1\kern #1#2\expandafter\@gobble\fi}

% Uses listing's internal method to output a sequence of letters
% makes them super far apart like all the other letters
\def\lsttt#1{%
    \bgroup
    \let\lst@currstyle\relax
    \lsttt@#1.
    \egroup
}
\def\lsttt@#1{\ifx#1.\else\lst@length=1\lst@token{#1}\lst@Output\expandafter\lsttt@\fi}
\makeatother

\definecolor{keyword}{RGB}{221,40,103}
\definecolor{classc}{RGB}{18,144,195}
\definecolor{code}{RGB}{217,232,247}
\definecolor{bg}{RGB}{35,35,35}
\definecolor{function}{RGB}{167,236,33} 


\makeHighlightCommand\classHighlight{{\ttfamily\color{keyword}\lsttt{class}}{\bfseries\color{classc}#1}}
\makeHighlightCommand\functionHighlight{{\ttfamily\color{keyword}\lsttt{def}}{\bfseries\color{function}#1}}
\lstdefinestyle{myPython}{
    language = Python,
    basicstyle = \color{code}\ttfamily,
    morekeywords=[1]{self,None,True,False,class},
    moredelim={*[is][\classHighlight]{class\ }{):}},
    moredelim={*[is][\functionHighlight]{def\ }{):}},
    moredelim={*[s][\color{code}]{(}{)}},
    backgroundcolor = \color{bg},
}
\lstset{style=myPython}

\begin{document}
\begin{lstlisting}
class Base(object):
    def new_function(self):
         contents
\end{lstlisting}

\end{document}

Here's the output:

Let me know if this doesn't work the way you want.

Edit: So Kevin complained that the space between the letters of the keywords def and class is not the same as the tons of space between the letters of everything else. To fix this, I added the code:

\def\lsttt#1{%
    \bgroup
    \let\lst@currstyle\relax
    \lsttt@#1.
    \egroup
}
\def\lsttt@#1{\ifx#1.\else\lst@length=1\lst@token{#1}\lst@Output\expandafter\lsttt@\fi}

and wrapped class and def with lsttt. With the spacing now consistently applied it looks like this: