Listings: color numbers only out of keywords

Question

With the listings package, I would like to color the integers out of strings and comments and out of keywords too. The question for coloring numbers has been asked several times. Some answer handle the numbers occurring within comments, strings or barewords, but no solution given work when a keyword contain a number. Which is annoying because C and C++ have a lot of types with a number in them, like uint32_t.

Here is a small example.

\documentclass{standalone}
\usepackage{xcolor}
\usepackage{listings}

\definecolor{darkgreen}{rgb}{0.18,0.54,0.34}
\definecolor{maroon}{rgb}{0.64,0.16,0.16}
\definecolor{darkpink}{rgb}{0.75,0.25,0.5}

\lstdefinestyle{myc++}{
    language=[ISO]C++,
    keywordstyle=\color{darkgreen}\bfseries,
    commentstyle=\color{blue}\textit,
    stringstyle=\color{darkpink}\ttfamily,
%
    literate=*
    {0}{{{\textcolor{red}0}}}1
    {1}{{{\textcolor{red}1}}}1
    {2}{{{\textcolor{red}2}}}1
    {3}{{{\textcolor{red}3}}}1
    {4}{{{\textcolor{red}4}}}1
    {5}{{{\textcolor{red}5}}}1
    {6}{{{\textcolor{red}6}}}1
    {7}{{{\textcolor{red}7}}}1
    {8}{{{\textcolor{red}8}}}1
    {9}{{{\textcolor{red}9}}}1,
    escapeinside={|}{|},
%
    morekeywords={int32_t}
}

\begin{document}

\begin{lstlisting}[style=myc++]
// In comment: 42
int32_t in_keyword = 80085;
char *in_identifier|2| = "And in string 1337\n";
\end{lstlisting}

\end{document}

And the result:

The star for literate avoid to coloring the digits inside comments or strings. The escapeinside avoid coloring the digits inside a bareword when specifically marked. But I have yet to find how to color int32_t as a keyword.

Answer 1

I know that the question is nearly three years old now, but I am dealing with the same problem at the moment. I have searched the internet and the same or closley related questions have been asked a lot of times without 100% satisfying answers to me because I am looking for a solution like the one that was desired here. As I have found one which is not perfect, but quite good, I want to share it in case someone else comes across here. But generally I would agree with @Jubobs, it's really a pain in the neck doing this in listings.

So basically what I did is I colored everything with the color I want to have for the numbers using the basicstyle attribute. Then I colored the keywords, comments and strings as usal using the keywordstyle, commentsytle and stringstlye attributes. Now the trick is to use the identifierstyle to color all variable names in the actual basic color. What is left concerning the basic style then is all digits as well as all operators, brakets and so on like ( + - * ... After that you obviously only have to color the operators, and breakets and so on with the basic color to get your result. This can be done with the literate attribute.

I will add a few lines of code from my Julia style that I am creating right now to illustrate what I mean. Notice that in my case I wanted to color the numbers with the same color as the strings, but you can obiously simply create a new one for that job.

% julia language definition

\lstdefinelanguage{julia}{%
morekeywords=[1]{end,export,finally},%
morekeywords=[2]{true,false,ARGS},%
morekeywords=[3]{ANY,AbstractArray,AbstractChannel},%
morecomment=[l]{\#},%
morecomment=[n]{\#=}{=\#},%
morestring=[b]{"},%
morestring=[m]{'},%
}[keywords,comments,strings]



% defining the colors for
\definecolor{jlbase}{HTML}{444444}                       % julia's base color
\definecolor{jlkeyword}{HTML}{444444}                    % julia's keywords
\definecolor{jlliteral}{HTML}{78A960}                    % julia's literals
\definecolor{jlbuiltin}{HTML}{397300}                    % julia's built-ins
\definecolor{jlcomment}{HTML}{888888}                    % julia's comments
\definecolor{jlstring}{HTML}{880000}                     % julia's strings



% basic font
\def\lstbasicfont{\color{jlstring}\fontfamily{pcr}\selectfont\scriptsize}

% defining the styles for
\lstset{keywordstyle={[1]\color{jlkeyword}\bfseries}}    % julia's keywords
\lstset{keywordstyle={[2]\color{jlliteral}}}             % julia's literals
\lstset{keywordstyle={[3]\color{jlbuiltin}}}             % julia's built-ins
\lstset{commentstyle={\color{jlcomment}}}                % julia's comments
\lstset{stringstyle={\color{jlstring}}}                  % julia's strings
\lstset{identifierstyle={\color{jlbase}}}        % variables


% coloring the operators with the basecolor
% and . in floating point numbers with the desired number color
\lstset{extendedchars=false}
\lstset{literate=*
{\\}{{{\color{jlbase}\textbackslash{}}}}{1} {\{}{{{\color{jlbase}\{}}}{1} {\}}{{{\color{jlbase}\}}}}{1}
{!}{{{\color{jlbase}!}}}{1} {\%}{{{\color{jlbase}\%}}}{1} {&}{{{\color{jlbase}\&}}}{1}
{(}{{{\color{jlbase}(}}}{1} {)}{{{\color{jlbase})}}}{1} {*}{{{\color{jlbase}*}}}{1}
{+}{{{\color{jlbase}+}}}{1} {,}{{{\color{jlbase},}}}{1} {-}{{{\color{jlbase}-}}}{1}
{.}{{{\color{jlbase}.}}}{1} {/}{{{\color{jlbase}/}}}{1} {:}{{{\color{jlbase}:}}}{1}
{;}{{{\color{jlbase};}}}{1} {<}{{{\color{jlbase}<}}}{1} {=}{{{\color{jlbase}=}}}{1}
{>}{{{\color{jlbase}>}}}{1} {?}{{{\color{jlbase}?}}}{1} {[}{{{\color{jlbase}[}}}{1}
{]}{{{\color{jlbase}]}}}{1} {^}{{{\color{jlbase}\^{}}}}{1} {|}{{{\color{jlbase}|}}}{1}
{~}{{{\color{jlbase}\textasciitilde{}}}}{1}
{.0}{{{\color{jlstring}.0}}}{2} {.1}{{{\color{jlstring}.1}}}{2} {.2}{{{\color{jlstring}.2}}}{2}
{.3}{{{\color{jlstring}.3}}}{2} {.4}{{{\color{jlstring}.4}}}{2} {.5}{{{\color{jlstring}.5}}}{2}
{.6}{{{\color{jlstring}.6}}}{2} {.7}{{{\color{jlstring}.7}}}{2} {.8}{{{\color{jlstring}.8}}}{2}
{.9}{{{\color{jlstring}.9}}}{2}}



% activating the julia style
\lstset{language=julia}

Ok, I guess this is what you consider a MWE:

\documentclass{standalone}
\usepackage{xcolor}
\usepackage{listings}

\definecolor{darkgreen}{rgb}{0.18,0.54,0.34}
\definecolor{maroon}{rgb}{0.64,0.16,0.16}
\definecolor{darkpink}{rgb}{0.75,0.25,0.5}

\lstdefinestyle{myc++}{
    language=[ISO]C++,
    keywordstyle=\color{darkgreen}\bfseries,
    commentstyle=\color{blue}\textit,
    stringstyle=\color{darkpink}\ttfamily,
    basicstyle={\color{red}},
    identifierstyle={\color{black}}, 
%
    literate=*
    {*}{{{\color{black}*}}}{1}
    {=}{{{\color{black}=}}}{1}
    {;}{{{\color{black};}}}{1}
    {.7}{{{\color{red}.7}}}{2},%
%
morekeywords={int32_t}
}

\begin{document}

\begin{lstlisting}[style=myc++]
// In comment: 42
int32_t in_keyword = 80085.7;
char *in_identifier2 = "And in string 1337\n";
\end{lstlisting}

\end{document}

The result looks like:

Note that with the approach you dont even need to escape the 2 in in_identifier2.

Answer 2

A disadvantage of the solutions presented in the various threads so far, in my opinion, is that they heavily rely on the literate feature, which is very invasive and may break the processing of other code elements. So here's another approach that is still far from being perfect but can be used nicely on top of existing language and style definitions.

The basic idea

First some words about how listings parses its syntax elements. When it tries to parse an identifier, it looks for something it considers a letter character, followed by several other letters or digits. All those are merged together into an identifier token which is then subject to further processing, e.g. keyword detection. Whenever a digit or other character is found that is not part of an identifier name, it is formed with all following digits and others into one sequence, which is then output as one group of characters.

Now the idea is simply to hook into this output process and identify all character sequences that form a legal number literal. Whenever such a number is found, we want to apply a special numbers style, otherwise we just leave the output as it is.

Implementation notes

For special treatment of such non-identifier sequences listings already provides a hook called OutputOther that we will use to preprocess those sequences (stored in \lst@token).

To parse numbers we need a method that is able to process at least the minus sign differently than by just looking for a - character, as in listings those are handled through the \lst@um- macro internally. Instead a literal copy of the macros provided by this answer was used. Additionally, the output of a hyphen in listings is prefixed by a macro called \lst@nolig to prevent TeX from building ligatures if several of them occur in a row. As for numbers a hyphen/minus can only occur at the beginning of a number, we just need to test the first token of the sequence for \lst@nolig and discard it if necessary.

The rest of the implementation is quite straight-forward. To allow easier customization, two new listings options parsenumbers=(true|false) and numbersstyle={...} were defined (not to be confused with the already existing numberstyle option!). The former enables/disables the parsing routine for numbers, the latter sets the style to be applied. We also check if mode changes are allowed (\lst@ifmode is false) before parsing the number, such that the numbers style is not applied in non-starred delimiters.

As demonstrated in the example below, this solution works quite well in many standard cases, but it fails (at least) in the following:

• For sequences like <identifier><plus/minus><integer>, because listings breaks parsing after the identifier and treats the plus/minus and integer as one signed number. This is a general lexing problem.
• For floating point literals in scientific notation, like 0.12e3, the parsing process would treat e as an identifier and thus break the whole number group. I can't see a good fix for this. As a workaround e could be made a digit character, but that would break parsing of identifiers/keywords that start with the letter e.
• If digits are used as literate characters, parsing the whole number group will also fail. Using + and - as literate characters works to some extent.

Code with example document

\documentclass{article}
\usepackage{listings}
\usepackage{xcolor}

\makeatletter

%%% Copied from https://tex.stackexchange.com/a/500690/23765
% Some conditional tests
\def\@genericif#1{#1\expandafter\@firstoftwo\else\expandafter\@secondoftwo\fi}
\def\@ifdigit#1{\@genericif{\ifnum1<1\noexpand#1\relax}}
\def\@ifempty#1{\@genericif{\if\relax\detokenize{#1}\relax}}

% The main parsing macros
\def\parse@num#1{%
    \@ifempty{#1}%
        {\parse@num@false}%
        {\@genericif{\parsesign}%
            {\parse@num@sign#1{}\@end}%
            {\parse@num@dig#1{}\@end}%
        }%
}
% Parse sign
\def\parse@num@sign#1#2\@end{%
    \@genericif{\ifx\parse@num@minus#1}%
        {\@ifempty{#2}{\parse@num@false}{\parse@num@dig#2\@end}}%
        {\@genericif{\ifx\parse@num@plus#1}%
            {\@ifempty{#2}{\parse@num@false}{\parse@num@dig#2\@end}}%
            {\parse@num@dig#1#2\@end}%
        }%
}
% Parse first digit
\def\parse@num@dig#1#2\@end{%
    \@ifdigit{#1}%
        {\@ifempty{#2}{\parse@num@true}{\parse@num@digs#2\@end}}%
        {\parse@num@false}%
}
% Parse optional following digits
\def\parse@num@digs#1#2\@end{%
    \@ifdigit{#1}{%
        \@ifempty{#2}%
            {\parse@num@true}%
            {\parse@num@digs#2\@end}%
    }{%
        \@genericif{\parsefloat}{%
            \@genericif{\ifx\parse@num@point#1}%
                {\@ifempty{#2}{\parse@num@false}{\parse@num@decs#2\@end}}%
                {\parse@num@false}%
        }{\parse@num@false}%
    }%
}
% Parse decimal places
\def\parse@num@decs#1#2\@end{%
    \@ifdigit{#1}{%
        \@ifempty{#2}%
            {\parse@num@true}%
            {\parse@num@decs#2\@end}%
    }{\parse@num@false}%
}

% User interface
\newcommand\ifnumber[4][]{%
    \begingroup
    \let\parsesign=\iftrue
    \let\parsefloat=\iftrue
    \let\parse@num@minus=-%
    \let\parse@num@plus=+%
    \let\parse@num@point=.%
    #1%
    \def\parse@num@true{\endgroup#3}%
    \def\parse@num@false{\endgroup#4}%
    \parse@num{#2}%
}   


%%% Additions to the listings package
\lst@Key{numbersstyle}{}{\def\lst@numbersstyle{#1}}
\lst@Key{parsenumbers}{false}[t]{\lstKV@SetIf{#1}\lst@ifparsenumbers}

\lst@AddToHook{OutputOther}{%
    \lst@ifparsenumbers
        % Only if mode changes are not prohibited
        \lst@ifmode\else
            \expandafter\@hook@ifnumber\the\lst@token\@end
                {\let\lst@thestyle=\lst@numbersstyle}%
                {}%
        \fi
    \fi
}
\def\@hook@ifnumber#1#2\@end{%
    \@genericif{\ifx\lst@nolig#1}%
        {\@hook@ifnumber@{#2}}%
        {\@hook@ifnumber@{#1#2}}%
}
\def\@hook@ifnumber@{%
    \ifnumber[\expandafter\let\expandafter\parse@num@minus\csname lst@um-\endcsname]%
}

\makeatother

%%% Example document
\lstset{
    basicstyle = \ttfamily,
    identifierstyle = \color{blue},
    keywordstyle = \color{green!80!black},
    keywords = {foo},
    moredelim = [il][]{**},
    moredelim = [l][\color{gray}]{/},
    morestring = [d][\color{gray}]{"},
    morestring = *[d][\color{gray}\itshape]{!},
    morestring = **[d][\color{gray}\itshape]{?},
    % Apply new number coloring routine
    parsenumbers = true,
    numbersstyle = {\color{magenta}}
}

\begin{document}
\begin{lstlisting}
**good cases:
   foo   foo123   _123
   123   12.345   .123
  -123  -12.345  -.123
  +123  +12.345  +.123
  +++1  1++2++3   ---1
   1-2  12 - 34    1+2

**problematic cases:
   i+1   0.1e12    i-1

**delimiters:
  / foo  123    line mode
  " foo  123 "  **delimited 
  ! foo  123 !  **inner styles
  ? foo  123 ?  **cumulative styles
\end{lstlisting}

\lstset{
    literate = {+}{{\textcolor{green}{\char`\+}}}1
               {-}{{\textcolor{red}{\char`\-}}}1
               {num}{123}3
}
\begin{lstlisting}
**literate cases:
  -123  -12.345  -.123
  +123  +12.345  +.123
  ++1+  1++2--3   --1-
   1-2  12 - 34    1+2
   num  num+num   -num
\end{lstlisting}
\end{document}