How to automatically skip leading white spaces in listings

Question

This question led to a new package:
lstautogobble (lstaddons bundle)

For my programming lecture slides I heavily use the listings package with \lstnewenvironment to typeset source code. It's a great package (no, I do not want to switch to minted), the only thing that bugs me is that I have to pass the gobble=<indent> option to get well formatted source as well as output:

\documentclass{beamer}
\usepackage{listings}
\lstloadlanguages{C}

\lstnewenvironment{C}[1][] {\lstset{language=C, basicstyle=\scriptsize\ttfamily, #1}}{} 

\begin{document}

\begin{frame}[fragile]{A simple truth\ldots}
  \begin{C}[gobble=4] % <-- ** This is annoying! **
    #include <stdio.h>
    int main(){
      printf("tex.stackexchange.com: the coolest community ever!\n");
    }
  \end{C}
\end{frame}

\end{document}

Manual handling of gobble is especially painful, if one moves code examples around and changes the nesting level. I constantly forget to update the parameter in this situations. Ideally, the C environment in the above example would automatically determine the gobble by the preceeding indentation (that is, number of spaces) before it in the source. Something like the following (pseudo code):

\lstnewenvironment{C}[1][] {\PreceedingWS{\WScount}%
  \lstset{language=C, basicstyle=\scriptsize\ttfamily, 
  gobble=\the\numexpr\WScount+2\relax, #1}}{} 

The idea is that \PreceedingWS magically determines the number of spaces in the current line before the \begin{C} and stores it in some macro (\WScount).

Is this possible?

---

Summary of Results (2011-06-27)

I got three answers, two of which did actually help me:

• Aditya's solution is based on ConTeXt, which doesn't help me, but may be a good reference for those guys who actually use ConTeXt.
• Bruno's solution is a bit hacky, but has the advantage that it can be applied to (probably) any verbatim-style environment. Moreover, I have learned a bit about TeX scanning from his well-documented answer. It came "right in time" for me to safe me a reasonable amount of work.
• Martin's solution showed up relatively late. It is much cleaner as it integrates directly into the listings package and is provided as an own package. I am surprised by how little code he needed for this, which for me is an indicator that his solution integrates well and has a good chance to survive further evolution of the listings package. I certainly hope his lstautogobble package will make it to CTAN or eventually be directly integrated into listings.

I have tested the latter two with my lecture slides, which contain ~100 small listings in very different settings and environments (inside tables, minipages, in combination with beamer overlays, ...), so I can say that both approaches do actually work.

To sum up: Bruno's answer became the accepted answer, as it helped me most when I actually needed it. Martin's answer received the bounty, as it is the cleaner and more stable solution which I am going to use in any future project.

Answer 1

A different approach, hence separate answer. Capture the body of the environment (not in a very robust way, but I think you don't nest those environments), and count the number of leading spaces. Somewhat tricky to get right with the \endlinechar and \newlinechar. Alas, if some lines are less indented than the first, they will be trimmed.

\documentclass{article}

\usepackage{listings}
\lstloadlanguages{C}
\lstnewenvironment{CCCC}[1][]
  {\lstset{language=C, basicstyle=\scriptsize\ttfamily, #1}}{}
\makeatletter
% ===== Helper.
% Use some tokens after fully expanding them: for this, redefine
% \use@x within a group, and use it outside using \expandafter.
\newcommand{\use@x}[1]
  {\begingroup\edef\use@x{#1}\expandafter\endgroup\use@x}

% ===== Environment.
% Give "special" characters the catcode "other", and tell TeX
% to insert the "newline" character at the end of every line
% (this "newline" character causes TeX to produce a new line
% when writing to a file (done in \scantokens later)).
\newenvironment{C}
  {%
    \let\do\@makeother\dospecials
    \endlinechar\newlinechar
    \LS@hack@i
  }{}
% Look for an optional argument. 
\newcommand{\LS@hack@i}[1][]{\LS@hack@ii{#1}}
% Expand...
\use@x{%
  % Roughly \def\LS@hack@ii#1#2\end{C}^^J{\LS@hack@iii{#2}{#1}}
  % except that some pieces of \end{C} are converted to strings,
  % and one space is removed from #2.
  \unexpanded{\def\LS@hack@ii#1#2}%
    \expandafter\@gobble\string\\end\string{C\string}^^J%
    \unexpanded{{\expandafter\LS@hack@iii\expandafter{\romannumeral-`0#2}{#1}}}%
  %
  % Essentially,
  % \def\LS@hack@iii{\end{C}
  %   \begin{CCCC}[gobble=numberofspaces]##1\end{CCCC}}
  % Restore catcodes of special characters and reread,
  % with \scantokens, the body of the environment, feeding
  % it to the listing CCCC environment. Note that the user-
  % provided optional argument to begin{C} is transmitted
  % to begin{CCCC} (not tested). All the \noexpand are there
  % to prevent the full expansion of \use@x, necessary
  % for \string\begin and \string\end.
  \noexpand\newcommand{\noexpand\LS@hack@iii}[2]
    {\noexpand\end{C}%
      \noexpand\LS@hack@countspaces{##1}%
      \noexpand\scantokens{%
        \string\begin{CCCC}[gobble=\noexpand\value{LS@hack@spacecount},##2]^^J%
          ##1%
        \string\end{CCCC}}}}

\newcounter{LS@hack@spacecount}
\newcommand{\LS@hack@countspaces@}[1]{%
  \if#1 \else
    \expandafter\remove@to@nnil
  \fi
  +1\LS@hack@countspaces@}
\newcommand{\LS@hack@countspaces}[1]{%
  \setcounter{LS@hack@spacecount}
    {\the\numexpr0\LS@hack@countspaces@#1\@nnil\relax}}


\begin{document}
  A piece of code:
  \begin{C}
    #include <stdio.h>
    int main(){
      printf("tex.stackexchange.com: the coolest community ever!\n");
    }
  \end{C}
  and the same with no gobble:
  \begin{CCCC}
    #include <stdio.h>
    int main(){
      printf("tex.stackexchange.com: the coolest community ever!\n");
    }
  \end{CCCC}
\end{document}

Answer 2

IMHO such a feature should be implemented using the internals of listings itself, e.g. by using the same catcodes etc., to maximize the compatibility with the normal operations of the environment.

The listings package uses an internal macro \lstenv@Process to process the lstlisting and other listing environments defined by \lstnewenvironment. This macro is used after the environment arguments are processed and could be locally redefined to count the spaces (then active!) after the first line break (^^M, the one after \begin{lstlisting}[options]), then set the gobble option with the right number and call the original \lstenv@Process. That macro actually calls itself recursively, so its original definition must be also restored.

Here now my solution as package. It adds the above functionality as boolean option autogobble, which can be used like any other option with all listing environments defined by listings, including custom ones. A test file to show the proper function of it also follows.

MWE (based on original MWE):

\documentclass{beamer}
\usepackage{listings}
\usepackage{lstautogobble}
\lstloadlanguages{C}

\lstnewenvironment{C}[1][] {\lstset{language=C, basicstyle=\scriptsize\ttfamily, autogobble, #1}}{} 

\begin{document}

\begin{frame}[fragile]{A simple truth\ldots}
  \noindent Normal text as reference
  \begin{C}
    #include <stdio.h>
    int main(){
      printf("tex.stackexchange.com: the coolest community ever!\n");
    }
  \end{C}
\end{frame}

\end{document}

Package:

(v1.1 2012/02/04: Now with support for tabulators)

% LaTeX Package `lstautogobble`
% Counts the leading spaces of the first line and sets `gobble` to this number
% Copyright (c) 2011 by Martin Scharrer <[email protected]>
% for http://tex.stackexchange.com/questions/19953/how-to-automatically-skip-leading-white-spaces-in-listings
% This is free software under the LPPL v1.3c or later and was also posted under the CC BY-SA 3.0 license.
\ProvidesPackage{lstautogobble}[2012/02/04 v1.1 Implements 'autogobble' option for 'listings']

% This is an add-on to the `listings` package
\RequirePackage{listings}

% Counter for leading spaces
\newcount\lstag@spacecount

% Some macros for comparison:
\def\lstag@activespace{\lst@ProcessSpace}%  Definition of an active space
\def\lstag@tabulator{\lst@ProcessTabulator}%  Definition of an tabulator

\begingroup
\catcode`\^^M=\active%
\gdef\lstag@activenl{^^M}%  Active CR (ASCII 13) character which is used as line break
\endgroup


% Define `autogobble` option as boolean (by default off)
\lst@Key{autogobble}{false}[t]{\lstKV@SetIf{#1}\lst@ifautogobble}

% `ungobble` option
\lst@Key{ungobble}{0}{\def\lst@ungobble{#1}}

% Insert required code at environment init
\lst@AddToHook{Init}{\lst@autogobble}

% Autogobble init macro.
% If the option is active and `gobble` is not set, init vars and overwrite the process macro with own definition.
\def\lst@autogobble{%
    \lst@ifautogobble
        \ifnum\lst@gobble>0\else
            \def\lst@gobble{\lstag@gobble}%
            \def\lstag@gobble{0}%
            \lstag@spacecount\z@
            \def\lstag@spaceaccu{}%
            \let\lstag@restofline\empty
            \let\lstag@origlstenv@Process\lstenv@Process
            \let\lstenv@Process\lstag@countleadingspaces
        \fi
    \fi
}

% Checks if the next following character (read as argument) is a line break (as it is supposed to be)
% Otherwise there is some text direct after the `\begin{<env>}[<options>]` which is dropped by `listings`.
\def\lstag@countleadingspaces#1{%
    \expandafter\ifx\lstag@activenl#1\relax
        \expandafter\lstag@countleadingspaces@
    \else
        \def\lstag@restofline{Dummy replacement of text after begin of listing to trigger original warning message}%
        \expandafter\lstag@countleadingspaces
    \fi
}

% After the new line is found this macro counts the spaces and tabulators
\def\lstag@countleadingspaces@#1{%
    \ifx\lstag@activespace#1\relax
        \advance\lstag@spacecount by \@ne
        % Accumulate spaces (i.e. their definitions) for later re-insertion:
        \expandafter\def\expandafter\lstag@spaceaccu\expandafter{\lstag@spaceaccu\lst@ProcessSpace}%
        \let\next\lstag@countleadingspaces@
    \else% Character wasn't a space
    \ifx\lstag@tabulator#1\relax
        \advance\lstag@spacecount by \lst@tabsize\relax
        % Accumulate spaces (i.e. their definitions) for later re-insertion:
        \@tempcnta=\lst@tabsize\relax
        \loop
        \ifnum\@tempcnta>\z@
            \expandafter\def\expandafter\lstag@spaceaccu\expandafter{\lstag@spaceaccu\lst@ProcessSpace}%
            \advance\@tempcnta\m@ne
        \repeat
        \let\next\lstag@countleadingspaces@
    \else% Character wasn't a tabulator either
        % Set gobble option (indirect):
        \xdef\lstag@gobble{\the\numexpr\lstag@spacecount-\lst@ungobble\relax}%
        % Restore original definition of process macro:
        \global\let\lstenv@Process\lstag@origlstenv@Process
        % Re-insert all collected material or appropriate replacement material:
        \edef\next{\noexpand\lstenv@Process\lstag@restofline\expandafter\noexpand\lstag@activenl\expandafter\unexpanded\expandafter{\lstag@spaceaccu}\noexpand#1}%
    \fi\fi
    \next
}

\endinput

Test File:

\documentclass{article}

\usepackage{listings}
\usepackage{lstautogobble}


\lstnewenvironment{test}[1][]{%
    \lstset{basicstyle=\ttfamily,autogobble=true,#1}%
}{}

\parindent=0pt% for test only
\begin{document}

Normal text as reference

\hrule

    Only environment
    \begin{test}
        test
        it
    \end{test}

    With options
    \begin{test}[basicstyle=\ttfamily\scriptsize]
        test
        it
    \end{test}

    Manual gobble option (override)
    \begin{test}[gobble=7]
        test
        it
    \end{test}

    Autogobble off:
    \begin{test}[autogobble=false]
        test
        it
    \end{test}

    With some material on the same line as \texttt{\string\begin} (dropped by listings. The warning message got preserved)
    \begin{test} some text at the first line
        test
        it
    \end{test}

    dito with optional argument
    \begin{test}[] some text at the first line
        test
        it
    \end{test}

    Different indention levels:
\begin{test}
    test
    it
\end{test}

        \begin{test}
            test
            it
        \end{test}

        \begin{test}
    test
    it
        \end{test}

            \begin{test}
                            test
                            it
            \end{test}

    \begin{test}
                    test
                    it
    \end{test}


\hrule

  Some real C Code
  \begin{lstlisting}[autogobble]
    #include <stdio.h>
    int main(){
      printf("tex.stackexchange.com: the coolest community ever!\n");
    }
  \end{lstlisting}

\end{document}

Test Results:

Answer 3

This is relatively straight forward to do in LuaTeX. The ConTeXt wiki has this exact question as an example of programming in luatex. The solution consists of two parts:

• capturing the content of the environment: in ConTeXt, this is done using buffers. I think that one of the many LaTeX verbatim packages provide a similar functionality.

• remove the leading spaces from the content: This is easy in Lua. The complexity of the solution depends on how robust you want the implementation to be (what should happen if the source if badly formatted, i.e., later lines have less white space than the first line. Here is the dedenting function from the ConTeXt wiki

\startluacode
  -- Initialize a userdata name space to keep our own functions in.
  -- That way, we won't interfere with anything ConTeXt keeps in 
  -- the global name space.
  userdata = userdata or {}

  function userdata.dedentedtyping(content)
    local lines    = string.splitlines(content)
    local indent   = string.match(lines[1], '^ +') or ''
    local pattern  = '^' .. indent
    for i=1,#lines do
      lines[i] = string.gsub(lines[i],pattern,"")
    end

    content = table.concat(lines,'\n')

    tex.sprint("\\starttyping\n" .. content .. "\\stoptyping\n")

  end
\stopluacode

For listings, instead of passing \starttyping ... \stoptyping back to TeX, you can pass \begin{listing} ... \end{listing}

Answer 4

I'm cheating: using the file that beamer produces (\jobname.vrb) for fragile frames, and counting spaces between some extra command (\setgobble) and the following \begin. This is limited to one \setgobble per frame, and to values of gobbling at most 8 (or 10?).

I think that it would be better to count spaces within the listings environment somehow, and put its input back to what it expects.

\documentclass{beamer}
\usepackage{listings}
\lstloadlanguages{C}

\lstnewenvironment{C}[1][] {\lstset{language=C, basicstyle=\scriptsize\ttfamily, #1}}{} 


\makeatletter
\newcount\gobblecount
\newcommand{\setgobble}
  {%
    \begingroup
    \everyeof{\@nnil}%
    \obeyspaces
    \expandafter\setgobble@aux\@@input\jobname.vrb%
    \endgroup
  }
\newcommand\@ninthofnine[9]{#9}
\def\setgobble@aux#1\setgobble#2\begin{%
    \global\gobblecount
      \expandafter\@car\@ninthofnine #2876543210\@nil\relax
    \global\advance\gobblecount\tw@
    \remove@to@nnil
  }
\makeatother

\begin{document}

\begin{frame}[fragile]{A simple truth\ldots}
  \setgobble
  \begin{C}[gobble=\the\gobblecount] % <-- ** This is annoying! **
    #include <stdio.h>
    int main(){
      printf("tex.stackexchange.com: the coolest community ever!\n");
    }
  \end{C}
\end{frame}

\end{document}