# How to automatically skip leading white spaces in listings

This question led to a new package:
lstautogobble (lstaddons bundle)

For my programming lecture slides I heavily use the listings package with \lstnewenvironment to typeset source code. It's a great package (no, I do not want to switch to minted), the only thing that bugs me is that I have to pass the gobble=<indent> option to get well formatted source as well as output:

\documentclass{beamer}
\usepackage{listings}

\lstnewenvironment{C}[1][] {\lstset{language=C, basicstyle=\scriptsize\ttfamily, #1}}{}

\begin{document}

\begin{frame}[fragile]{A simple truth\ldots}
\begin{C}[gobble=4] % <-- ** This is annoying! **
#include <stdio.h>
int main(){
printf("tex.stackexchange.com: the coolest community ever!\n");
}
\end{C}
\end{frame}

\end{document}


Manual handling of gobble is especially painful, if one moves code examples around and changes the nesting level. I constantly forget to update the parameter in this situations. Ideally, the C environment in the above example would automatically determine the gobble by the preceeding indentation (that is, number of spaces) before it in the source. Something like the following (pseudo code):

\lstnewenvironment{C}[1][] {\PreceedingWS{\WScount}%
\lstset{language=C, basicstyle=\scriptsize\ttfamily,
gobble=\the\numexpr\WScount+2\relax, #1}}{}


The idea is that \PreceedingWS magically determines the number of spaces in the current line before the \begin{C} and stores it in some macro (\WScount).

Is this possible?

Summary of Results (2011-06-27)

I got three answers, two of which did actually help me:

• Aditya's solution is based on ConTeXt, which doesn't help me, but may be a good reference for those guys who actually use ConTeXt.
• Bruno's solution is a bit hacky, but has the advantage that it can be applied to (probably) any verbatim-style environment. Moreover, I have learned a bit about TeX scanning from his well-documented answer. It came "right in time" for me to safe me a reasonable amount of work.
• Martin's solution showed up relatively late. It is much cleaner as it integrates directly into the listings package and is provided as an own package. I am surprised by how little code he needed for this, which for me is an indicator that his solution integrates well and has a good chance to survive further evolution of the listings package. I certainly hope his lstautogobble package will make it to CTAN or eventually be directly integrated into listings.

I have tested the latter two with my lecture slides, which contain ~100 small listings in very different settings and environments (inside tables, minipages, in combination with beamer overlays, ...), so I can say that both approaches do actually work.

To sum up: Bruno's answer became the accepted answer, as it helped me most when I actually needed it. Martin's answer received the bounty, as it is the cleaner and more stable solution which I am going to use in any future project.

• I think that the right approach would be to count the number of white spaces at the start of the first line of C code. Jun 6, 2011 at 3:33
• I considered this as well, however, guess that this is even more challenging as one probably has to hack deeply inside the listings package. Another point is that there may be occasions where the first line does not actually define the indenting, for example, if continuing the listing of some longer function. (Well, the latter could count as a pathological case where it would be perfectly acceptable to specify gobble manually.) Jun 6, 2011 at 6:35
• Easy solution: don't indent your source code. It doesn't look nice/properly nested but large non-TeX code blocks ruin your code flow anyway! Alternatively put them in external files and use \lstinputlisting. Jun 21, 2011 at 11:46
• @Martin Scharrer: I did this, but have stopped a while ago. IMHO the LaTex code becomes too obfuscated by that: The vast majority of my listings are small code snippets (5-10 lines); however I have lots of them. I also tried \lstinputlisting, but figured that it is organizational overkill and reduces the comprehensibility of both sources (LatTeX and listing) quite a bit, especially as I tend to "pollute" my listings with beamer overlay stuff. To sum up: Embedding and indenting them in the LaTeX code really is what I am up to. Jun 21, 2011 at 12:15
• FWIW, I added such an option to the t-vim module, which can be activated by trimspaces=on. Since the syntax highlighting is done in vim, calculating the maximum leading white space is easy. Jun 27, 2011 at 1:39

A different approach, hence separate answer. Capture the body of the environment (not in a very robust way, but I think you don't nest those environments), and count the number of leading spaces. Somewhat tricky to get right with the \endlinechar and \newlinechar. Alas, if some lines are less indented than the first, they will be trimmed.

\documentclass{article}

\usepackage{listings}
\lstnewenvironment{CCCC}[1][]
{\lstset{language=C, basicstyle=\scriptsize\ttfamily, #1}}{}
\makeatletter
% ===== Helper.
% Use some tokens after fully expanding them: for this, redefine
% \use@x within a group, and use it outside using \expandafter.
\newcommand{\use@x}[1]
{\begingroup\edef\use@x{#1}\expandafter\endgroup\use@x}

% ===== Environment.
% Give "special" characters the catcode "other", and tell TeX
% to insert the "newline" character at the end of every line
% (this "newline" character causes TeX to produce a new line
% when writing to a file (done in \scantokens later)).
\newenvironment{C}
{%
\let\do\@makeother\dospecials
\endlinechar\newlinechar
\LS@hack@i
}{}
% Look for an optional argument.
\newcommand{\LS@hack@i}[1][]{\LS@hack@ii{#1}}
% Expand...
\use@x{%
% Roughly \def\LS@hack@ii#1#2\end{C}^^J{\LS@hack@iii{#2}{#1}}
% except that some pieces of \end{C} are converted to strings,
% and one space is removed from #2.
\unexpanded{\def\LS@hack@ii#1#2}%
\expandafter\@gobble\string\\end\string{C\string}^^J%
\unexpanded{{\expandafter\LS@hack@iii\expandafter{\romannumeral-0#2}{#1}}}%
%
% Essentially,
% \def\LS@hack@iii{\end{C}
%   \begin{CCCC}[gobble=numberofspaces]##1\end{CCCC}}
% Restore catcodes of special characters and reread,
% with \scantokens, the body of the environment, feeding
% it to the listing CCCC environment. Note that the user-
% provided optional argument to begin{C} is transmitted
% to begin{CCCC} (not tested). All the \noexpand are there
% to prevent the full expansion of \use@x, necessary
% for \string\begin and \string\end.
\noexpand\newcommand{\noexpand\LS@hack@iii}[2]
{\noexpand\end{C}%
\noexpand\LS@hack@countspaces{##1}%
\noexpand\scantokens{%
\string\begin{CCCC}[gobble=\noexpand\value{LS@hack@spacecount},##2]^^J%
##1%
\string\end{CCCC}}}}

\newcounter{LS@hack@spacecount}
\newcommand{\LS@hack@countspaces@}[1]{%
\if#1 \else
\expandafter\remove@to@nnil
\fi
+1\LS@hack@countspaces@}
\newcommand{\LS@hack@countspaces}[1]{%
\setcounter{LS@hack@spacecount}
{\the\numexpr0\LS@hack@countspaces@#1\@nnil\relax}}

\begin{document}
A piece of code:
\begin{C}
#include <stdio.h>
int main(){
printf("tex.stackexchange.com: the coolest community ever!\n");
}
\end{C}
and the same with no gobble:
\begin{CCCC}
#include <stdio.h>
int main(){
printf("tex.stackexchange.com: the coolest community ever!\n");
}
\end{CCCC}
\end{document}

• That looks promising! I will try it today within my beamer setup. With respect to the trimming of lines that are less intended than the first line: I could imagine an extraindent key, whose value is just added to the automatically extracted indent. As this value depends only on the listing, but not on the LaTeX nesting level, this would be stable even if the listing is moved around. Jun 22, 2011 at 7:13
• First tests (with about 20 listings) have been quite successful; also passing of optional arguments works fine. However, there is a trailing newline inserted after the original listing content, which disrupts the layout on slides with many listings. I have fixed this by changing the inner ##1^^J% to ##1% in the definition of \LS@hack@iii. Is this newline necessary for any purpose? Jun 22, 2011 at 10:01
• I don't think it is necessary. I just thought that listings would need it (the previous ^^J is very important). I'll just remove it, then. Jun 22, 2011 at 12:52
• Still testing, no show-stoppers so far :-) For the few cases where the first line is more indented than some other lines I have defined a new ungobble=<num> listing key that alters the value of already extracted gobble parameter by subtracting its value: \lst@Key{ungobble}{0}{\edef\lst@gobble{\the\numexpr\lst@gobble-#1}} Jun 22, 2011 at 19:40

IMHO such a feature should be implemented using the internals of listings itself, e.g. by using the same catcodes etc., to maximize the compatibility with the normal operations of the environment.

The listings package uses an internal macro \lstenv@Process to process the lstlisting and other listing environments defined by \lstnewenvironment. This macro is used after the environment arguments are processed and could be locally redefined to count the spaces (then active!) after the first line break (^^M, the one after \begin{lstlisting}[options]), then set the gobble option with the right number and call the original \lstenv@Process. That macro actually calls itself recursively, so its original definition must be also restored.

Here now my solution as package. It adds the above functionality as boolean option autogobble, which can be used like any other option with all listing environments defined by listings, including custom ones. A test file to show the proper function of it also follows.

### MWE (based on original MWE):

\documentclass{beamer}
\usepackage{listings}
\usepackage{lstautogobble}

\lstnewenvironment{C}[1][] {\lstset{language=C, basicstyle=\scriptsize\ttfamily, autogobble, #1}}{}

\begin{document}

\begin{frame}[fragile]{A simple truth\ldots}
\noindent Normal text as reference
\begin{C}
#include <stdio.h>
int main(){
printf("tex.stackexchange.com: the coolest community ever!\n");
}
\end{C}
\end{frame}

\end{document}


### Package:

(v1.1 2012/02/04: Now with support for tabulators)

% LaTeX Package lstautogobble
% Counts the leading spaces of the first line and sets gobble to this number
% Copyright (c) 2011 by Martin Scharrer <martin@scharrer-online.de>
% This is free software under the LPPL v1.3c or later and was also posted under the CC BY-SA 3.0 license.
\ProvidesPackage{lstautogobble}[2012/02/04 v1.1 Implements 'autogobble' option for 'listings']

% This is an add-on to the listings package
\RequirePackage{listings}

% Counter for leading spaces
\newcount\lstag@spacecount

% Some macros for comparison:
\def\lstag@activespace{\lst@ProcessSpace}%  Definition of an active space
\def\lstag@tabulator{\lst@ProcessTabulator}%  Definition of an tabulator

\begingroup
\catcode\^^M=\active%
\gdef\lstag@activenl{^^M}%  Active CR (ASCII 13) character which is used as line break
\endgroup

% Define autogobble option as boolean (by default off)
\lst@Key{autogobble}{false}[t]{\lstKV@SetIf{#1}\lst@ifautogobble}

% ungobble option
\lst@Key{ungobble}{0}{\def\lst@ungobble{#1}}

% Insert required code at environment init

% Autogobble init macro.
% If the option is active and gobble is not set, init vars and overwrite the process macro with own definition.
\def\lst@autogobble{%
\lst@ifautogobble
\ifnum\lst@gobble>0\else
\def\lst@gobble{\lstag@gobble}%
\def\lstag@gobble{0}%
\lstag@spacecount\z@
\def\lstag@spaceaccu{}%
\let\lstag@restofline\empty
\let\lstag@origlstenv@Process\lstenv@Process
\fi
\fi
}

% Checks if the next following character (read as argument) is a line break (as it is supposed to be)
% Otherwise there is some text direct after the \begin{<env>}[<options>] which is dropped by listings.
\expandafter\ifx\lstag@activenl#1\relax
\else
\def\lstag@restofline{Dummy replacement of text after begin of listing to trigger original warning message}%
\fi
}

% After the new line is found this macro counts the spaces and tabulators
\ifx\lstag@activespace#1\relax
% Accumulate spaces (i.e. their definitions) for later re-insertion:
\expandafter\def\expandafter\lstag@spaceaccu\expandafter{\lstag@spaceaccu\lst@ProcessSpace}%
\else% Character wasn't a space
\ifx\lstag@tabulator#1\relax
% Accumulate spaces (i.e. their definitions) for later re-insertion:
\@tempcnta=\lst@tabsize\relax
\loop
\ifnum\@tempcnta>\z@
\expandafter\def\expandafter\lstag@spaceaccu\expandafter{\lstag@spaceaccu\lst@ProcessSpace}%
\repeat
\else% Character wasn't a tabulator either
% Set gobble option (indirect):
\xdef\lstag@gobble{\the\numexpr\lstag@spacecount-\lst@ungobble\relax}%
% Restore original definition of process macro:
\global\let\lstenv@Process\lstag@origlstenv@Process
% Re-insert all collected material or appropriate replacement material:
\edef\next{\noexpand\lstenv@Process\lstag@restofline\expandafter\noexpand\lstag@activenl\expandafter\unexpanded\expandafter{\lstag@spaceaccu}\noexpand#1}%
\fi\fi
\next
}

\endinput


### Test File:

\documentclass{article}

\usepackage{listings}
\usepackage{lstautogobble}

\lstnewenvironment{test}[1][]{%
\lstset{basicstyle=\ttfamily,autogobble=true,#1}%
}{}

\parindent=0pt% for test only
\begin{document}

Normal text as reference

\hrule

Only environment
\begin{test}
test
it
\end{test}

With options
\begin{test}[basicstyle=\ttfamily\scriptsize]
test
it
\end{test}

Manual gobble option (override)
\begin{test}[gobble=7]
test
it
\end{test}

Autogobble off:
\begin{test}[autogobble=false]
test
it
\end{test}

With some material on the same line as \texttt{\string\begin} (dropped by listings. The warning message got preserved)
\begin{test} some text at the first line
test
it
\end{test}

dito with optional argument
\begin{test}[] some text at the first line
test
it
\end{test}

Different indention levels:
\begin{test}
test
it
\end{test}

\begin{test}
test
it
\end{test}

\begin{test}
test
it
\end{test}

\begin{test}
test
it
\end{test}

\begin{test}
test
it
\end{test}

\hrule

Some real C Code
\begin{lstlisting}[autogobble]
#include <stdio.h>
int main(){
printf("tex.stackexchange.com: the coolest community ever!\n");
}
\end{lstlisting}

\end{document}


### Test Results:

• @Martin: good approach as well. I thought of it, but (1) I was too lazy to probe the internals of listings and (2) I wanted to maximize compatibility with other verbatim environments which may use different catcodes than listings. This second point would have made more sense if I had implemented the gobble option myself, as in the LuaTeX solution given by Aditya. Jun 25, 2011 at 22:20
• Martin, that is an impressive solution that should go to CTAN, eventually. I am especially impressed by the little amount of code you needed for this. I tried to look into (and understand the internals of) the listings package before submitting my question, but did not get very far. Jun 26, 2011 at 20:56
• @someonr: Have a look at the updated version and tell me if it works fine with you. Feb 4, 2012 at 10:27
• @MartinScharrer is there a version of lstautogobble that works for \lstinputlisting? Google is failing me for this one for some reason.
– Luke
Jun 21, 2013 at 19:19
• @MartinScharrer This looks very nice! One question: listings's built-in gobble only works in environment listings, and not in lstinputlistings. Is there any chance that autogobble can handle listings whose content comes from an outside file, and with the linerange features? It would be very useful to be able to include snippets from actual source (where indentation is to be expected). Would this require lots more work? (I don't know why the stock gobble doesn't work for input listings.) Jul 16, 2013 at 21:53

This is relatively straight forward to do in LuaTeX. The ConTeXt wiki has this exact question as an example of programming in luatex. The solution consists of two parts:

• capturing the content of the environment: in ConTeXt, this is done using buffers. I think that one of the many LaTeX verbatim packages provide a similar functionality.

• remove the leading spaces from the content: This is easy in Lua. The complexity of the solution depends on how robust you want the implementation to be (what should happen if the source if badly formatted, i.e., later lines have less white space than the first line. Here is the dedenting function from the ConTeXt wiki


\startluacode
-- Initialize a userdata name space to keep our own functions in.
-- That way, we won't interfere with anything ConTeXt keeps in
-- the global name space.
userdata = userdata or {}

function userdata.dedentedtyping(content)
local lines    = string.splitlines(content)
local indent   = string.match(lines[1], '^ +') or ''
local pattern  = '^' .. indent
for i=1,#lines do
lines[i] = string.gsub(lines[i],pattern,"")
end

content = table.concat(lines,'\n')

tex.sprint("\\starttyping\n" .. content .. "\\stoptyping\n")

end
\stopluacode


For listings, instead of passing \starttyping ... \stoptyping back to TeX, you can pass \begin{listing} ... \end{listing}

• +1 for another example on the advantages of an embedded "real" programming language into TeX. Thanks for sharing this! Jun 22, 2011 at 7:16

I'm cheating: using the file that beamer produces (\jobname.vrb) for fragile frames, and counting spaces between some extra command (\setgobble) and the following \begin. This is limited to one \setgobble per frame, and to values of gobbling at most 8 (or 10?).

I think that it would be better to count spaces within the listings environment somehow, and put its input back to what it expects.

\documentclass{beamer}
\usepackage{listings}

\lstnewenvironment{C}[1][] {\lstset{language=C, basicstyle=\scriptsize\ttfamily, #1}}{}

\makeatletter
\newcount\gobblecount
\newcommand{\setgobble}
{%
\begingroup
\everyeof{\@nnil}%
\obeyspaces
\expandafter\setgobble@aux\@@input\jobname.vrb%
\endgroup
}
\newcommand\@ninthofnine[9]{#9}
\def\setgobble@aux#1\setgobble#2\begin{%
\global\gobblecount
\expandafter\@car\@ninthofnine #2876543210\@nil\relax
\remove@to@nnil
}
\makeatother

\begin{document}

\begin{frame}[fragile]{A simple truth\ldots}
\setgobble
\begin{C}[gobble=\the\gobblecount] % <-- ** This is annoying! **
#include <stdio.h>
int main(){
printf("tex.stackexchange.com: the coolest community ever!\n");
}
\end{C}
\end{frame}

\end{document}

• +1 The limitations make this solution somewhat impractical in real settings, but the approach is pretty interesting. Jun 6, 2011 at 8:55
• Thinking a bit more about this, I note that \the\inputlineno holds the line of \jobname.vrb at which we are (try [gobble=\the\inputlineno] ;-). Then \setgobble could throw this number of lines of the vrb file, and check the amount of whitespace at the start of the next one. Jun 6, 2011 at 9:29