6

This question is related to my previous question answered by Jubobs here. I compiled the given code below with both TeX Live 2013 and TeX Live 2014 pretest. Both produce the same result.

Case 1

The expected result with autounindent is as follows.

enter image description here

Case 2

When I load hyperref, the autounindent no longer works as follows.

enter image description here

Case 3

The worst case happens when I load both hyperref and hypcap, I got some errors as follows.

enter image description here

MWE

\documentclass[12pt]{article}

%TODO
% - handle tabs

\usepackage{accsupp}
\newcommand*{\noaccsupp}[1]{\BeginAccSupp{ActualText={}}#1\EndAccSupp{}}

\usepackage{xcolor}
\usepackage{listings}

\makeatletter

% -------- ugly details --------

% custom key
\lst@Key{autounindent}f[t]{\lstKV@SetIf{#1}\lst@ifautounindent}

% --- accumulation of tokens ---
\toks@={
    \lst@DefSaveDef{`\ }\lsts@myspace{\processmyspace}
    %\lst@DefSaveDef{`\^^I}\lsts@mytab{\processmytab}
}


% --- egreg's helper function for accumulating tokens ---
\def\add@savedef#1#2{%
  \begingroup\lccode`?=#1\relax
  \lowercase{\endgroup
  \edef\@temp{%
    \noexpand\lst@DefSaveDef{\number#1}%
    \expandafter\noexpand\csname lsts@?\endcsname{\noexpand#2% % I had to swap things around to fix a bug here;
      \expandafter\noexpand\csname lsts@?\endcsname}%          % otherwise, comment highlighting would be broken.
  }}%
  \toks@=\expandafter{\the\expandafter\toks@\@temp}%
}

\count@=33
\loop
    \add@savedef\count@\processchar
  \ifnum\count@<127
  \advance\count@\@ne
\repeat

% --- to keep track of the state ---
\newcount\spacesToGobble
\newcount\spacesGobbledSoFar
\newif\ifafterindent
\newif\ifonBasisLine
\newif\ifPostponeCountToNextLine


\lst@AddToHook{Init}
{%
  \lst@ifautounindent%
    % patch hook macros
    \let\@ddedToInitVarsBOLhook\@@ddedToInitVarsBOLhook%
    \let\@ddedToEOLhook\@@ddedToEOLhook%
    % initialise things
    \global\onBasisLinetrue%
    \global\afterindentfalse%
    \global\spacesToGobble=0%
    \lst@ifincluderangemarker%
      \ifnum9999999=\lst@firstline% (i.e. if firstline option not used)
        \global\PostponeCountToNextLinetrue%
      \fi
    \fi
  \fi
}


\lst@AddToHook{SelectCharTable}
{%
    \lst@ifautounindent%
        \lst@lAddTo\lst@DeveloperSCT{\the\toks@}%
    \fi
}


\lst@AddToHook{InitVarsBOL}{\@ddedToInitVarsBOLhook}
\newcommand\@ddedToInitVarsBOLhook{}
\newcommand\@@ddedToInitVarsBOLhook
{%
  \global\afterindentfalse%
}

\lst@AddToHook{EOL}{\@ddedToEOLhook}
\newcommand\@ddedToEOLhook{}
\newcommand\@@ddedToEOLhook
{%
  \ifonBasisLine%
    \ifPostponeCountToNextLine%
    \else
      \global\onBasisLinefalse%
    \fi
  \fi
  \global\spacesGobbledSoFar=0%
  \global\PostponeCountToNextLinefalse%
}

\lst@AddToHook{DeInit}
{%
  \lst@ifautounindent%
    % undo patches
    \def\@ddedToInitVarsBOLhook{}%
    \def\@ddedToEOLhook{}%
  \fi
}


% --- helper macros ---
\newcommand\processchar
{%
  \global\afterindenttrue%
}

\newcommand\processmyspace
{%
  \ifafterindent%
    \lsts@myspace%
  \else
    \ifonBasisLine%
      \global\advance\spacesToGobble\@ne\relax%
    \else
      \ifnum\spacesGobbledSoFar<\spacesToGobble\relax%
        \global\advance\spacesGobbledSoFar\@ne\relax%
      \else
        \lsts@myspace%
      \fi
    \fi
  \fi
}

\makeatother

% ---------- end of ugly details ----------

\usepackage{filecontents}
\begin{filecontents*}{Program.cs}
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace Delegate
{
    class Program 
    {
        // start
        sdfgsd fgdsfg  static void Main(string[] args)
        {
            for (int x = 0; x < 10; x++)
                Console.WriteLine(x);
        }
        /* sdflkjhsdf */
        // stop
    }
}
\end{filecontents*}

% --- general style definition ---
\lstdefinestyle{Common}
{   
    language={[Sharp]C},
    numbers=left,
    numbersep=1em,
    numberstyle=\tiny\noaccsupp,
    frame=single,
    framesep=\fboxsep,
    framerule=\fboxrule,
    rulecolor=\color{red},
    xleftmargin=\dimexpr\fboxsep+\fboxrule,
    xrightmargin=\dimexpr\fboxsep+\fboxrule,
    breaklines=true,
    breakindent=0pt,
    tabsize=2,
    columns=flexible,
    %includerangemarker=false,
    rangeprefix=//\ ,
}

\lstdefinestyle{A}
{
    style=Common,
    backgroundcolor=\color{yellow!10},
    basicstyle=\scriptsize\ttfamily,
    keywordstyle=\color{blue}\bf,
    identifierstyle=\color{black},
    stringstyle=\color{red},
    commentstyle=\color{green},
}

%\usepackage[colorlinks,bookmarksnumbered,bookmarksopen]{hyperref} % makes  autounindent no longer work
%\usepackage[all]{hypcap}% causes errors.

\begin{document}


\section*{Using \texttt{linerange} with \texttt{autounindent}}
\lstinputlisting[style=A,linerange=start-stop,autounindent]{Program.cs}


\end{document}

Questions

How to fix these issues?

  • I've managed to isolate the bug (it has do to with my modification of the character table), but I don't know how to solve it. I hope Heiko will see your question and chime in... – jub0bs Jun 11 '14 at 20:11
5
+50

The problem is not the package order. The code that patches listings uses the token register \toks@, a temporary scratch register, as permanent token register. Thus it happens that the token register \toks@ is also used by other packages like hyperref or hypcap and that they can leave garbage afterwards. Then this garbage is called inside lstlisting by the "autounindent" feature causing unwanted surprising effects like error messages.

\toks@ is a temporary scratch token register, which can be used by any package or command. And if the register is cleared before usage, it will not help either, because the settings of the token register in the preamble would be lost for the "autounindent" feature.

The following fix replaces \toks@ by a fresh allocated token register:

\documentclass[12pt]{article}

%TODO
% - handle tabs

\usepackage{accsupp}
\newcommand*{\noaccsupp}[1]{\BeginAccSupp{ActualText={}}#1\EndAccSupp{}}

\usepackage{xcolor}
\usepackage{listings}

\makeatletter

% -------- ugly details --------

\newtoks\mylsttoks

% custom key
\lst@Key{autounindent}f[t]{\lstKV@SetIf{#1}\lst@ifautounindent}

% --- accumulation of tokens ---
\mylsttoks={
    \lst@DefSaveDef{`\ }\lsts@myspace{\processmyspace}
    %\lst@DefSaveDef{`\^^I}\lsts@mytab{\processmytab}
}

% --- egreg's helper function for accumulating tokens ---
\def\add@savedef#1#2{%
  \begingroup\lccode`?=#1\relax
  \lowercase{\endgroup
  \edef\@temp{%
    \noexpand\lst@DefSaveDef{\number#1}%
    \expandafter\noexpand\csname lsts@?\endcsname{\noexpand#2%
      % I had to swap things around to fix a bug here;
      \expandafter\noexpand\csname lsts@?\endcsname}%
      % otherwise, comment highlighting would be broken.
  }}%
  \mylsttoks=\expandafter{\the\expandafter\mylsttoks\@temp}%
}

\count@=33
\loop
    \add@savedef\count@\processchar
  \ifnum\count@<127
  \advance\count@\@ne
\repeat

% --- to keep track of the state ---
\newcount\spacesToGobble
\newcount\spacesGobbledSoFar
\newif\ifafterindent
\newif\ifonBasisLine
\newif\ifPostponeCountToNextLine


\lst@AddToHook{Init}
{%
  \lst@ifautounindent%
    % patch hook macros
    \let\@ddedToInitVarsBOLhook\@@ddedToInitVarsBOLhook%
    \let\@ddedToEOLhook\@@ddedToEOLhook%
    % initialise things
    \global\onBasisLinetrue%
    \global\afterindentfalse%
    \global\spacesToGobble=0%
    \lst@ifincluderangemarker%
      \ifnum9999999=\lst@firstline% (i.e. if firstline option not used)
        \global\PostponeCountToNextLinetrue%
      \fi
    \fi
  \fi
}


\lst@AddToHook{SelectCharTable}
{%
    \lst@ifautounindent%
        \lst@lAddTo\lst@DeveloperSCT{\the\mylsttoks}%
    \fi
}


\lst@AddToHook{InitVarsBOL}{\@ddedToInitVarsBOLhook}
\newcommand\@ddedToInitVarsBOLhook{}
\newcommand\@@ddedToInitVarsBOLhook
{%
  \global\afterindentfalse%
}

\lst@AddToHook{EOL}{\@ddedToEOLhook}
\newcommand\@ddedToEOLhook{}
\newcommand\@@ddedToEOLhook
{%
  \ifonBasisLine%
    \ifPostponeCountToNextLine%
    \else
      \global\onBasisLinefalse%
    \fi
  \fi
  \global\spacesGobbledSoFar=0%
  \global\PostponeCountToNextLinefalse%
}

\lst@AddToHook{DeInit}
{%
  \lst@ifautounindent%
    % undo patches
    \def\@ddedToInitVarsBOLhook{}%
    \def\@ddedToEOLhook{}%
  \fi
}


% --- helper macros ---
\newcommand\processchar
{%
  \global\afterindenttrue%
}

\newcommand\processmyspace
{%
  \ifafterindent%
    \lsts@myspace%
  \else
    \ifonBasisLine%
      \global\advance\spacesToGobble\@ne\relax%
    \else
      \ifnum\spacesGobbledSoFar<\spacesToGobble\relax%
        \global\advance\spacesGobbledSoFar\@ne\relax%
      \else
        \lsts@myspace%
      \fi
    \fi
  \fi
}

\makeatother

% ---------- end of ugly details ----------

\usepackage{filecontents}
\begin{filecontents*}{Program.cs}
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace Delegate
{
    class Program 
    {
        // start
        sdfgsd fgdsfg  static void Main(string[] args)
        {
            for (int x = 0; x < 10; x++)
                Console.WriteLine(x);
        }
        /* sdflkjhsdf */
        // stop
    }
}
\end{filecontents*}

% --- general style definition ---
\lstdefinestyle{Common}
{   
    language={[Sharp]C},
    numbers=left,
    numbersep=1em,
    numberstyle=\tiny\noaccsupp,
    frame=single,
    framesep=\fboxsep,
    framerule=\fboxrule,
    rulecolor=\color{red},
    xleftmargin=\dimexpr\fboxsep+\fboxrule,
    xrightmargin=\dimexpr\fboxsep+\fboxrule,
    breaklines=true,
    breakindent=0pt,
    tabsize=2,
    columns=flexible,
    %includerangemarker=false,
    rangeprefix=//\ ,
}

\lstdefinestyle{A}
{
    style=Common,
    backgroundcolor=\color{yellow!10},
    basicstyle=\scriptsize\ttfamily,
    keywordstyle=\color{blue}\bf,
    identifierstyle=\color{black},
    stringstyle=\color{red},
    commentstyle=\color{green},
}

\usepackage[colorlinks,bookmarksnumbered,bookmarksopen]{hyperref}
\usepackage[all]{hypcap}

\begin{document}
  \section*{Using \texttt{linerange} with \texttt{autounindent}}
  \lstinputlisting[style=A,linerange=start-stop,autounindent]{Program.cs}
\end{document}

Result

  • That's great. I had a feeling that \toks@ was the culprit. I think egreg's answer should probably be amended accordingly. What do you think? – jub0bs Jun 14 '14 at 17:12
  • @Jubobs: Yes, I have written a comment to his answer. – Heiko Oberdiek Jun 14 '14 at 18:02
3

Change the order: if you start your document with

\documentclass[12pt]{article}
\usepackage[colorlinks,bookmarksnumbered,bookmarksopen]{hyperref}
\usepackage[all]{hypcap}

and leave the rest as it is, you should get the expected result.

  • But those packages must be loaded last, according to their manual. :D – kiss my armpit Jun 11 '14 at 15:57
  • Yep, I was trying to find where it was mentionned in the hyperref manual. I agree this solution can mess up in a larger document, but for your MWE, it works like a charm. – Clément Jun 11 '14 at 15:59
  • @FifaEarthCup2014 (your nickname chages every day !) : could you specify why this "charming" solution is not enough ? – Clément Jun 14 '14 at 16:25
  • 1
    Ok, Heiko Oberdiek made it clear that the problem was deeper… – Clément Jun 14 '14 at 17:02
  • It is of course not enough for the case where the order does matter. – kiss my armpit Jun 14 '14 at 19:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.