Line break long verbatim text with highlighted background

Question

I created an MWE-look-alike in the original document then 'background highlighted' (somehow 'highlight' means something else in verbatim) some text to show where/what to focus on. This MWE-look-alike is targetted at those not familiar at all with LaTeX.

The verbatim text was produced using the listings package, and the highlight was produced using custom code from here: https://tex.stackexchange.com/a/49309, tikz, and xcolor. The problem with this custom code, however, is line breaks.

The MWE-look-alike in the original document looks something like this at 100% zoom:

In the screenshot I highlighted four lines:

• Line 1: Whole line.
• Line 2: Partial line.
• Line 3: Code with empty space before the highlight.
• Line 4: Code inline with text.

Line 1 and 2 are OK. Line 3 and 4 are problematic.

Line 3 and 4 are too long that they extend beyond the margin. They cannot respect the margin because they cannot line break. Line 3 also does not respect the empty space before the highlighted text.

My question is:

1. Can anyone help unhighlight the empty space before the highlighted text in Line 3?
2. Can anyone help with the long lines/ line breaks problem for this particular code?

It has been many years since the custom code was written, so I wonder if anyone is(/has been) able to solve this line break problem. I can't do it because I don't know how.

The reason I chose this custom code is:

• It preserves the 'language highlight' (whereas \colorbox removes the language highlight),
• I don't have to change the backslash character "\" to {\char92}, and
• It's cleaner to use because the customizing is done in the preamble (which is less cluttery than customizing in the verbatim text itself).

I really want this custom code to work because it's really good. Can anyone help improve the code, please.

This is the MWE:

\documentclass{scrbook}

\usepackage{tikz}

\usepackage{listings}
 \lstset{%
    language=[LaTeX]TeX,
    commentstyle=\color{gray},
    breaklines=true,
    frame=single,
    frameround=tttt,%
    framextopmargin=10pt,
    framexbottommargin=7pt,
    xleftmargin=22pt,
    framexleftmargin=10pt,
    framesep=10pt
}

% - - - - - - - - - - - - - % Code for highlighting.

\makeatletter%
\newenvironment{btHighlight}[1][]%
{\begingroup\tikzset{bt@Highlight@par/.style={#1}}\begin{lrbox}{\@tempboxa}}%
{\end{lrbox}\bt@HL@box[bt@Highlight@par]{\@tempboxa}\endgroup}

\newcommand\btHL[1][]{%
  \begin{btHighlight}[#1]\bgroup\aftergroup\bt@HL@endenv%
}
\def\bt@HL@endenv{%
  \end{btHighlight}%   
  \egroup%
}
\newcommand{\bt@HL@box}[2][]{%
  \tikz[#1]{%
    \pgfpathrectangle{\pgfpoint{1pt}{0pt}}{\pgfpoint{\wd #2}{\ht #2}}%
    \pgfusepath{use as bounding box}%
    \node[anchor=base west, fill={oposmy!30},outer sep=0pt,inner xsep=1pt, inner ysep=0pt, minimum height=\ht\strutbox+1pt,#1]{\raisebox{1pt}{\strut}\strut\usebox{#2}};%
  }%
}%
\makeatother%

\lstdefinestyle{SQL}{%
    moredelim=**[is][\btHL]{`}{`},%
    moredelim=**[is][{\btHL[fill=green!30,draw=red,dashed,thin]}]{@}{@}%
}

% - - - - - - - - - - - - - %

\usepackage{xcolor}
\definecolor{oposmy}{HTML}{c0f1ff}

\begin{document}

\begin{lstlisting}[style=SQL]
\documentclass{scrbook}

`\usepackage{fontspec`}% #1
\setmainfont{`Latin Modern Sans Demi Cond`}% #2

\usepackage{tikz}
\usetikzlibrary{shapes.misc}

\begin{document}

\begin{tikzpicture}
   `\node[draw,rounded rectangle, rounded rectangle arc length=180] {Text};`% #3
\end{tikzpicture}

Press the `F1` key for help. Press the `\tikz[baseline] \draw node[anchor=base,draw,double,rounded corners] {{\texttt{F11}}};` key for full screen.% #4

\end{document}
\end{lstlisting}

\end{document}

Answer 1

The problem with the extra spaces being highlighted is quite easy to solve by setting the listings option keepspaces=true. The documentation mentions for this option

keepspaces=true tells the package not to drop spaces to fix column alignment and always converts tabulators to spaces.

With that option the indentation sometimes changes a bit when you have a linebreak in the listings, but the result is otherwise almost identical. So I presume it's not a big deal to use this option for all your listings.

Breaking the highlighted lines is trickier. listings comes with several hooks you can use to insert extra code at the begin or end of input lines, empty lines etc. Unfortunately, there doesn't seem to be a way to reliably detect the begin and end of each output line that also takes line breaks into account. This is because internally the standard TeX mechanism for breaking lines in conjunction with \discretionary is used. As TeX executes the text in all discreationary breaks, we also cannot use them to let TeX figure out the finally chosen break to find the end of the line and the start of the next one.

So the approach chosen here is to break the highlighting at every \discretionary and start anew after it. This will give a lot of small highlight pieces, glued together to a visibly continuous one.

For the implementation, I had to switch to the highlighting approach that uses TikZ' remember picture feature as it doesn't require to wrap a box around the highlighted part, which caused trouble with getting properly balanced braces right in my approach. As long as you only need a simple background color, this approach is fine. If you need to draw boxes or similar, the implementation will become much more difficult.

The extensions are a \hlstyle and \hlstyleend macro which just sets the global \hl@active switch to determine if highlighting is active, and the actual discretionary break redefinition. It's not necessary to redefine \discrtionary itself because listings has its own version \lst@discretionary which can be hijacked for this purpose.

Edit: I noticed that the earlier solution breaks if a highlight spans not only across two lines but also across two pages. The modified version fixes this by defining commands \hl@p@XXX, where XXX is the current highlight marker number, for each start of a highlight. Each of these commands maps to the corresponding page number the marker occurs on. As the page number information is needed on the next compilation run, all commands are written to the .aux file when the shipout for that page occurs and the exact page number is known, and are used on the next run to determine what highlight segments should be printed on the current page only.

The full example code looks like this:

\documentclass{scrbook}

\usepackage{atbegshi,ifthen,listings,tikz}
\usepackage{xcolor}

\definecolor{oposmy}{HTML}{C0F1FF}
\tikzstyle{highlighter} = [ oposmy, line width = \baselineskip ]

\makeatletter

\newcounter{highlight}
\newif\ifhl@active

\newcommand{\tikzhighlightanchor}[1]{%
    \ensuremath{\vcenter{\hbox{%
        \tikz[remember picture, overlay]{\coordinate (#1 highlight \arabic{highlight});}%
    }}}%
}
\newcommand{\bh}{%
    \stepcounter{highlight}%
    \edef\hl@temp{%
        \unexpanded{\noexpand\expandafter\gdef\noexpand\csname}%
        hl@p@\arabic{highlight}%
        \unexpanded{\noexpand\endcsname{\arabic{page}}}
    }%
    \expandafter\write\expandafter\@auxout\expandafter{\hl@temp}%
    \tikzhighlightanchor{begin}%
}
\newcommand{\eh}{%
    \tikzhighlightanchor{end}%
}
\newcommand{\hlstyle}{%
    \global\hl@activetrue
    \bh
    \aftergroup\hlstyleend
}
\newcommand{\hlstyleend}{%
    \eh
    \global\hl@activefalse
}

\AtBeginShipout{\AtBeginShipoutUpperLeft{%
    \ifthenelse{\value{highlight} > 0}{%
        \tikz[remember picture, overlay]{\foreach \stroke in {1,...,\arabic{highlight}} {%
            \ifcsname hl@p@\stroke\endcsname
                \ifthenelse{\csname hl@p@\stroke\endcsname=\arabic{page}}{\draw[highlighter]
                    (begin highlight \stroke) -- (end highlight \stroke);}{}%
            \fi
        }}%
    }{}%
}}

\usepackage{listings}
 \lstset{%
    language=[LaTeX]TeX,
    commentstyle=\color{gray},
    breaklines=true,
    frame=single,
    frameround=tttt,%
    framextopmargin=10pt,
    framexbottommargin=7pt,
    xleftmargin=22pt,
    framexleftmargin=10pt,
    framesep=10pt,
    keepspaces=true
}

\lstdefinestyle{SQL}{%
    moredelim=**[is][\hlstyle]{`}{`},%
%    moredelim=**[is][{\btHL[fill=green!30,draw=red,dashed,thin]}]{@}{@}%
}

\let\orig@lst@discretionary=\lst@discretionary
\gdef\lst@discretionary{%
    \ifhl@active \eh \fi
    \orig@lst@discretionary
    \ifhl@active \bh \fi
}

\makeatother

\begin{document}

\vspace*{12cm}

\begin{lstlisting}[style=SQL]
\documentclass{scrbook}

`\usepackage{fontspec`}% #1
\setmainfont{`Latin Modern Sans Demi Cond`}% #2

\usepackage{tikz}
\usetikzlibrary{shapes.misc}

\begin{document}

\begin{tikzpicture}
   `\node[draw,rounded rectangle, rounded rectangle arc length=180] {Text};`% #3
\end{tikzpicture}

Press the `F1` key for help. Press the `\tikz[baseline] \draw node[anchor=base,draw,double,rounded corners] {{\texttt{F11}}};` key for full screen.% #4
Press the `F1` key for help.

\end{document}
\end{lstlisting}

\end{document}

---

One downside of this approach is that a lot of TikZ markers have to be processed which notably slows down compilation when many/long highlights are used. Also note that you have to compile twice to get the highlight positions right.