How can I preserve all whitespace in an environment?

Question

I'm typesetting literate programs; they look like this:

\documentclass{article}

\usepackage{fontspec}
\newfontfamily{\Symbola}{Symbola}
\newfontfamily{\UbuntuMono}{Ubuntu Mono}
\DeclareTextFontCommand{\S}{\Symbola}

\usepackage{xcolor}

\newenvironment{code}{\setlength{\parindent}{0pt}\UbuntuMono}{}

\begin{document}
\begin{code}
  (fold\_left (\S{λ} x \textcolor{red}{y}.  x \S{⊕} \textcolor{red}{y})\\
             (a \S{∷} b \S{∷} c \S{∷} d \S{∷} nil))
\end{code}
\end{document}

How can I make sure that the code environment preserves all whitespaces? In particular, that whitespaces should be in the generated PDF too: that is, copy-pasting from the PDF should copy the whitespaces as well.

Here are a few attempts:

• Use \hspace{}: not very nice when looking at the sources, and spaces can't be copied from resulting PDF.
• Replace all spaces by ~, and add \- for beginning-of-line spaces. Sources become ugly (not a show-stopper), but spaces still can't be copied.
• Use \catcode32=12 in definition of code. This works fine alignment-wise, it looks perfect… but spaces still can't be copied.
• Using the listings package: would be great, but it dies on Unicode, and I'm tired of the literate-based workarounds
• Using the fancyvrb package: Has an escapechar property that could be promising, and line breaks being significant in the source are not a deal-breaker, but it seems heavyweight (?). Ideally, I'd like to be able to use this environment inline, too.

What magic do I need to make my environment respect all spaces, and make them copy-pastable from the PDF?

Answer 1

ANSWER REVISITED

When it was pointed out to me that, except in computer modern, my original answer was no different that what the OP was able to achieve with \catcode32=12, I took another look at things. I came up with a wholly satisfactory answer when using pdflatex, and a less satisfying one for xelatex. In both cases, they start by making the space an active character and then redefining it to print a particular character in white.

EDITED to make the space the proper width.

-----pdflatex

Here I use package palatino just to make sure I am not relying on some special aspect of computer modern. Here, the "particular character" I print for a space, in white, is the visible space from the lmodern font.

\documentclass{article}
\usepackage{xcolor}
\usepackage{palatino}
\newsavebox\spacewd
\savebox\spacewd{\texttt{ }}
\newenvironment{code}{\par\catcode32=\active \setlength{\parindent}{0pt}\ttfamily}{\par}
{
\catcode32=\active %
\gdef {\makebox[\wd\spacewd][l]{%
\textcolor{white}{\fontfamily{lmtt}\selectfont\large\smash{\char32}}}}% 
}
\begin{document}
\noindent xyz
\begin{code}
  (fold\_left (\S{λ} x \textcolor{red}{y}.  x \S{⊕} \textcolor{red}{y})\\
             (a \S{∷} b \S{∷} c \S{∷} d \S{∷} nil))
\end{code}
\noindent this is
  a
    test to make    sure all 
      is back to normal
\end{document}

The PDF copy/paste performs as follows:

which is exactly what is desired.

-----xelatex

Here I use libertine font. The only quirk here is that \textvisiblespace takes up less horizontal width than a \ttfamily space and so to avoid the PDF from interpreting the spaced out \textvisiblespaces as a double space, I made the \textvisiblespace that gets inserted both \large and \smashed to avoid PDF interpreting it as an extra gap. Further, I make the \makebox left aligned, so that the large white \textvisiblespace didn't overwrite the prior character.

\documentclass{article}
\usepackage{lmodern}
\usepackage{xcolor}
\usepackage{fontspec,libertine}
\newsavebox\spacewd
\savebox\spacewd{\texttt{ }}
\newenvironment{code}{\par\catcode32=\active \setlength{\parindent}{0pt}\ttfamily}{\par}
{
\catcode32=\active %
\gdef {\makebox[\wd\spacewd][l]{\textcolor{white}{\large\smash{\textvisiblespace}}}}% 
}

\begin{document}
\noindent xyz
\begin{code}
  (fold\_left (\S{λ} x \textcolor{red}{y}.  x \S{⊕} \textcolor{red}{y})\\
             (a \S{∷} b \S{∷} c \S{∷} d \S{∷} nil))
\end{code}
\noindent this is
  a
    test to make    sure all 
      is back to normal
\end{document}

The resulting copy/paste from the PDF is

in which another character is substituted for the space, in this case I had designated \textvisiblespace. I verified that my TeXworks editor can replace the invisible space character with a real space on a global basis.

ORIGINAL ANSWER

This answer is based on my year-old suggestion of using \obeyspaces in the code environment to preserve blank spaces. However, as noted in the comments, it does not detect leading spaces on a line.

To remedy that, I used egreg's answer at Using end-of-line delimiter in plain Tex macro. With it, I am able to intercept the end of line (EOL) inside the code environment. I use this fact to place a \mbox{} at the beginning of each new line inside code, so that the \obeyspaces can detect the leading blank spaces of each subsequent line.

The hard part (for me) was restoring the EOL to its rightful place upon exiting code. I did that with a test on a flag macro \endthis, which gets changed from F to T during the exit code from code.

However, the underlying problem of copy/paste of whitespace from the PDF remains.

\documentclass{article}

%\usepackage{fontspec}
%\newfontfamily{\Symbola}{Symbola}
%\newfontfamily{\UbuntuMono}{Ubuntu Mono}
%\DeclareTextFontCommand{\S}{\Symbola}

\usepackage{xcolor}

\newenvironment{code}
{\par\setlength{\parindent}{0pt}\obeyspaces\ttfamily\gdef\endthis{F}\mymacro}
{\gdef\endthis{T}\par}

\def\mymacro{%
  \if\endthis T\def\next{}\else\def\next{\begingroup\catcode`\^^M=12 \xmymacro}\fi\next}
{\catcode`\^^M=12 %
 \gdef\xmymacro#1^^M{\mbox{}#1\endgroup\mymacro}%
}
\begin{document}
\noindent xyz
\begin{code}
  (fold\_left (\S{λ} x \textcolor{red}{y}.  x \S{⊕} \textcolor{red}{y})\\
             (a \S{∷} b \S{∷} c \S{∷} d \S{∷} nil))
\end{code}
\noindent this is
  a
    test to make    sure all 
      is back to normal
\end{document}

The use of \catcode32=12, which the OP had mentioned, in the definition of code is similar, but it produces "visual spaces"