# Spacing issue with literate in listings

I answered this question and OP showed me a problem with the answer. The code:

\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{listings}

\lstset{
% keepspaces, % Apparently this works...
literate=
{à}{{\a}}1
}

\begin{document}

\begin{lstlisting}[mathescape=true]
Adicionar $v$ à lista.
Adicionar $v$ onde?
R: à lista.
Yoda style: à lista, $v$ adicionar.
Adicionar $v$ à lista e à outra lista.
\end{lstlisting}

\end{document}


prints:

and the space after the à is eaten.

From the four examples above, I concluded that the problem is isolated at the first literated character in lines after mathescaped text.

• The second and sixth lines show the issue.
• The fourth line shows that the effect from the third line stops after the line break.
• The fifth line shows that the effect starts only after the mathescaped text
• The sixth line shows that the problem does not propagate to the following literated characters.

Using the keepspaces option apparently solves the problem.

From the look of it I think it's a bug. How to fix this?

I also concluded that v was definitely added to the list :)

(August 12, 2018—it's Sunday and I don't have the nerve for delving deeply into the source code of the listings package.)

But when skimming over that source code, I got the impression that

1. the macro underlying the replacement-process for the things provided via the literate key is called \lst@Literate.

2. there is a switch \lst@ifwhitespace which indicates whether the last processed thingie has been a space which has been printed as a whitespace.

3. the code of \lst@Escape\lst@Escape seems to underlie all those thingies for escaping to LaTeX—does set that switch to false in case of being used due to mathescape being true. So with mathescape=true the switch is false after $...$. This leads to processing the space following $...$ as whitespace which is not to be discarded but to be printed. This processing of the next space in turn leads to setting the switch to true again. Now the literate-thingie is processed. The underlying macro \lst@Literate does not set the switch to false. Thus due to the space before the literate-thingie being processesd as printed whitespace, the "machinery" after processing the literate-thingie still "thinks" that it has just processed and printed a whitespace and thus should discard following spaces.

This "thinking" of the machinery seems wrong in cases where the literate-thingie did as last item of output produce something other than whitespace .

Perhaps \lst@Literate should be redefined to set that switch to false.

Could look like this:

\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{listings}

\lstset{
literate=
{à}{{\a}}1
}

\makeatletter
\def\lst@Literate#1#2#3{%
\ifx\relax#2\@empty\else
\lst@CArgX #1\relax\lst@CDef
{}
{\let\lst@next\@empty
\lst@ifxliterate
\lst@ifmode \let\lst@next\lst@CArgEmpty \fi
\fi
\ifx\lst@next\@empty
\ifx\lst@OutputBox\@gobble\else
\lst@XPrintToken \let\lst@scanmode\lst@scan@m
\lst@token{#2}\lst@length#3\relax
\lst@XPrintToken
\lst@whitespacefalse %!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
\fi
\let\lst@next\lst@CArgEmptyGobble
\fi
\lst@next}%
\@empty
\expandafter\lst@Literate
\fi}
\makeatother

\begin{document}

\begin{lstlisting}[mathescape=true]
Adicionar $v$ à lista.
Adicionar $v$ onde?
R: à lista.
Yoda style: à lista, $v$ adicionar.
Adicionar $v$ à lista e à outra lista.
\end{lstlisting}

\end{document}


(Addition in August 18, 2018—it's Saturday and I still don't have the nerve for delving deeply into the source code of the listings package. ;-) )

But when doing so, we loose the possibility of having the literate-mechanism produce material whereafter a following space-token does not lead to producing more whitespace.

Thus perhaps it is better to just set the \lst@whitespace-switch to false within the literate-directive whenever that directive is used for having LaTeX typeset non-whitespace.

Could look like this:

\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{listings}

\makeatletter
\lstset{
literate=
{à}{{\a\lst@whitespacefalse}}1
}
\makeatother

\begin{document}

\begin{lstlisting}[mathescape=true]
Adicionar $v$ à lista.
Adicionar $v$ onde?
R: à lista.
Yoda style: à lista, $v$ adicionar.
Adicionar $v$ à lista e à outra lista.
\end{lstlisting}

\end{document}


I can absolutely not guarantee that my analysis of the situation is correct.
Maybe it is a starting point.

Seems that things work out correctly when

• either adding \lst@whitespacefalse to the definition of \lst@Literate

• or applying \lst@whitespacefalse inside the literate-directive.

Again: I can absolutely not guarantee that my analysis of the situation is correct.
Maybe it is a starting point.

In case setting \lst@whitespacefalse inside the literate-directive is to be considered good practice, both this should be mentioned somewhere in the manual, and for the end-user there should be a possibility of setting that switch without using \makeatletter..\makeatother beforehand.

In case you come to the same conclusion, feel free to hand in a bug report. ;-)

• For a Sunday, your answer is impressive! I tried (not very hard, I confess) to decipher this before asking the question, but failed... miserably. The code of listings` is quite a thing to understand. Recently I mailed Jobst Hoffmann about this issue and he answered. I'll try my luck again. Thanks a lot for your answer :) – Phelype Oleinik Aug 13 '18 at 12:46
• I have no idea what's going on, but you're contribution surely helped me! – Hugo Raguet Dec 9 '18 at 22:19