14

I am trying to input code listings with unicode characters and everything seems to work (more or less) except some unicode characters being displayed out of order. For example instead of

służąca do obsługi

I get

łżąsuca do łobsugi

How can I fix that?

EDIT: I was asked for providing some code, but unfortunately I doubt it will be any helpful. Basically code like this:

\documentclass[11pt,polish]{article}

\usepackage{fontspec}
\usepackage{listings}
\usepackage{xunicode}
\usepackage{xltxtra}
\usepackage{babel}

\setmainfont{Charis SIL} % Some font with good unicode support

\begin{document}
\begin{lstlisting}
służący
\end{lstlisting}
\end{document}

will output this:

łżą

sucy

(yes there is a line break, I don't know where it came from, when I input code in form of a file there are characters out of order, but no line breaks).

4
  • I'm guessing the problem is that characters that don't fit into one byte are moved to the beginning of the word for some reason...
    – zduny
    Commented Nov 6, 2012 at 19:01
  • The display problem occurs in LyX or the final pdf or both?
    – scottkosty
    Commented Nov 7, 2012 at 0:10
  • @scottsky Final pdf. It's listings package problem like others have explained. LyX uses native engines for display in editor, not LaTeX.
    – zduny
    Commented Nov 7, 2012 at 11:09
  • Any new solution since 2012? Has lstlistings been adapted to XeLaTeX?
    – jjmerelo
    Commented Dec 27, 2016 at 21:07

2 Answers 2

10

Package listings is not aware of characters with character code greater than 8-bit. Internally the package redefines each character as active character. The following example allows the support of additional Unicode characters by adding them to the list of extended chars.

Supporting all Unicode characters this way is difficult:

  • High memory usage.
  • Bad run time behavior.
  • There are different classes of characters (\lst@ProcessLetter, \lst@ProcessOther, …)

Example file:

\documentclass[11pt,polish]{article}

\usepackage{fontspec}
\usepackage{listings}
\lstset{extendedchars}

\begingroup
  \catcode0=12 %
  \makeatletter
  \g@addto@macro\lst@DefEC{%
    \lst@CCECUse\lst@ProcessLetter
    łżą% *** add Unicode characters ***
    ^^00% end marker
  }%
\endgroup

\begin{document}
\begin{lstlisting}
służący
\end{lstlisting}
\end{document}

Result

6
  • 1
    Btw. Why no one have updated listings package to support unicode? I mean TeX community seems to be quite active, and it doesn't look to me as something very hard to fix...
    – zduny
    Commented Nov 7, 2012 at 12:14
  • @mrpyo It is very hard to fix. For one thing, Unicode can appear in different flavors, multiple byte strings or as big chars (XeTeX/LuaTeX). For the other thing, the internal parsing stuff very likely needs to be rewritten. Commented Nov 7, 2012 at 12:50
  • @HeikoOberdiek, is this the fix to listings, ctan.org/pkg/listingsutf8? or is it something unrelated? (I just found it be chance)
    – alfC
    Commented Nov 23, 2014 at 18:55
  • 1
    @alfC It's a limited workaround. The listings has to be in a file, the environment and inline form are not supported. Also an 8-bit encoding is needed, which contains all the needed characters. Commented Nov 23, 2014 at 19:04
  • @HeikoOberdiek, Under which conditions "an one-byte encoding exists that the file can be converted to" (quote from the manual)?
    – alfC
    Commented Nov 23, 2014 at 19:07
5

Here's another solution, if listings isn't a requirement. My PythonTeX package provides syntax highlighting for most languages, via the Python highlighting library Pygments. The package offers full Unicode support, under pdflatex, lualatex, and xelatex. It works under Windows, Linux, and OS X.

PythonTeX provides a pygments environment for typesetting blocks of code, a \pygment command for inline use, and an \inputpygments command for bringing in external files. All of these take optional fancyvrb commands, so you can easily get features like line numbering.

Here's a slight modification of your minimal example:

\documentclass[11pt,polish]{article}

\usepackage{fontspec}
\usepackage{pythontex}
\usepackage{xunicode}
\usepackage{xltxtra}
\usepackage{babel}

\setmainfont{Charis SIL} % Some font with good unicode support

\begin{document}

% Just change `text' to the correct language for your code
\begin{pygments}{text}
służący
służąca do obsługi
łżąsuca do łobsugi
\end{pygments}

\end{document}

enter image description here

If your source code uses any tabs, you should use xelatex with the -8bit option, so that tabs can be correctly written to file by xelatex (see this for details).

Not the answer you're looking for? Browse other questions tagged .