35

Please, do you know if there's any possibility how to use the listings package together with XeLaTeX? I have to use XeLaTeX and therefore I have UTF-8 encoded files. This is not compatible with listings which allows one-byte encoding. (I know there is listingsutf8 package, but it converts files from UTF-8 which I don't need.)

The listings package works somehow, but I only can use English letters. I've fond some solution like a list of special characters (Having problems with listings and UTF-8. Can it be fixed?) but it seems it works only for some letters. It there anything that would support UTF-8 in listing 'naturally'?

3

5 Answers 5

35

The link you are mentioning is for 8-bit engines like pdflatex, it will not work with xetex or luatex.

With xelatex + lualatex chars with unicode positions below 256 will work out of the box:

\documentclass{article}
\usepackage{listings,fontspec}
\begin{document}
\begin{lstlisting}
öäü é è ß 
\end{lstlisting}
\end{document}

Things get more complicated if you want to use chars above 256. E.g. an € or the french "oe" or greek etc. In this case you will have to add them to the processing list of listings first and use extendedchars=true. Here an example which adds the euro and the french oe and Oe (the middle listings shows the problems without the extended definitions):

\documentclass{article}
\usepackage{fontspec}
\usepackage{listings}

\makeatletter
\lst@InputCatcodes
\def\lst@DefEC{%
 \lst@CCECUse \lst@ProcessLetter
  ^^80^^81^^82^^83^^84^^85^^86^^87^^88^^89^^8a^^8b^^8c^^8d^^8e^^8f%
  ^^90^^91^^92^^93^^94^^95^^96^^97^^98^^99^^9a^^9b^^9c^^9d^^9e^^9f%
  ^^a0^^a1^^a2^^a3^^a4^^a5^^a6^^a7^^a8^^a9^^aa^^ab^^ac^^ad^^ae^^af%
  ^^b0^^b1^^b2^^b3^^b4^^b5^^b6^^b7^^b8^^b9^^ba^^bb^^bc^^bd^^be^^bf%
  ^^c0^^c1^^c2^^c3^^c4^^c5^^c6^^c7^^c8^^c9^^ca^^cb^^cc^^cd^^ce^^cf%
  ^^d0^^d1^^d2^^d3^^d4^^d5^^d6^^d7^^d8^^d9^^da^^db^^dc^^dd^^de^^df%
  ^^e0^^e1^^e2^^e3^^e4^^e5^^e6^^e7^^e8^^e9^^ea^^eb^^ec^^ed^^ee^^ef%
  ^^f0^^f1^^f2^^f3^^f4^^f5^^f6^^f7^^f8^^f9^^fa^^fb^^fc^^fd^^fe^^ff%
  ^^^^20ac^^^^0153^^^^0152% nouveau pour xetex
  ^^00}
\lst@RestoreCatcodes
\makeatother
\begin{document}
\begin{lstlisting}[extendedchars=true]%default je pense
bœuf BŒeuf a€e
Dès Noël où un zéphyr haï me
vêt de glaçons würmiens je
dîne d'exquis rôtis de bœuf
au kir à l'aÿ d'âge mûr \& cætera!
\end{lstlisting}

\begin{lstlisting}[extendedchars=false]
bœuf BŒeuf a€e
Dès Noël où un zéphyr haï me
vêt de glaçons würmiens je
dîne d'exquis rôtis de bœuf
au kir à l'aÿ d'âge mûr \& cætera!
\end{lstlisting}


\begin{lstlisting}[language=XML]
bœuf BŒeuf a€e
Dès Noël où un zéphyr haï me
vêt de glaçons würmiens je
dîne d'exquis rôtis de bœuf
au kir à l'aÿ d'âge mûr \& cætera!
\end{lstlisting}
\end{document}

Output:

enter image description here

9
  • How do you know what to add to \lst@InputCatcodes? I was trying this William Shakespeare (/ˈʃeɪkspɪər/)
    – PatrickT
    Commented May 2, 2016 at 15:10
  • 2
    @PatrickT: You will have to look up the unicode positions. Commented May 2, 2016 at 15:13
  • Thanks Ulrike, where do you recommend I look for the example given?
    – PatrickT
    Commented May 2, 2016 at 15:14
  • @PatrickT en.wikipedia.org/wiki/Phonetic_symbols_in_Unicode Commented May 2, 2016 at 15:15
  • Got it thanks! The code for ʃ is 0283, do I just append to the existing list? like ``^^^^0152^^^^0283%`, that's all there is to it right? Tedious, but great! Thanks.
    – PatrickT
    Commented May 2, 2016 at 15:24
9

Thanks for posting this block of code. The same technique can be used to typeset UTF8 APL characters with APL385 Unicode font. Include the following:

% set lstlisting to accept UTF8 APL text
\makeatletter
\lst@InputCatcodes
\def\lst@DefEC{%
 \lst@CCECUse \lst@ProcessLetter
  ^^80^^81^^82^^83^^84^^85^^86^^87^^88^^89^^8a^^8b^^8c^^8d^^8e^^8f%
  ^^90^^91^^92^^93^^94^^95^^96^^97^^98^^99^^9a^^9b^^9c^^9d^^9e^^9f%
  ^^a0^^a1^^a2^^a3^^a4^^a5^^a6^^a7^^a8^^a9^^aa^^ab^^ac^^ad^^ae^^af%
  ^^b0^^b1^^b2^^b3^^b4^^b5^^b6^^b7^^b8^^b9^^ba^^bb^^bc^^bd^^be^^bf%
  ^^c0^^c1^^c2^^c3^^c4^^c5^^c6^^c7^^c8^^c9^^ca^^cb^^cc^^cd^^ce^^cf%
  ^^d0^^d1^^d2^^d3^^d4^^d5^^d6^^d7^^d8^^d9^^da^^db^^dc^^dd^^de^^df%
  ^^e0^^e1^^e2^^e3^^e4^^e5^^e6^^e7^^e8^^e9^^ea^^eb^^ec^^ed^^ee^^ef%
  ^^f0^^f1^^f2^^f3^^f4^^f5^^f6^^f7^^f8^^f9^^fa^^fb^^fc^^fd^^fe^^ff%
  ^^^^20ac^^^^0153^^^^0152%
  ^^^^20a7^^^^2190^^^^2191^^^^2192^^^^2193^^^^2206^^^^2207^^^^220a%
  ^^^^2218^^^^2228^^^^2229^^^^222a^^^^2235^^^^223c^^^^2260^^^^2261%
  ^^^^2262^^^^2264^^^^2265^^^^2282^^^^2283^^^^2296^^^^22a2^^^^22a3%
  ^^^^22a4^^^^22a5^^^^22c4^^^^2308^^^^230a^^^^2336^^^^2337^^^^2339%
  ^^^^233b^^^^233d^^^^233f^^^^2340^^^^2342^^^^2347^^^^2348^^^^2349%
  ^^^^234b^^^^234e^^^^2350^^^^2352^^^^2355^^^^2357^^^^2359^^^^235d%
  ^^^^235e^^^^235f^^^^2361^^^^2362^^^^2363^^^^2364^^^^2365^^^^2368%
  ^^^^236a^^^^236b^^^^236c^^^^2371^^^^2372^^^^2373^^^^2374^^^^2375%
  ^^^^2377^^^^2378^^^^237a^^^^2395^^^^25af^^^^25ca^^^^25cb%
  ^^00}
\lst@RestoreCatcodes
\makeatother

More details at: http://bakerjd99.wordpress.com/2011/08/15/typesetting-utf8-apl-code-with-the-latex-lstlisting-package/

1

Just an addition to previous answers. Here is table for UTF8 APL, greek and cyrillic symbols

\usepackage{listingsutf8}

\makeatletter
\lst@InputCatcodes
\def\lst@DefEC{%
    \lst@CCECUse \lst@ProcessLetter
    ^^80^^81^^82^^83^^84^^85^^86^^87^^88^^89^^8a^^8b^^8c^^8d^^8e^^8f%
    ^^90^^91^^92^^93^^94^^95^^96^^97^^98^^99^^9a^^9b^^9c^^9d^^9e^^9f%
    ^^a0^^a1^^a2^^a3^^a4^^a5^^a6^^a7^^a8^^a9^^aa^^ab^^ac^^ad^^ae^^af%
    ^^b0^^b1^^b2^^b3^^b4^^b5^^b6^^b7^^b8^^b9^^ba^^bb^^bc^^bd^^be^^bf%
    ^^c0^^c1^^c2^^c3^^c4^^c5^^c6^^c7^^c8^^c9^^ca^^cb^^cc^^cd^^ce^^cf%
    ^^d0^^d1^^d2^^d3^^d4^^d5^^d6^^d7^^d8^^d9^^da^^db^^dc^^dd^^de^^df%
    ^^e0^^e1^^e2^^e3^^e4^^e5^^e6^^e7^^e8^^e9^^ea^^eb^^ec^^ed^^ee^^ef%
    ^^f0^^f1^^f2^^f3^^f4^^f5^^f6^^f7^^f8^^f9^^fa^^fb^^fc^^fd^^fe^^ff%
    ^^^^20ac^^^^0153^^^^0152%
    ^^^^20a7^^^^2190^^^^2191^^^^2192^^^^2193^^^^2206^^^^2207^^^^220a%
    ^^^^2218^^^^2228^^^^2229^^^^222a^^^^2235^^^^223c^^^^2260^^^^2261%
    ^^^^2262^^^^2264^^^^2265^^^^2282^^^^2283^^^^2296^^^^22a2^^^^22a3%
    ^^^^22a4^^^^22a5^^^^22c4^^^^2308^^^^230a^^^^2336^^^^2337^^^^2339%
    ^^^^233b^^^^233d^^^^233f^^^^2340^^^^2342^^^^2347^^^^2348^^^^2349%
    ^^^^234b^^^^234e^^^^2350^^^^2352^^^^2355^^^^2357^^^^2359^^^^235d%
    ^^^^235e^^^^235f^^^^2361^^^^2362^^^^2363^^^^2364^^^^2365^^^^2368%
    ^^^^236a^^^^236b^^^^236c^^^^2371^^^^2372^^^^2373^^^^2374^^^^2375%
    ^^^^2377^^^^2378^^^^237a^^^^2395^^^^25af^^^^25ca^^^^25cb%
    %
    ^^^^0374^^^^0375^^^^037a^^^^037e^^^^0384^^^^0385^^^^0386^^^^0387^^^^0388^^^^0389^^^^038a%
    ^^^^038c^^^^038e^^^^038f^^^^0390^^^^0391^^^^0392^^^^0393^^^^0394^^^^0395^^^^0396^^^^0397%
    ^^^^0398^^^^0399^^^^039a^^^^039b^^^^039c^^^^039d^^^^039e^^^^039f^^^^03a0^^^^03a1^^^^03a3%
    ^^^^03a4^^^^03a5^^^^03a6^^^^03a7^^^^03a8^^^^03a9^^^^03aa^^^^03ab^^^^03ac^^^^03ad^^^^03ae%
    ^^^^03af^^^^03b0^^^^03b1^^^^03b2^^^^03b3^^^^03b4^^^^03b5^^^^03b6^^^^03b7^^^^03b8^^^^03b9%
    ^^^^03ba^^^^03bb^^^^03bc^^^^03bd^^^^03be^^^^03bf^^^^03c0^^^^03c1^^^^03c2^^^^03c3^^^^03c4%
    ^^^^03c5^^^^03c6^^^^03c7^^^^03c8^^^^03c9^^^^03ca^^^^03cb^^^^03cc^^^^03cd^^^^03ce^^^^03d0%
    ^^^^03d1^^^^03d2^^^^03d3^^^^03d4^^^^03d5^^^^03d6^^^^03d7^^^^03d8^^^^03d9^^^^03da^^^^03db%
    ^^^^03dc^^^^03dd^^^^03de^^^^03df^^^^03e0^^^^03e1^^^^03e2^^^^03e3^^^^03e4^^^^03e5^^^^03e6%
    ^^^^03e7^^^^03e8^^^^03e9^^^^03ea^^^^03eb^^^^03ec^^^^03ed^^^^03ee^^^^03ef^^^^03f0^^^^03f1%
    ^^^^03f2^^^^03f3^^^^03f4^^^^03f5^^^^03f6^^^^03f7^^^^03f8^^^^03f9^^^^03fa^^^^03fb^^^^1f00%
    ^^^^1f01^^^^1f02^^^^1f03^^^^1f04^^^^1f05^^^^1f06^^^^1f07^^^^1f08^^^^1f09^^^^1f0a^^^^1f0b%
    ^^^^1f0c^^^^1f0d^^^^1f0e^^^^1f0f^^^^1f10^^^^1f11^^^^1f12^^^^1f13^^^^1f14^^^^1f15^^^^1f18%
    ^^^^1f19^^^^1f1a^^^^1f1b^^^^1f1c^^^^1f1d^^^^1f20^^^^1f21^^^^1f22^^^^1f23^^^^1f24^^^^1f25%
    ^^^^1f26^^^^1f27^^^^1f28^^^^1f29^^^^1f2a^^^^1f2b^^^^1f2c^^^^1f2d^^^^1f2e^^^^1f2f^^^^1f30%
    ^^^^1f31^^^^1f32^^^^1f33^^^^1f34^^^^1f35^^^^1f36^^^^1f37^^^^1f38^^^^1f39^^^^1f3a^^^^1f3b%
    ^^^^1f3c^^^^1f3d^^^^1f3e^^^^1f3f^^^^1f40^^^^1f41^^^^1f42^^^^1f43^^^^1f44^^^^1f45^^^^1f48%
    ^^^^1f49^^^^1f4a^^^^1f4b^^^^1f4c^^^^1f4d^^^^1f50^^^^1f51^^^^1f52^^^^1f53^^^^1f54^^^^1f55%
    ^^^^1f56^^^^1f57^^^^1f59^^^^1f5b^^^^1f5d^^^^1f5f^^^^1f60^^^^1f61^^^^1f62^^^^1f63^^^^1f64%
    ^^^^1f65^^^^1f66^^^^1f67^^^^1f68^^^^1f69^^^^1f6a^^^^1f6b^^^^1f6c^^^^1f6d^^^^1f6e^^^^1f6f%
    ^^^^1f70^^^^1f71^^^^1f72^^^^1f73^^^^1f74^^^^1f75^^^^1f76^^^^1f77^^^^1f78^^^^1f79^^^^1f7a%
    ^^^^1f7b^^^^1f7c^^^^1f7d^^^^1f80^^^^1f81^^^^1f82^^^^1f83^^^^1f84^^^^1f85^^^^1f86^^^^1f87%
    ^^^^1f88^^^^1f89^^^^1f8a^^^^1f8b^^^^1f8c^^^^1f8d^^^^1f8e^^^^1f8f^^^^1f90^^^^1f91^^^^1f92%
    ^^^^1f93^^^^1f94^^^^1f95^^^^1f96^^^^1f97^^^^1f98^^^^1f99^^^^1f9a^^^^1f9b^^^^1f9c^^^^1f9d%
    ^^^^1f9e^^^^1f9f^^^^1fa0^^^^1fa1^^^^1fa2^^^^1fa3^^^^1fa4^^^^1fa5^^^^1fa6^^^^1fa7^^^^1fa8%
    ^^^^1fa9^^^^1faa^^^^1fab^^^^1fac^^^^1fad^^^^1fae^^^^1faf^^^^1fb0^^^^1fb1^^^^1fb2^^^^1fb3%
    ^^^^1fb4^^^^1fb6^^^^1fb7^^^^1fb8^^^^1fb9^^^^1fba^^^^1fbb^^^^1fbc^^^^1fbd^^^^1fbe^^^^1fbf%
    ^^^^1fc0^^^^1fc1^^^^1fc2^^^^1fc3^^^^1fc4^^^^1fc6^^^^1fc7^^^^1fc8^^^^1fc9^^^^1fca^^^^1fcb%
    ^^^^1fcc^^^^1fcd^^^^1fce^^^^1fcf^^^^1fd0^^^^1fd1^^^^1fd2^^^^1fd3^^^^1fd6^^^^1fd7^^^^1fd8%
    ^^^^1fd9^^^^1fda^^^^1fdb^^^^1fdd^^^^1fde^^^^1fdf^^^^1fe0^^^^1fe1^^^^1fe2^^^^1fe3^^^^1fe4%
    ^^^^1fe5^^^^1fe6^^^^1fe7^^^^1fe8^^^^1fe9^^^^1fea^^^^1feb^^^^1fec^^^^1fed^^^^1fee^^^^1fef%
    ^^^^1ff2^^^^1ff3^^^^1ff4^^^^1ff6^^^^1ff7^^^^1ff8^^^^1ff9^^^^1ffa^^^^1ffb^^^^1ffc^^^^1ffd%
    ^^^^1ffe^^^^0410^^^^0411^^^^0412^^^^0413^^^^0414^^^^0415^^^^0416^^^^0417^^^^0418^^^^0419%
    ^^^^041a^^^^041b^^^^041c^^^^041d^^^^041e^^^^041f^^^^0420^^^^0421^^^^0422^^^^0423^^^^0424%
    ^^^^0425^^^^0426^^^^0427^^^^0428^^^^0429^^^^042a^^^^042b^^^^042c^^^^042d^^^^042e^^^^042f%
    ^^^^0430^^^^0431^^^^0432^^^^0433^^^^0434^^^^0435^^^^0436^^^^0437^^^^0438^^^^0439^^^^043a%
    ^^^^043b^^^^043c^^^^043d^^^^043e^^^^043f^^^^0440^^^^0441^^^^0442^^^^0443^^^^0444^^^^0445%
    ^^^^0446^^^^0447^^^^0448^^^^0449^^^^044a^^^^044b^^^^044c^^^^044d^^^^044e^^^^044f^^^^0401%
    ^^^^0451% 
    ^^00}
\lst@RestoreCatcodes
\makeatother
0

Possible work-around from this answer on SO, is to escape some parts of your code to latex. For example if you use UTF-8 character only in code comments use texcl=true option.

\lstset{texcl=true}

\begin{lstlisting}
    int iLink = 0x01; // use here any UTF-8 symbols
\end{lstlisting}

For more escape options see Listings package docs on page 40.

You can even escape specific sequence surrounded with delimiter (like here).

\lstset{    
    escapeinside={\%*}{*)}
}
0

It seems that recent versions of listings support UTF-8 out of the box with XeLaTeX.

However I found myself in a situation where non-ASCII characters weren’t displayed properly (they were discarded).

It turned out that what I thought UTF-8 encoded files were ISO-8859 encoded files (check with file command), so I had to convert them :

iconv -f ISO-8859 -t UTF-8 inputfile outputfile

After conversion accents and other non-ASCII characters were displayed properly.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .