# encoding problem with \verbatiminput : Package inputenc Error: Unicode char \u8: not set up for use with LaTeX

I'm trying to import some logs files using \verbatiminput to do a automatic report from our logs. I got some encoding problem.

My objective: import some logs files with various special char (included hidden one) as verbatim.

• I believed the file to be in ISO-8859-1. (the server OS is windows)
• I got a LOT of file. So the usual solution "rewrite it"/"look for the faulty char" is not an option.
• I run pdflatex from a mac osx.
• I converted the files with (notice the -c option to drop faulty char)

.

for file in ./test*/*.txt
do
iconv -c -f ISO-8859-1 -t utf8 \$file  > \$file.n1
mv -f \$file.n1 \$file
done

\usepackage{verbatim}
\usepackage{lmodern}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}

error message I got :

! Package inputenc Error: Unicode char \u8:° not set up for use with LaTeX.

To solve the "degree" problem, I added

cat \$file.n1 | tr '°' 'º' > \$file.n2

then I got an other error on an other file, this time with an invisible char.

! Package inputenc Error: Unicode char \u8: not set up for use with LaTeX.

I first tried to not convert the file and import them as T1/latin1 or LY1/ascinew, but I got the error

! Package inputenc Error: Keyboard character used is undefined
(inputenc)                in inputencoding `latin1'.

on different files.

My question:

• Is there a way to ignore/replace ALL the faulty char?
• from LaTeX ? from the shell ?
• using "tr" or "iconv" ?
• Am I doing something wrong when importing the files / doing the conversion?
• with which encoding will I have less problem ?

any help is welcome, as I am blocked on this since two days.

PS: My problem is not with the degree symbole in particular, but with unicode char that LaTeX cannot printout.

-
Very strange, but, at the end, I just remove the \usepackage[utf8]{inputenc} and the iconv then everything works fine. (using pdflatex). Thanks a lot for the answers, I did learn useful stuffs. –  Loda Jan 4 '12 at 14:35
Well if your files are not utf8-encoded than if you use \usepackage[utf8]{inputenc} you are lying to LaTeX - and LaTeX doesn't like lies ;-). But you shouldn't remove inputenc completly (unless you are using an unicode engine like xetex). If you do it non-ascii-chars will only work partly. Load inputenc with the correct option (your encoding), and if you load files with another encoding than tell this LaTeX with the \inputencoding command. –  Ulrike Fischer Jan 4 '12 at 15:29

Yes you can replace the faulty char. You could replace ° with \ensuremath{^\circ}

\newcommand{\degree}{\ensuremath{^\circ}}
\DeclareUnicodeCharacter{B0}{\degree}

Or you could use xelatex then there is no need for \usepackage[utf8]{inputenc} because it has full utf8 support build in.

The encoding is all right Latex just doesn't know what to do with the unicode symbol :D

-
as I include the files as verbatim, changing the faulty char in the imported file will not help, no? I will look at the xelatex... –  Loda Jan 4 '12 at 11:48
No this would be hard inside of verbatim. XeLatex should be the easiest option. Just remove the \usepackage[utf8]{inputenc} from your header and use xelatex instead of pdflatex. If you want to see the unicode characters make sure that you use a font that supports them. (The standard cmtt10 hasn't support) –  someonr Jan 4 '12 at 11:58
You can use \usepackage{xeCJK} to get such a font. –  Martin Ueding Mar 3 '12 at 11:26

The character ° is actually not included in the definitions make by the utf8 module for inputenc.

\usepackage{textcomp} % for \textdegree
\usepackage{newunicodechar} % for \newunicodechar
\newunicodechar{°}{\textdegree}

Alternatively, you can resort to a "hand made symbol"

\newunicodechar{°}{\ensuremath{{}^\circ}}

that will work both in text and math mode (and textcomp is not needed).

The advantage of newunicodechar is that it doesn't require knowing the Unicode point of the character we want to define a command for.

Of course the "degree" case is an example: you can add all the characters you need.

However, if your files are Latin-1 encoded (ISO 8859-1), you can try switching to this encoding before inputting them:

\usepackage[latin1,utf8]{inputenc} % utf8 is default
\newcommand{\latinoneverbatiminput}{%
\begingroup\inputencoding{latin1}\verbatiminput{#1}\endgroup}

Then \latinoneverbatiminput{file} will use Latin-1 for file.

-
nice package. I always looked at a Unicode table for this. –  someonr Jan 4 '12 at 11:55
@someonr That's why I wrote it: I'm lazy. :) –  egreg Jan 4 '12 at 11:58
about the "degree" : it's working: I removed the extra "tr" from my .sh. But How do I solve the invisible faulty char ? –  Loda Jan 4 '12 at 12:00
You can look at the log file, for example with LESSCHARSET=ascii less file: search for u8 and you'll find the UTF-8 representation of the character; for example the degree is <C2><B0>. Add it to your question, so that a recipe for dealing with it can be suggested. –  egreg Jan 4 '12 at 12:08
I will keep in my notes the "\newunicodechar". nice function! –  Loda Jan 4 '12 at 14:36

You don't need to reencode your files. But you must declare/change the encoding of the files before loading them:

\begingroup
\inputencoding{whatever}
\verbatiminput{file}
\endgroup

I don't know what value you should use for "whatever". On windows I would at first try "ansinew" (latin1 has a less chars declared).

-