Tell Latex to ignore ALL unicode errors?

Question

I have a web interface that generates LaTeX-letters to customers. These letters are rendered dynamically. Some customers have some special characters in their names, that LaTeX can't translate, so I run into unicode errors all the time.

The *.tex-Template includes \usepackage[utf8]{inputenc}, but I keep having problems with unicode characters.

Here are two error messages from today:

Unicode char \\u8:\xc2\xb0 not set up for use with LaTeX.

Unicode char \\u8:\xe5\x8d\x97 not set up for use with LaTeX.

For now, I use a Python dictionary to convert some already as problematic identified chars into correct LaTeX code. But it's never ending.

Is there a way to tell LaTeX to ignore ALL unicode-related compiling errors, e.g. by telling LaTeX to convert all problematic chars into one ?-char?

Answer 1

The form of the error message you show suggests an older latex release but for all releases the error comes from this or an older version with the same name.

\def\UTFviii@defined#1{%
  \ifx#1\relax
      \PackageError{inputenc}{Unicode\space char\space\expandafter
                              \UTFviii@splitcsname\string#1\relax
                              \MessageBreak
                              not\space set\space up\space
                              for\space use\space with\space LaTeX}\@eha
  \else\expandafter
    #1%
  \fi
}

so

\makeatletter
\def\UTFviii@defined#1{%
  \ifx#1\relax
      ?%
  \else\expandafter
    #1%
  \fi
}

\makeatother

after loading inputenc should remove the error.

Answer 2

You have an XY-problem. You in fact want to use a modern TeX engine on your backend, probably LuaTeX, that understands Unicode natively. Legacy 8-bit TeX is the wrong tool for the job and will never be able to do what you want.

There’s no one font that supports all of Unicode, but you can detect most scripts and switch to an appropriate font automatically with either babel on LuaTeX or ucharclasses in XeTeX. It’s not always possible to distinguish Chinese and Japanese, or Russian and Bulgarian, without language tagging, but most languages should work out of the box with the right template.

If you don’t go that route, the vast majority of Unicode errors you will get will be from users typing in combining accents, which PDFTeX cannot understand. The best you can (reasonably) do—if you aren’t already—is to convert your input, before passing it to TeX, to NFC form, at the same time as you sanitize it. This converts as many combining accents to precomposed characters as possible. You can also strip out any Unicode characters that your backend does not support, but many fonts will give you a fallback character like you want.

If you really do want to add support for individual missing Unicode characters, the best way to do so is with newunicodechar. This will let you define, for example, zero-width space as {\hskip0} and zero-width non-joiner as {}. You could in theory convert all unsupported Unicode codepoints to ? this way, but you probably are better off doing it on the front end and giving your user a better error message.