Why this UTF-8 error is solved with a simple \clearpage

An example is shown below, which also includes a description of the problem:

\documentclass{book}

\usepackage{CJKutf8}

\begin{document}
\begin{CJK*}{UTF8}{gbsn}

Some Chinese text.

\section{一些中文}

Compiling this with pdflatex will generate the following common error:

%! PACKAGE INPUTENC ERROR: UNICODE CHAR \U8:Ɨ� NOT SET UP FOR USE WITH LATEX.

However, when a clearpage command is included at the end of the document
(i.e., to uncomment the line below), then the error will disappear and the
document compiles correctly.
%\clearpage  % This is important; otherwise there will be a nasty UTF-8 error.

If we delete section command, the document also compiles correctly
(even without clearpage).

\end{CJK*}
\end{document}


My question is: why does the \clearpage command solve the problem?

• I can't reproduce this problem. My method: Duplicate your two lines of Chines enough times to get multiple pages. Compile. It went without a hitch. Perhaps your setup is old? I run on an up to date TeXLive 2015. – Harald Hanche-Olsen Apr 21 '16 at 7:03
• I get always an error with your document above, but because of the error line you included. If I remove it everything works fine. Is the error you get really in uppercase? – Ulrike Fischer Apr 21 '16 at 7:07
• @HaraldHanche-Olsen That's my fault; now I edited the file, which can readily reproduce the problem in my computer; I am using the 2014 version of TeXLive though. – user789977 Apr 21 '16 at 7:16
• @UlrikeFischer Now I edited the example file, which may be a bit clearer. – user789977 Apr 21 '16 at 7:17
• If you replace the section command with \section[]{一些中文}´, it compiles cleanly. So it's most likely a problem with a moving argument. – Harald Hanche-Olsen Apr 21 '16 at 7:23

It happens when latex tries to typeset the page heading. This happens at page shipout time; if the CJK* environment has ended before the last page ships out, the necessary definitions for doing the typesetting are gone, and you get this error.

You see this clearly if you add \errorcontextlines=99 to your document (outside the CJK* environment please). Also, the problem goes away with \pagestyle{empty} (same caveat), further supporting my diagnosis.

You have already found the proper workaround: Add \clearpage where you did.

Edited to add this version of the workaround – with thanks to egreg in the comments:

\AtBeginDocument{\csname CJK*\endcsname{UTF8}{gbsn}}
\AtEndDocument{\clearpage\csname endCJK*\endcsname}


And then, of course, you remove the CJK* environment from within the document body.

• I guess there's a way for telling that the whole document is in Chinese, without enclosing it in a CJK environment – egreg Apr 21 '16 at 7:45
• @egreg I would certainly hope so! Unfortunately, texdoc CJKutf8 produces a less than helpful result. I'll leave the digging to others. – Harald Hanche-Olsen Apr 21 '16 at 8:23
• I found that \AtBeginDocument{\csname CJK*\endcsname{UTF8}{gbsn}} and \AtEndDocument{\clearpage\csname endCJK*\endcsname} seem to work. – egreg Apr 21 '16 at 8:25
• @egreg That's a neat way to do it. I added it to the answer. – Harald Hanche-Olsen Apr 21 '16 at 13:16

The problem is that the section command sets a header. The problem is that the the CJK environments ends before the header is typeset and so the chinese chars are no longer set up. Using \clearpage makes the CJK-environment span two pages so that is is active when the header is typeset, but is clearly only a work-around. The second problem is that the book class headers are set with \Makeuppercase which can do a lot of harm (that's beside other the reason why the error message uses capitals).

\documentclass{book}
\renewcommand{\sectionmark}[1]{\markright{\thesection.\ #1}} %no uppercase

\usepackage{CJKutf8}

\begin{document}
\begin{CJK*}{UTF8}{gbsn}

Some Chinese text.

\section[\protect\begin{CJK*}{UTF8}{gbsn}一些中文\protect\end{CJK*}]
{一些中文}

Compiling this with pdflatex will generate the following common error:

%! PACKAGE INPUTENC ERROR: UNICODE CHAR \U8:Ɨ� NOT SET UP FOR USE WITH LATEX.

However, when a clearpage command is included at the end of the document
(i.e., to uncomment the line below), then the error will disappear and the
document compiles correctly.
%\clearpage  % This is important; otherwise there will be a nasty UTF-8 error.

If we delete section command, the document also compiles correctly
(even without clearpage).

\end{CJK*}
\end{document}
`