Take the 2-minute tour ×
TeX - LaTeX Stack Exchange is a question and answer site for users of TeX, LaTeX, ConTeXt, and related typesetting systems. It's 100% free, no registration required.

As a preamble to all of this I am using tufte-latex, bibentry, inputenc with the utf8 option, and natbib packages with MikTeX 2.9.

There seems to be a bug/incompatibility with bibentry and UTF8 characters. The quick description of the issue is this:

A bibliography entry contains UTF8 characters (I'm using CJK in the file) in one of it's fields (title specifically).

Using BibTeX, and then LaTeX the document compiles and the bibliography appears correctly typeset, UTF8 characters and all. However, using \bibentry{bibliogrpahy key} to cite a UTF8 reference in the text body itself gives a garbled entry, i.e., the UTF8 chars are not being encoded properly.

Here is a MWE showing the "bug":

 \documentclass[openbib]{tufte-book}
 \usepackage{CJK}
 \usepackage[T1]{fontenc}
 \usepackage[utf8]{inputenc}
 \begin{document}
 \frontmatter

 \mainmatter
 \chapter{testing bibenty}
 Using bibentry with unicode:

\bibentry{utf8key}
\backmatter
\bibliographystyle{plain}
\bibliography{utf8bib}

\end{document}
%%% Local Variables: 
%%% mode: latex
%%% TeX-master: t
%%% End: 

The bibliography file utf8bib.bib contains:

@misc{utf8key,
title="The unicode chars are here: \begin{CJK}{UTF8}{cyberbit}日本\end{CJK}",
author="Using Unicode CJK with bibentry",
howpublished="MWE on Stackoverflow."
}

From searching for a solution to this it seems that this is a failing with bibentry itself. I attempted to contact the package author however received no reply.

As an idea for a potential workaround I thought of this:

could I create two bibliography files one of them with all of my bib. items, including the UTF8 entries however having these characters stripped, and a second one duplicating the bib items with the UTF8 chars.

Here's the catch. I'd like to use the stripped bibliography for using \bibentry, however I want to use the unstripped bibliography to be used for the printed bibliography at the end of the text. The problem is to avoid having duplicated bib items, stripped and non-stripped, in the final bibliography.

Is there a way to flag or select which item is printed or is this idea just not possible?

share|improve this question
    
Please add a minimal working example (MWE) that illustrates your problem. –  Werner Dec 13 '11 at 17:20
3  
bibtex is really not compatible with UTF8 at all. When it works, you're lucky. If you need a UTF8 bib file you really should switch to biblatex and biber which are designed to handle them properly. You can emulate bibentry trivially with the \fullcite command. –  Alan Munn Dec 13 '11 at 19:55
    
@AlanMunn So is the UTF8 issue with BibTex or with the bibentry package? Very interesting. I'll look into the biblatex and biber and see what happens. –  Steve Dec 14 '11 at 2:41
    
It may well be that there is a also problem with bibentry and UTF8, (as your example shows) but the main point is that bibtex itself cannot handle UTF8 in the general case, so if you are using UTF8 for your .bib file you should switch. –  Alan Munn Dec 14 '11 at 2:43
    
Unfortunately the tufte-latex package, due to its use of natbib, is not compatible with biblatex. There is work on patching this, however I was unable to make biblatex / biber work correctly with it. –  Steve Dec 14 '11 at 3:54
show 1 more comment

migrated from stackoverflow.com Dec 13 '11 at 17:16

This question came from our site for professional and enthusiast programmers.

1 Answer

up vote 2 down vote accepted

Hack solution

Use \scantokens{日本\endinput} instead of 日本. It is about the catcodes, but I don't want to explain too much. It is just a catcode hack with eTeX to ensure that the catcodes of the first byte of CJK symbols are active (13).

If you use XeTeX with xeCJK, things may be easier. Because xeCJK does not change the catcodes.

Full example:

\begin{filecontents*}{utf8bib.bib}
@misc{utf8key,
title="The unicode chars are here: \scantokens{日本\endinput}",
author="Using Unicode CJK with bibentry",
howpublished="MWE on Stackoverflow."
}
\end{filecontents*}

\documentclass[openbib]{tufte-book}
\usepackage{CJK}
\usepackage[T1]{fontenc}
\usepackage{natbib}

\begin{document}
\begin{CJK}{UTF8}{bsmi} % suggest usage of CJK. It doesn't matter to use it "global"
\frontmatter
\mainmatter
\chapter{testing bibenty}
Using bibentry with unicode:

\bibentry{utf8key}
\backmatter
\bibliographystyle{plainnat}
\bibliography{utf8bib}

\clearpage\end{CJK} % use \clearpage to fix an old bug of CJK
\end{document}

Additional notes

Although BibTeX does not support multibyte characters as Alan said, most times it works. Personally I usually use BibTeX without problems.

  • It is usually better to use CJK environments only once just after \begin{document} and before \end{document}.

  • (Hard) wrap the .bib file manually. Don't use a long line, or the multibyte CJK character may be truncated by BibTeX in the middle.

share|improve this answer
    
The \scantokens worked perfectly. Although I had to alter your solution slightly. I did not use the \begin{filecontents*}{utf8bib.bib} <bib. entry> \end{filecontents*} I also had to remove the \endinput from your \scantokens line above. The bibliography entry which worked for me looks as follows: @misc{utf8key, title="The unicode chars are here: \scantokens{\begin{CJK}{UTF8}{cyberbit}日本\end{CJK}}", author="Using Unicode CJK with bibentry", howpublished="MWE on Stackoverflow." } –  Steve Dec 16 '11 at 1:14
    
@Steve: I just use filecontents environment to generate the .bib file, you certainly don't need it. And you are free to use CJK environment anywhere. However, you'd better use \endinput at the end of \scantokens to prevent unexpected spaces. –  Leo Liu Dec 16 '11 at 2:34
    
I attempted to use the \endinput as you indicated however it started creating errors about the "line ending before par was finished" or something like that. (I can repost the error if you wish to see it exactly). The errors was centered on the line I was testing the \scantokens \endinput. Once I removed the \endinput it worked fine. –  Steve Dec 17 '11 at 14:40
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.