BibLaTeX not working with Biber 2.8 - error reading .bib file in ascii format

Question

When I try to execute BibTeX, I get the following error message:

INFO - This is Biber 2.8
INFO - Logfile is 'test.blg'
INFO - Reading 'test.bcf'
INFO - Found 1 citekeys in bib section 0
INFO - Processing section 0
INFO - Looking for bibtex format file 'H:/PhD/Work/Papers/library.bib' for section 0
ERROR - Data file 'H:/PhD/Work/Papers/library.bib' cannot be read in encoding 'ascii': ascii "\xE2" does not map to Unicode at C:\Users\rp1513\AppData\Local\Temp\par-727031353133\cache-72ca3f5324b5c5d2fae0b0f98af3c6b7e3920161\inc\lib/File/Slurper.pm line 63.
INFO - ERRORS: 1

The library.bib file is automatically generated by Mendeley. I have no problem if I change the backend to “bibtex” and then change the bibtex execution to bibtex as well.

Here is an MWE using my library.bib file:

% preamble to define document settings
% basic page setup
\documentclass[a4paper,12pt,oneside]{book}

% bibliography settings
\usepackage[backend=biber, style=authoryear, citestyle=authoryear]{biblatex}
\bibliography{H:/PhD/Work/Papers/library}

\begin{document}

\tableofcontents

Here is a reference to one of the papers in my Mendeley library (\cite{Cancer2011}).

\printbibliography[heading=bibintoc]

\end{document}

The .bib file is automatically generated by Mendeley, but here is the head:

Automatically generated by Mendeley Desktop 1.17.11
Any changes to this file will be lost if it is regenerated by Mendeley.

BibTeX export options can be customized via Options -> BibTeX in Mendeley Desktop

@article{Kim2013,
abstract = {Despite the advances in biomedical research and clinical applications, cancer remains a leading cause of death worldwide. Given the limitations of conventional chemotherapeutics, including serious toxicities and reduced quality of life for patients, the development of safe and efficacious alternatives with known mechanism of action is much needed. Prevention of cancer through dietary intervention may hold promise and has been investigated extensively in the recent years. AMP-activated protein kinase (AMPK) is an energy sensor that plays a key role in the regulation of protein and lipid metabolism in response to changes in fuel availability. When activated, AMPK promotes energy-producing catabolic pathways while inhibiting anabolic pathways, such as cell growth and proliferation - thereby antagonizing carcinogenesis. Other anti-cancer effects of AMPK may include promoting autophagy and DNA repair upon UVB damage. In the last decade, interest in AMPK has grown extensively as it emerged as an attractive target molecule for cancer prevention and treatment. Among the latest developments is the activation of AMPK by naturally occurring dietary constituents and plant products - termed phytochemicals. Owing to their efficacy and safety, phytochemicals are considered as an alternative to the conventional harmful chemotherapy. The rising popularity of using phytochemicals for cancer prevention and therapy is supported by a substantial progress in identifying the molecular pathways involved, including AMPK. In this article, we review the recent progress in this budding field that suggests AMPK as a new molecular target in the prevention and treatment of cancer by phytochemicals.},
author = {Kim, Inyoung and He, Yu-Ying},
doi = {10.3389/fonc.2013.00175},
file = {:C$\backslash$:/Users/rp1513/AppData/Local/Mendeley Ltd./Mendeley Desktop/Downloaded/Kim, He - 2013 - Targeting the AMP-Activated Protein Kinase for Cancer Prevention and Therapy.pdf:pdf},
issn = {2234-943X},
journal = {Frontiers in oncology},
keywords = {AMPK,DNA repair,ampk,apoptosis,cancer prevention,dna repair,phytochemicals,proliferation},
number = {175},
pages = {1--12},
pmid = {23875169},
title = {{Targeting the AMP-Activated Protein Kinase for Cancer Prevention and Therapy.}},
url = {http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=3711071{\&}tool=pmcentrez{\&}rendertype=abstract},
volume = {3},
year = {2013}
}
@article{Ostrom2014,
abstract = {Gliomas are the most common primary intracranial tumor, representing 81{\%} of malignant brain tumors. Although relatively rare, they cause significant mortality and morbidity. Glioblastoma, the most common glioma histology (∼45{\%} of all gliomas), has a 5-year relative survival of ∼5{\%}. A small portion of these tumors are caused by Mendelian disorders, including neurofibromatosis, tuberous sclerosis, and Li-Fraumeni syndrome. Genomic analyses of glioma have also produced new evidence about risk and prognosis. Recently discovered biomarkers that indicate improved survival include O⁶-methylguanine-DNA methyltransferase methylation, isocitrate dehydrogenase mutation, and a glioma cytosine-phosphate-guanine island methylator phenotype. Genome-wide association studies have identified heritable risk alleles within 7 genes that are associated with increased risk of glioma. Many risk factors have been examined as potential contributors to glioma risk. Most significantly, these include an increase in risk by exposure to ionizing radiation and a decrease in risk by history of allergies or atopic disease(s). The potential influence of occupational exposures and cellular phones has also been examined, with inconclusive results. We provide a “state of the science” review of current research into causes and risk factors for gliomas in adults.},
author = {Ostrom, Quinn T. and Bauchet, Luc and Davis, Faith G. and Deltour, Isabelle and Fisher, James L. and Langer, Chelsea Eastman and Pekmezci, Melike and Schwartzbaum, Judith A. and Turner, Michelle C. and Walsh, Kyle M. and Wrensch, Margaret R. and Barnholtz-Sloan, Jill S.},
doi = {10.1093/neuonc/nou087},
file = {:/icnas3.cc.ic.ac.uk/rp1513/PhD/Work/Papers/nou087.pdf:pdf},
isbn = {1523-5866 (Electronic)$\backslash$r1522-8517 (Linking)},
issn = {15235866},
journal = {Neuro-Oncology},
keywords = {brain tumors,epidemiology,genome-wide association studies,glioma,risk factors},
number = {7},
pages = {896--913},
pmid = {24842956},
title = {{The epidemiology of glioma in adults: A state of the science review}},
volume = {16},
year = {2014}
}

Any help would be greatly appreciated!

Answer 1

You can enforce utf8 encoding with bibencoding=utf8.

So instead use:

\usepackage[backend=biber, bibencoding=utf8, style=authoryear, citestyle=authoryear]{biblatex}

I had a similar issue. Biber 2.7 recoded the problematic characters automatically and logged a warning. It seems Biber 2.8 and newer use File::Slurper which better enforces ascii encoding but now doesn't automatically recode the problematic characters.

Update: The development version of Biber 2.10 now includes changes so that it will read ascii files as utf8 by default and log relevant warnings. This behaviour closely resembles the behaviour of Biber 2.7 and earlier.

Answer 2

The problem is as JacobD points out in his answer that Biber 2.8 and later use a newer version of File::Slurper that is less forgiving when it comes to reading files with incorrect encoding.

By default biblatex and Biber try to figure out the encoding of your .tex and .bib file automatically. They will assume that the .tex and .bib file use the same encoding. If you load the inputenc, inputenx, or luainputenc package, the encoding set there is assumed for the .tex file. If you don't use one of those packages, biblatex assumes UTF-8 for LuaTeX or XeTeX, and ASCII for all other engines. (If your LaTeX version is from April 2018 or later, it uses UTF-8 by default even with pdfTeX, biblatex recognises this and assumes UTF-8 instead of US-ASCII.)

Your MWE does not load any of the inputenc-like packages and will be assumed to be ASCII if compiled with pdfLaTeX (that is if you use an old LaTeX format from before April 2018). The .bib file will then also be assumed to be ASCII-encoded (even though it probably is in UTF-8 - it definitely is not ASCII-only since there are some characters in the various abstract fields that are not in ASCII range).

You have several options now.

1. Load inputenc with the correct encoding option (probably \usepackage[utf8]{inputenc}) or use a Unicode engine (XeLaTeX, LuaLaTeX) directly. I think this is the best option, but you have to make sure that both your .tex and .bib file use the encoding you pass to inputenc. (This happens automatically in LaTeX version 2018-04-01 or later.)

2. Specify the encoding of your .bib file when loading biblatex with bibencoding

   \usepackage[backend=biber, bibencoding=utf8, style=authoryear]{biblatex}
   

3. Make sure that your .bib file is ASCII-only. This might be the easy way out if you don't need any non-ASCII chars (for example if you only have them in the abstract field that you don't print.)

Biber >= 2.10 is a bit more forgiving again. When instructed to read files in ASCII it reads them in UTF-8 instead. Since UTF-8 is backwards compatible to ASCII this does not cause problems for pure ASCII files, and gives better results in case your .bib file is actually UTF-8 encoded (it often is nowadays). Of course it still fails spectacularly if your file has non-ASCII chars and uses an encoding other than UTF-8, but in that case you are asking for trouble anyway if you don't declare the right encoding to LaTeX in the first place.

See also https://github.com/plk/biber/issues/201. Since the error happens at a very early stage when reading the file, Biber can't give a more detailed error message that locates the problematic character more specifically.

Note also that this applies to all characters in the .bib file, even to comments and fields that are ignored or not usually printed in the bibliography. The abstract field is often problematic.