Is it expected that Biber converts escape sequences into non-processable Unicode representations?

Question

I am trying to compile a document using BibLaTeX and Biber that references a document by a Latvian author whose surname contains an a with a macron diacritical mark (ā).

Without the --output-safechars flag for Biber, pdflatex cannot process the .bbl file and says:

Package inputenc Error: Unicode char \u8:-ü not set up for use with LaTeX.

This seems to refer to the aforementioned special character ā, even though I took care to use the appropriate LaTeX escape sequence {\=a} in my .bib file.

---

MWE:

File test.tex:

\documentclass{article}

\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}

\usepackage[
  backend=biber,
  style=alphabetic
]{biblatex}
\addbibresource{test.bib}

\begin{document}

abc~\cite{SIOC}

\printbibliography

\end{document}

File test.bib:

@MISC{SIOC,
  author = {Uldis Boj\={a}rs and John Breslin},
  title = {{SIOC} Core Ontology Specification},
  howpublished = {\url{http://rdfs.org/sioc/spec/}},
  year = {2010},
  timestamp = {2015.07.06}
}

Steps to reproduce the error:

pdflatex test.tex
biber test
pdflatex test.tex

Steps to work around the error and get the desired PDF file:

pdflatex test.tex
biber --output-safechars test
pdflatex test.tex
biber --output-safechars test
pdflatex test.tex
pdflatex test.tex

---

Interestingly, with --output-safechars, the author name is rendered in the .bbl file as follows:

{{hash=4364fe18faaa147d121961f15b77b40a}{Boj\={a}rs}{B\bibinitperiod}{Uldis}{U\bibinitperiod}{}{}{}{}}%

Without --output-safechars, it is rendered like this:

{{hash=4364fe18faaa147d121961f15b77b40a}{Bojārs}{B\bibinitperiod}{Uldis}{U\bibinitperiod}{}{}{}{}}%

That means, Biber actively changes my escape sequence (that would have worked in pdflatex) back to a Unicode character (that does not work).

---

Is this the expected behaviour?

More concretely:

Is it safe in terms of side-effects to use --output-safechars instead, and if so, is there any reason why one would ever call Biber without --output-safechars?

Alternatively, it seems to be possible to define the respective Unicode characters, or allegedly to use a --bblsafechars option, even though that does not appear in the Biber manual.

Answer 1

Yes it is the expected behaviour. biber will (try to) convert commands in the input to unicode to be able to sort the entries.

For accent commands it will in a number of case use the unicode "glyph + combining accent" representation.

It will then later output this "glyph + combining accent" in the bbl and pdflatex won't be able to use it (and it is not possible to change this).

The best way to avoid the problem is not to use accent commands like \= in the bib but only the correct utf8-input ā. --output-safechars should normally work ok too, but it will convert other non-ascii input to commands too (e.g. the euro) and you should check if you really always get the command you want.