6

This is a follow-up to my question Translate content in fields in bibliography entries (biblatex), which revealed a problem when trying to replace content in a .bib file when that content contain non-ascii characters like æøå.

Compiling the following MWE works fine. In this case, the string def is replaced by abc:

\documentclass{article}
\usepackage[style = authoryear-comp]{biblatex}
\usepackage{filecontents}
\begin{filecontents}{\jobname.bib}
@BOOK{lennon1968,
    AUTHOR = "John Lennon",
    TITLE = "def",
    YEAR = "1968"}
\end{filecontents}
\DeclareSourcemap{
  \maps[datatype = bibtex]{
    \map{
       \step[fieldsource = title,
          match = {def}, 
          replace = {abc}]
    }
  }
}
\addbibresource{\jobname.bib}
\begin{document}
\nocite{*}
\printbibliography
\end{document}

enter image description here

But when the string contains non-ascii characters like æøå, no output is produced with LaTeX:

\documentclass{article}
\usepackage[style = authoryear-comp]{biblatex}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{filecontents}
\begin{filecontents}{\jobname.bib}
@BOOK{lennon1968,
    AUTHOR = "John Lennon",
    TITLE = "æøå",
    YEAR = "1968"}
\end{filecontents}
\DeclareSourcemap{
  \maps[datatype = bibtex]{
    \map{
       \step[fieldsource = title,
          match = {æøå}, 
          replace = {abc}]
    }
  }
}
\addbibresource{\jobname.bib}
\begin{document}
\nocite{*}
\printbibliography
\end{document}

The console produces this output (biblatextest64.tex being the name of the document) :

Failure to make 'biblatextest64.pdf'
Collected error summary (may duplicate other messages):
  biber biblatextest64: Command for 'biber biblatextest64' gave return code 256
Latexmk: Use the -f option to force complete processing,
 unless error was exceeding maximum runs of latex/pdflatex.
Latexmk: Errors, so I did not complete making targets
C:\texlive2013\bin\win32\runscript.tlu:650: command failed with exit code 12:
perl.exe c:/texlive2013/texmf-dist/scripts/latexmk/latexmk.pl -pdf biblatextest64.tex

A pdf is produced when compiling with XeLaTeX, but no substitution has taken place:

\documentclass{article}
\usepackage[style = authoryear-comp]{biblatex}
\usepackage{fontspec}
\usepackage{filecontents}
\begin{filecontents}{\jobname.bib}
@BOOK{lennon1968,
    AUTHOR = "John Lennon",
    TITLE = "æøå",
    YEAR = "1968"}
\end{filecontents}
\DeclareSourcemap{
  \maps[datatype = bibtex]{
    \map{
       \step[fieldsource = title,
          match = {æøå}, 
          replace = {abc}]
    }
  }
}
\addbibresource{\jobname.bib}
\begin{document}
\nocite{*}
\printbibliography
\end{document}

enter image description here

6
  • Move the \DeclareSourcemap before inputenc. Commented Apr 30, 2014 at 11:48
  • Sorry I confuse match and replace. Moving \DeclareSoureMap gets rid off the errors but the match is not replaced. Commented Apr 30, 2014 at 12:02
  • Did you try match = \regexp{æøå} - this is used to protect special chars in matches.
    – PLK
    Commented May 3, 2014 at 12:34
  • @PLK I tried it now, still no substitution is made.
    – Sverre
    Commented May 3, 2014 at 12:39
  • 1
    This was a bug with a NFD unicode boundary. It should be fixed in 1.9 DEV version on SF now (you need to be using biblatex 2.9 with biber 1.9 which is also on SF).
    – PLK
    Commented May 3, 2014 at 13:42

1 Answer 1

5

It seems that the value given to match is passed through TeX's expansion mechanism, which is a big problem, but also a help, since we can (also selectively) use \detokenize.

\documentclass{article}
\usepackage[style = authoryear-comp]{biblatex}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}

\usepackage{filecontents}
\begin{filecontents}{\jobname.bib}
@BOOK{lennon1968,
    AUTHOR = "John Lennon",
    TITLE = "æøå",
    YEAR = "1968"}
\end{filecontents}

\DeclareSourcemap{
  \maps[datatype = bibtex]{
    \map{
       \step[fieldsource = title,
          match = {\detokenize{æøå}},
          replace = {abc}]
    }
  }
}
\addbibresource{\jobname.bib}
\begin{document}
\nocite{*}
\printbibliography
\end{document}

This is what is written in the .bcf file:

  <!-- SOURCEMAP -->
  <bcf:sourcemap>
    <bcf:maps datatype="bibtex" level="user">
      <bcf:map>
        <bcf:map_step map_field_source="title" map_match="æøå" map_replace="abc"/>
      </bcf:map>
    </bcf:maps>

Without \detokenize, the result is

  <!-- SOURCEMAP -->
  <bcf:sourcemap>
    <bcf:maps datatype="bibtex" level="user">
      <bcf:map>
        <bcf:map_step map_field_source="title" map_match="\T1\ae \T1\o <E5>" map_replace="abc"/>
      </bcf:map>
    </bcf:maps>

So it seems that \detokenize works correctly. However, the replacement doesn't take place, which seems to be a problem with Biber.

To wit, I tried the following Perl script (adapted from https://stackoverflow.com/a/1572156)

use warnings;
use strict;
use utf8;
binmode STDOUT, "utf8";
my $string = "æøå";

$string =~ s/æøå/abc/gi; 

print "$string\n";

and the result on the terminal was

abc
6
  • I'm with you until your Perl script. I see that your Perl script works, but how does that relate to biber and the fact that it doesn't replace the characters?
    – Sverre
    Commented Apr 30, 2014 at 12:19
  • @Sverre Biber is programmed in Perl.
    – egreg
    Commented Apr 30, 2014 at 12:23
  • Does that mean there is a solution to this ... ?
    – Sverre
    Commented Apr 30, 2014 at 12:23
  • @Sverre Not from me, but probably from Biber's developers.
    – egreg
    Commented Apr 30, 2014 at 12:24
  • In other words a bug report should be made? Presumably here: github.com/plk/biber/issues? I'd recommend that someone who knows what they're talking about (i.e. you) submit it rather than me (I would do little more than link to your answer here).
    – Sverre
    Commented Apr 30, 2014 at 12:26

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .