Take the 2-minute tour ×
TeX - LaTeX Stack Exchange is a question and answer site for users of TeX, LaTeX, ConTeXt, and related typesetting systems. It's 100% free, no registration required.

This is a follow-up to my question Translate content in fields in bibliography entries (biblatex), which revealed a problem when trying to replace content in a .bib file when that content contain non-ascii characters like æøå.

Compiling the following MWE works fine. In this case, the string def is replaced by abc:

\documentclass{article}
\usepackage[style = authoryear-comp]{biblatex}
\usepackage{filecontents}
\begin{filecontents}{\jobname.bib}
@BOOK{lennon1968,
    AUTHOR = "John Lennon",
    TITLE = "def",
    YEAR = "1968"}
\end{filecontents}
\DeclareSourcemap{
  \maps[datatype = bibtex]{
    \map{
       \step[fieldsource = title,
          match = {def}, 
          replace = {abc}]
    }
  }
}
\addbibresource{\jobname.bib}
\begin{document}
\nocite{*}
\printbibliography
\end{document}

enter image description here

But when the string contains non-ascii characters like æøå, no output is produced with LaTeX:

\documentclass{article}
\usepackage[style = authoryear-comp]{biblatex}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{filecontents}
\begin{filecontents}{\jobname.bib}
@BOOK{lennon1968,
    AUTHOR = "John Lennon",
    TITLE = "æøå",
    YEAR = "1968"}
\end{filecontents}
\DeclareSourcemap{
  \maps[datatype = bibtex]{
    \map{
       \step[fieldsource = title,
          match = {æøå}, 
          replace = {abc}]
    }
  }
}
\addbibresource{\jobname.bib}
\begin{document}
\nocite{*}
\printbibliography
\end{document}

The console produces this output (biblatextest64.tex being the name of the document) :

Failure to make 'biblatextest64.pdf'
Collected error summary (may duplicate other messages):
  biber biblatextest64: Command for 'biber biblatextest64' gave return code 256
Latexmk: Use the -f option to force complete processing,
 unless error was exceeding maximum runs of latex/pdflatex.
Latexmk: Errors, so I did not complete making targets
C:\texlive2013\bin\win32\runscript.tlu:650: command failed with exit code 12:
perl.exe c:/texlive2013/texmf-dist/scripts/latexmk/latexmk.pl -pdf biblatextest64.tex

A pdf is produced when compiling with XeLaTeX, but no substitution has taken place:

\documentclass{article}
\usepackage[style = authoryear-comp]{biblatex}
\usepackage{fontspec}
\usepackage{filecontents}
\begin{filecontents}{\jobname.bib}
@BOOK{lennon1968,
    AUTHOR = "John Lennon",
    TITLE = "æøå",
    YEAR = "1968"}
\end{filecontents}
\DeclareSourcemap{
  \maps[datatype = bibtex]{
    \map{
       \step[fieldsource = title,
          match = {æøå}, 
          replace = {abc}]
    }
  }
}
\addbibresource{\jobname.bib}
\begin{document}
\nocite{*}
\printbibliography
\end{document}

enter image description here

share|improve this question
    
Move the \DeclareSourcemap before inputenc. –  Ulrike Fischer Apr 30 at 11:48
    
Sorry I confuse match and replace. Moving \DeclareSoureMap gets rid off the errors but the match is not replaced. –  Ulrike Fischer Apr 30 at 12:02
    
Did you try match = \regexp{æøå} - this is used to protect special chars in matches. –  PLK May 3 at 12:34
    
@PLK I tried it now, still no substitution is made. –  Sverre May 3 at 12:39
1  
This was a bug with a NFD unicode boundary. It should be fixed in 1.9 DEV version on SF now (you need to be using biblatex 2.9 with biber 1.9 which is also on SF). –  PLK May 3 at 13:42
show 1 more comment

1 Answer 1

It seems that the value given to match is passed through TeX's expansion mechanism, which is a big problem, but also a help, since we can (also selectively) use \detokenize.

\documentclass{article}
\usepackage[style = authoryear-comp]{biblatex}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}

\usepackage{filecontents}
\begin{filecontents}{\jobname.bib}
@BOOK{lennon1968,
    AUTHOR = "John Lennon",
    TITLE = "æøå",
    YEAR = "1968"}
\end{filecontents}

\DeclareSourcemap{
  \maps[datatype = bibtex]{
    \map{
       \step[fieldsource = title,
          match = {\detokenize{æøå}},
          replace = {abc}]
    }
  }
}
\addbibresource{\jobname.bib}
\begin{document}
\nocite{*}
\printbibliography
\end{document}

This is what is written in the .bcf file:

  <!-- SOURCEMAP -->
  <bcf:sourcemap>
    <bcf:maps datatype="bibtex" level="user">
      <bcf:map>
        <bcf:map_step map_field_source="title" map_match="æøå" map_replace="abc"/>
      </bcf:map>
    </bcf:maps>

Without \detokenize, the result is

  <!-- SOURCEMAP -->
  <bcf:sourcemap>
    <bcf:maps datatype="bibtex" level="user">
      <bcf:map>
        <bcf:map_step map_field_source="title" map_match="\T1\ae \T1\o <E5>" map_replace="abc"/>
      </bcf:map>
    </bcf:maps>

So it seems that \detokenize works correctly. However, the replacement doesn't take place, which seems to be a problem with Biber.

To wit, I tried the following Perl script (adapted from http://stackoverflow.com/a/1572156)

use warnings;
use strict;
use utf8;
binmode STDOUT, "utf8";
my $string = "æøå";

$string =~ s/æøå/abc/gi; 

print "$string\n";

and the result on the terminal was

abc
share|improve this answer
    
I'm with you until your Perl script. I see that your Perl script works, but how does that relate to biber and the fact that it doesn't replace the characters? –  Sverre Apr 30 at 12:19
    
@Sverre Biber is programmed in Perl. –  egreg Apr 30 at 12:23
    
Does that mean there is a solution to this ... ? –  Sverre Apr 30 at 12:23
    
@Sverre Not from me, but probably from Biber's developers. –  egreg Apr 30 at 12:24
    
In other words a bug report should be made? Presumably here: github.com/plk/biber/issues? I'd recommend that someone who knows what they're talking about (i.e. you) submit it rather than me (I would do little more than link to your answer here). –  Sverre Apr 30 at 12:26
show 1 more comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.