11

Introduction

Mendeley's desktop software has the ability to export the references to a bibtex compatible file for use in TeX based documents.

However when exporting, Mendeley escapes all special characters, in particular, underscore characters. For the most part this is fine, but this is also done in the 'url' field of the bibliography entry and can cause output errors. Conversely, this escaping step can be disabled but causes many more problems than it solves.

The Problem

The desired result is to have hyperlink equivalent to:

http://www.example.com/content/news/sep_09/12.html

However, the bibliography uses a url value of

http://www.example.com/content/news/sep{\_}09/12.html

which is retrieved from the external .bib file. This is then escaped by \url (which escapes all HTML special characters) such that the link is now:

http://www.example.com/content/news/sep%7B%5C_%7D09/12.html

What I have tried

I have considered using Biber to preparse the file as demonstrated here by npdoty. I have included what I think is the equivalent of this below as the MWE adapted from another relevant substitution from moewe's answer here.

Regex in use: {\\_}

Have tried: {\\\_}, \{\\\_\}

The potential answers

  • Is my problem as simple as having an incorrect regex?
  • Is biber doing some parsing when reading the file in and thus I am no longer matching {\_}?.
  • Is there a biber/biblatex switch that allows URLs to be latex-escaped rather than verbatim?

MWE

\RequirePackage{filecontents}
\begin{filecontents*}{\jobname.bib}

@misc{enc2015,
    author = {{Example News Company}},
    title = {{Daily News for September 9 2015}},
    url = {http://www.example.com/content/news/sep{\_}09/12.html},
    urldate = {2016-03-08},
    year = {2015}
}
\end{filecontents*}

\documentclass{article}

% ---- Bibliography Settings ----
\usepackage{csquotes} % Babel bibliography support package
\usepackage[english]{babel}
\usepackage[backend=biber,style=numeric]{biblatex}
\usepackage{hyperref}

\addbibresource{\jobname.bib}

\DeclareSourcemap{
  \maps{
    \map{
      \step[fieldsource=url,
            match=\regexp{\{\textbackslash\textbackslash\textunderscore\}},
            replace=\regexp{\textunderscore}]
    }
  }
}

\begin{document}
\nocite{enc2015}
\printbibliography
\end{document}

Footnotes

If Mendeley find this, a relevant bug report/suggestion is Mendeley Forums: Suggestion #2088193.

  • Firefox tells me not to follow your link, so I cam back. This is clearly a bug in Mendeley's export software. I'd use sed to fix it, probably. Or not use Mendeley. – cfr May 17 '16 at 1:20
  • @cfr haha, its not a real link. It's just an example that shows what the link looks like when you hover over it and proves that it doesn't actually work either. – samthecodingman May 17 '16 at 3:18
13

After further investigation and head scratching I have reached a working solution.

Updated MWE

\RequirePackage{filecontents}
\begin{filecontents*}{\jobname.bib}

@misc{enc2015,
    author = {{Example News Company}},
    title = {{Daily News for September 9 2015}},
    url = {http://www.example.com/content/news/sep{\_}09/12},
    urldate = {2016-03-08},
    year = {2015}
}
\end{filecontents*}

\documentclass{article}

% ---- Bibliography Settings ----
\usepackage{csquotes} % Babel bibliography support package
\usepackage[english]{babel}
\usepackage[backend=biber,style=numeric]{biblatex}
\usepackage{hyperref}

\addbibresource{\jobname.bib}

\addbibresource{library.bib}
\DeclareSourcemap{ % Used when .bib/Bibliography is compiled, not when document is
    \maps{
        \map{ % Replaces '{\_}', '{_}' or '\_' with just '_'
            \step[fieldsource=url,
                  match=\regexp{\{\\\_\}|\{\_\}|\\\_},
                  replace=\regexp{\_}]
        }
        \map{ % Replaces '{'$\sim$'}', '$\sim$' or '{~}' with just '~'
            \step[fieldsource=url,
                  match=\regexp{\{\$\\sim\$\}|\{\~\}|\$\\sim\$},
                  replace=\regexp{\~}]
        }
        \map{ % Replaces '{\$}'
            \step[fieldsource=url,
                  match=\regexp{\{\\\x{26}\}},
                  replace=\regexp{\x{26}}]
        }
    }
}

\begin{document}
\nocite{enc2015}
\printbibliography
\end{document}

Reasoning

So in comparison to what is written above in the question, this version uses direct escaping of the characters. i.e. \\ and \_ instead of \textbackslash and \textunderscore. I originally tried this but it didn't work, now I must have done something slightly differently.

A theory I have is that this is because \t is converted to the TAB character by \regexp resulting in searches for <tab>extbackslash and <tab>extunderscore instead of the \ and _ characters.

Summary

For those who are having trouble with escaped characters in URLs with Mendeley, by using Biber & BibLaTeX, the following snippet can be used to unescape those pesky _, ~ and & characters.

In the version of Mendeley I am using (v1.16.1) these characters are escaped as {\_} and {~}. However I have included support for older \_ and $\sim$ escape sequences reported in other questions and around the internet while searching for a solution.

% This snippet must be in the preamble.
\usepackage[english]{babel} % untested with other languages
\usepackage[backend=biber]{biblatex}
\usepackage{hyperref} % for clickable urls

\addbibresource{library.bib} % Mendeley BibTeX library

\DeclareSourcemap{
    \maps{
        \map{ % Replaces '{\_}', '{_}' or '\_' with just '_'
            \step[fieldsource=url,
                  match=\regexp{\{\\\_\}|\{\_\}|\\\_},
                  replace=\regexp{\_}]
        }
        \map{ % Replaces '{'$\sim$'}', '$\sim$' or '{~}' with just '~'
            \step[fieldsource=url,
                  match=\regexp{\{\$\\sim\$\}|\{\~\}|\$\\sim\$},
                  replace=\regexp{\~}]
        }
        \map{ % Replaces '{\$}'
            \step[fieldsource=url,
                  match=\regexp{\{\\\x{26}\}},
                  replace=\regexp{\x{26}}]
        }
    }
}

Hope this helps someone else dealing with this problem.

2

It seems in Mendeley 1.17.2 one can now disable the escaping horror. Disabling this field worked for me...

enter image description here

  • This option was present in the Mendeley version I was using (v1.16.1) and unfortunately it didn't solve the issue at the time. This is the option I vaguely referenced with "...this escaping step can be disabled but causes many more problems than it solves." in the original question. However I am yet to test if the latest version has changed the way it handles things 'under the hood'. The solution marked as the current answer just accepts that the export has escaped characters and deals with it accordingly. – samthecodingman Nov 14 '16 at 16:18
  • Sorry I did not properly read your question then. In my case it solved the problem. Maybe I will run into others soon... – n1000 Nov 14 '16 at 16:20
  • For people who find this question in the future, this solution does seem to work (as of 2017). I don't know if it retroactively applies to already created references however, so you might have to re-edit any references which you want to fix in Mendeley for it to apply. – Ruvi Lecamwasam Oct 5 '17 at 1:02
  • 1
    @RuviLecamwasam November 2017 still not working. As samthecodingman says: not really an option to turn off everything. Creates more problems, than it solves. – Dschoni Nov 23 '17 at 12:20
2

It's better to let Mendeley use braces to escape such symbols. In my case, I had problems with the URLs in the references using a Springer Template in Latex, so I came up to process the references file (henceforth, ref.bib) and replace such bad usages of the braces, specifically, the fields affected were like url or doi.

The solution was:

$ sed --version
sed (GNU sed) 4.2.2
$ cat ref.bib
...
url = {http://link.springer.com/10.1007/3-540-48256-3{\_}21}, 
...
$ sed -i.backup "/^(url|doi)/ s/{.\?\(.\)}/\1/g" ref.bib
$ cat ref.bib
...
url = {http://link.springer.com/10.1007/3-540-48256-3_21}, 
...

I hope it helps. If something needs clarification, just let me know.

  • I use a more explicit version (which for non-sed-users is probably a little more readable): sed -i.backup '/^url/ s|{\\_}|_|g' ref.bib – Dschoni Nov 23 '17 at 15:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.