Redundancy in bib file: conditionally suppress url if same as doi?

Question

Mathscinet gives bibtex entries with both a doi and a url field included. Typically they look something like

@article {some-article,
   […] 
   DOI = {10.1016/j.apal.2008.12.003},
   URL = {http://dx.doi.org/10.1016/j.apal.2008.12.003},
}

I'm using amsalpha.bst, modified with urlbst to process both these fields. Of course, this means that they both appear, completely redundantly, in my bibliography!

My question is: what's the principled way to deal with this? Here are the options I've thought of so far:

• Modify the bib file so it checks for this redundancy, and doesn't typeset the url in such cases. This seems ideal if it's possible; has anyone already done something like this? If not, is it likely to be doable by someone with a little programming experience but no existing understanding of .bst files?

• Use a bibstyle that typesets one of url and doi but not both. Not ideal: other bib entries might have only one but not both, or might have a url different from the doi.

• Comment out (or delete) the `url' field by hand, in the bib file, in these cases. This is what I'm currently doing. Seems a little clunky; also, somewhat violates “separation of form from content”: having the fields the same is correct as content, it’s just inappropriate for them then to both be typeset.

Related question: How to get DOI links in bibliography

Answer 1

The modifications made by urlbst are quite clear, so the change you want is actually not too hard (by BibTeX standards). If you open up your .bst files, you need to search for a function called output.web.refs. It needs modifying to read

FUNCTION {output.web.refs}
{
  new.block
  output.url
  addeprints eprint empty$ not and
    { format.eprint output.nonnull }
    'skip$
  if$
  adddoiresolver doi empty$ not and
    { 
      url empty$
        { format.doi output.nonnull }
        {
          doiurl doi * url =     
            'skip$
            { format.doi output.nonnull }
          if$  
        } 
      if$  
    }
    'skip$
  if$
  addpubmedresolver pubmed empty$ not and
    { format.pubmed output.nonnull }
    'skip$
  if$
}

All that has happened here is that I've added a test for an empty URL and a second for the URL being the same as the DOI once the prefix is added.

Answer 2

The original question is about a standard BibTeX style, but inspired by lockstep I've worked out a biblatex solution as a complement to my other answer. The method here is to add the appropriate prefix to the raw DOI, then test this for equivalence to the URL. The prefix needs to be processed with \detokenize as this is how the URL field is formatted.

\documentclass{article}

\usepackage{biblatex}

\renewbibmacro*{doi+eprint+url}{%
  \iftoggle{bbx:doi}
    {%
      \iffieldundef{doi}
        {}
        {%
          \begingroup
          \edef\URLorDOI{%
            \detokenize{http://dx.doi.org/}%
            \thefield{doi}%
          }%
          \iffieldequals{url}{\URLorDOI}
            {\endgroup}
            {%
              \endgroup
              \printfield{doi}%
            }%  
        }%
    }
    {}%
  \newunit\newblock
  \iftoggle{bbx:eprint}
    {\usebibmacro{eprint}}
    {}%
  \newunit\newblock
  \iftoggle{bbx:url}
    {\usebibmacro{url+urldate}}
    {}}

\usepackage{filecontents}

\begin{filecontents}{\jobname.bib}
@misc{A01,
  author = {Author, A.},
  year = {2001},
  title = {Alpha},
  doi = {10.1016/j.apal.2008.12.003},
}

@misc{B02,
  author = {Buthor, B.},
  year = {2002},
  title = {Bravo},
  url = {http://dx.doi.org/10.1016/j.apal.2008.12.003},
  urldate = {2010-11-22},
}

@misc{C03,
  author = {Cuthor, C.},
  year = {2003},
  title = {Charlie},
  doi = {10.1016/j.apal.2008.12.003},
  url = {http://dx.doi.org/10.1016/j.apal.2008.12.003},
  urldate = {2010-11-22},
}
\end{filecontents}

\bibliography{\jobname}

\begin{document}

\nocite{*}

\printbibliography

\end{document}

Answer 3

I didn't manage to test for "sort-of-identical" doi und url fields, but here's a solution using biblatex - url and urldate fields will only be typeset if the respective entry doesn't include a doi field.

\documentclass{article}

\usepackage{biblatex}

\DeclareFieldFormat{url}{%
  \iffieldundef{doi}{%
    \mkbibacro{URL}\addcolon\space\url{#1}%
  }{%
  }%
}

\DeclareFieldFormat{urldate}{%
  \iffieldundef{doi}{%
    \mkbibparens{\bibstring{urlseen}\space#1}%
  }{%
  }%
}

\usepackage{filecontents}

\begin{filecontents}{\jobname.bib}
@misc{A01,
  author = {Author, A.},
  year = {2001},
  title = {Alpha},
  doi = {10.1016/j.apal.2008.12.003},
}

@misc{B02,
  author = {Buthor, B.},
  year = {2002},
  title = {Bravo},
  url = {http://dx.doi.org/10.1016/j.apal.2008.12.003},
  urldate = {2010-11-22},
}

@misc{C03,
  author = {Cuthor, C.},
  year = {2003},
  title = {Charlie},
  doi = {10.1016/j.apal.2008.12.003},
  url = {http://dx.doi.org/10.1016/j.apal.2008.12.003},
  urldate = {2010-11-22},
}
\end{filecontents}

\bibliography{\jobname}

\begin{document}

\nocite{*}

\printbibliography

\end{document}

Answer 4

This is really a comment to Joseph Wright's answer above, but with included code.

I only just now stumbled across this answer (I'm the author of urlbst).

This is great, Joseph -- thanks. I've incorporated something rather like this into the urlbst repository at https://bitbucket.org/nxg/urlbst, and this should appear in the next release of urlbst (whenever that is).

FUNCTION {output.web.refs}
{
  new.block
  inlinelinks
    'skip$ % links were inline -- don't repeat them
    { % If the generated DOI will be the same as the URL,
      % then don't print the URL (thanks to Joseph Wright for this code,
      % at http://tex.stackexchange.com/questions/5660)
      adddoiresolver 
          doiurl doi empty$ { "X" } { doi } if$ * % DOI URL to be generated
          url empty$ { "Y" } { url } if$          % the URL, or "Y" if empty
          =                                       % are the strings equal?
          and
        'skip$
        { output.url }
      if$
      addeprints eprint empty$ not and
        { format.eprint output.nonnull }
        'skip$
      if$
      adddoiresolver doi empty$ not and
        { format.doi output.nonnull }
        'skip$
      if$
      addpubmedresolver pubmed empty$ not and
        { format.pubmed output.nonnull }
        'skip$
      if$
    }
  if$
}

The difference is that this avoids printing the URL if it would equal the DOI, rather than the other way round, and also checks if either or both is empty.

For what it's worth, my response to the initial problem would be to delete the semi-redundant URL field in the .bib file. I would only have both if the URL gave an alternative (possibly openly available) version of the reference.

Answer 5

When using BibLaTeX, this can be very easily done on the data level, without fuddling with output templates or even changing the .bib file.

Just use the \iffieldundef{field name}{undefined case}{defined case} macro:

\AtEveryBibitem{%
  \iffieldundef{url}{}{\clearfield{doi}}%
}

Note that this doesn't do the if DOI and URL are the same test as suggested by the OP, but just the simpler only show DOI when there's no URL case as some of the other answers.

Answer 6

A quick solution, without BST manipulation, can be a sed script that deletes the URL field if followed by a DOI field, or similar. Worked for me; I have a bib library of dblp bib entries from which I extract the relevant ones with bibtool, and then delete the spurious URL fields with sed.

Here is my sed script, acknowledging https://unix.stackexchange.com/questions/26284/how-can-i-use-sed-to-replace-a-multi-line-string for the magic incantation.

# First, a magic incantation to turn sed from a line-based editor
# to an editor on the whole input.

1h;2,$H;$!d;g

# Then, the actual command, replacing url=A,doi=B by doi=B

s/\s*url\s*=\s*{[^}]*},\(\s*doi\s*=\s*{[^}]*}\)/\1/g

# Note that \s matches a whitespace character (including \n).