My current workflow is

  1. edit CiteULike database on-line
  2. wget http://www.citeulike.org/bibtex/user/MYUSER/tag/MYTAG?key_type=4
  3. biber biblio
  4. pdflatex biblio

where biblio.tex is a file for a bibliography/reference list, generated with a \nocite{*}.

The problem is that the CiteULike algorithm for generating the AuthorYearTitle key (key_type=4 in the wget URL above) may generate collisions, i. e. two different entries with the same key.

While this is for sure a CiteULike bug, I would like to patch this problem by inserting another step between wget and biber that rewrites the .bib database resolving duplicate keys, inserting a trailing 'a', 'b', ... letter after the duplicated AuthorYearTitle key. (Edit: I would like to maintain an AuthorYearTitle key in case the bibliography needs further editing.)

I would like a solution where no user intervention is required, so that steps 2., 3., 4., ... can be run in a script.

I was not able to find an existing solution to this (simple?) problem, and before reinventing the wheel I would like to ask for advice on this forum.


By omitting the key_type=4 parameters, one gets unique numeric keys (citeulike:123456), so a possible strategy is to use a more robust key generator and forget about the ones generated by CiteULike.

  • 1
    If you're willing to add another software into the loop Jabref has a duplicate resolving function
    – Elad Den
    May 10, 2017 at 15:03
  • @EladDen is it possibile to use jabref duplicate resolution from the command line?
    – Stefano M
    May 10, 2017 at 15:58
  • Not that I am aware of. Also, Jabref does ask you which of the two versions you wish to keep for each duplicate it finds (I know this might not be what you're looking for, therefore I did not put it in an answer)
    – Elad Den
    May 10, 2017 at 16:51

1 Answer 1


Since you are using \nocite{*}, you do not need the bib tags to represent anything meaningful. So an easy solution would very simply replace all tags in the .bib file by some random character sequence (or append one, see below). For me, this worked:

sed -i -e 's~^@\(.\+\){.\+,$~(echo @\1{; cat /dev/urandom | tr -dc "a-zA-Z0-9" | fold -w 8 | head -1; echo ,) | paste -sd ""~e' biblio.bib

What does this do?


well, call sed


do inline-replacement


use extended regular expressions (for capture-group and replacement)


begin sed command




substitution start delimiter


Search for start of line, @, a group of non-whitespace followed by {, another group of non-whitespace, ,, end of line; e.g., @article{AuthorYear,


substitution middle delimiter

(echo @\1{; cat /dev/urandom | tr -dc "a-zA-Z0-9" | fold -w 8 | head -1; echo ,) | paste -sd ""

Replace by

  1. echo @, the first non-whitespace group captured earlier, {
  2. echo one random tag (cat through head)
  3. echo ,

(After removing newlines in between with paste)


substitution end delimiter


call the replacement as a shell command and use output as replacement text


end sed command



Update 1 If you want to keep the original tags, just capture them and append the random tag:

sed -i -e 's~^@\(.\+\){\(.\+\),$~(echo @\1{\2_; cat /dev/urandom | tr -dc "a-zA-Z0-9" | fold -w 8 | head -1; echo ,) | paste -sd ""~e' biblio.bib

Update 2 Instead of replacing by a random tag, this one add a, b, ... to the tags that were seen before; so the first occurrence of "AuthorYear" remains unchanged, the second becomes "AuthorYeara", and so forth.

echo > cache
sed -i -e 's~^@\(.\+\){\(.\+\),$~(echo @\1{\2; awk "BEGIN{printf \\\"%c\\\", 96+`grep \2 cache | wc -l`}" | sed "s/\\\`//"; echo \2 >> cache; echo ,) | paste -sd ""~e' biblio.bib

Can be changed to make the second "b", of course:

echo > cache
sed -i -e 's~^@\(.\+\){\(.\+\),$~(echo @\1{\2; awk "BEGIN{printf \\\"%c\\\", 97+`grep \2 cache | wc -l`}" | sed "s/a//"; echo \2 >> cache; echo ,) | paste -sd ""~e' biblio.bib
  • I would prefer to maintain the AuthorYearTitle keys, for later editing. Sorry for not having mentioned this in the question.
    – Stefano M
    May 5, 2017 at 8:02
  • @StefanoM easy, see last update.
    – bers
    May 5, 2017 at 8:08
  • I would like something more "stable": key disambiguation should be done only in case of collisions and two subsequent runs of the script should generate the same .bib database.
    – Stefano M
    May 5, 2017 at 8:20
  • @StefanoM stability is easily achieved by piping /dev/urandom to a cache file once and using that cache file in subsequent runs instead of /dev/urandom; this way, two subsequence runs of the script will generate the same .bib file. On the other hand, I have no idea what "key disambiguation should be done only in case of collisions " has to do with "stability".
    – bers
    May 5, 2017 at 8:56
  • @StefanoM "Update 2" solves "key disambiguation should be done only in case of collisions".
    – bers
    May 5, 2017 at 9:20

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .