Merge two BibTeX files

Question

I was constructing a BibTeX file for my Thesis. At the middle of that I had copied that bib file to another document I had to produce, I had to add some new citations to the copy of the original.

Now I have 2 bib files that have a lot in common but differ in some citations. Is there a way to merge them? So that I have one single file with all the citations from both files but no duplicates?

Answer 1

I think there is no easy way to do this. Though it is possible to write a perl script alike to use regex to do this. But the problem is

1) Sometimes two entries are duplicates though they are not exactly the same, for example different capitalization in titles or extra {and }. So we have to workaround this by define some threshold for difference.

2) Sometimes two entries from the same author of the same year with a little difference in the title are indeed two papers. This make the threshold in 1) hard to define.

So I suggest you can only do this by hand. A relative easier way is do it with a BibTeX editor, for example BibDesk on Mac OS X. Import both files into it, and it shall warn you when two entries have the same citekey. So for those entries with same citekeys, which are almost surely duplicates, you can clean them up easily. If I remember correctly BibDesk and other editors do have this functionality. After this, sort all entries by title, and spot further duplicates by your eyes.

Answer 2

bibtool -s bibliography1.bib bibliography2.bib should do the trick. Bibtool is on CTAN. Not to be confused with bibtools which is also on CTAN and probably also has the capacity to do this sort of thing...

Answer 3

JabRef is a bibliography manager that has tools for merging bib-files. Or rather, for importing one file into another: File --> Import into current database. Not sure if it checks for duplicates on merge, but there is a tool for checking for duplicates as well: Tools --> Scan database --> Find duplicates.