6

Let's say I have one and only one bibliography file on my computer, in which I write every piece of literature I read and that I import in every paper I write or on what my PhD thesis will be at the end. \nocite{*} allows me to do that without putting uncited literature in my document. I also use Git for version control of my LaTeX documents.

What I would like to do is to have a new Git repo for my bibliography file and for that file only, which I could ideally pull everytime I need it, update it, use it and push it back to the repo as updated version.

My questions are:

  1. is this a good idea?
  2. once I set up the repo for my bibliography, how do I pull the file in my paper's repo without having two repos in the same folder at the same time? I wouldn't want to refer to any absolute path, as this would make my documents not portable

The bibliography document is supposed to be updated within the paper repo and then pushed to both the paper repo and to the bibliography repo

  • Which OS are you using? Sounds like a symbolic link could help. Or you have the files in two subdirectories (sa, sb) of the same parent directory (p) and reference the bibliography as "../sb/bibliography.bib" from documents that are in "sa" – riddleculous Mar 3 '16 at 9:38
  • I am using Ubuntu. I thought about symbolic links, but I discarded that idea (see my edited question). About the second suggestion (subdirectories), how does git behave with repos inside repos? Because I may need to include my bibliography file in the paper repo as well. – fmonegaglia Mar 3 '16 at 9:44
  • 4
    With Biber you can pull a .bib directly from a remote source (also from a github repo). So there would be no need to have the .bib in the folder at all. (BTW: I think the general question about how repositories work might be slightly off topic here, I'm quite confident that some people here know the answer, but still ...) – moewe Mar 3 '16 at 9:50
  • I see, but even though the question was pretty much about git, I wanted some feedback on my idea of having a repo for the bibliography from some latex user. Moreover, I use bitbucket and my repo is private. And, some journals still use bibtex, which does not allow for remote access. – fmonegaglia Mar 3 '16 at 9:56
  • 1
    @moewe the thing is this is a X/Y question. It looks like a question about git, but it is actually a question about the local texmftrees. But if the asker had known it was about the texmftree, then they would already know the answer. (P.S. This is an awesome question for a starting PhD student and I wish I had asked it 12 months ago) – Lyndon White Mar 3 '16 at 10:18
3

I use the approach detailed in http://andrius.velykis.lt/2012/06/master-bibtex-file-git-submodules/ for my own work.

So I have one repository "bib" that contains my BibTeX files. For this repository, I also have a private remote at BitBucket for convenience and portability.

In each of my papers and thesis, I use git submodule add <bitbucket-URL> biblio to make a submodule directory with my document. In you document you can then just specify to use the BIB files in that directory.

Typically, I try to only edit my main BIB repository (that's the only one that is loaded within JabRef for me), push those changes to bitbucket and then pull in the changes from within each document repository. However, in some cases I just want to change e.g. how the authors are formatted, don't show URLs in the bibliography for a paper, ... The proper way is to tinker with BST files and the like, but in the heat of the moment it's often a lot faster to just tweak the "biblio" module of a paper.

I think there are a few advantages to this general approach:

  • your bibliography is versioned, so you can easily track what exact version of a BIB file you used,
  • you can transfer changes from your main "bib" repository to each document, or the other way around,
  • you can tweak the bibliography that are specific to one document and not push them back to the main repository,
  • everything can be done using relative paths, so, it is portable across computers, users, OSes, ...

But also some disadvantages:

  • When you add many references, you end up pushing and pulling quite often: push from the main one, pull in the document.
  • Git submodules make your document repository – and how to work with it – more complicated. So you should already feel comfortable with git before you use this a workflow.

Remark about network access While my approach normally relies on bitbucket, I also like to add another remote in the submodule. That extra remote points to the location of my main bibliography repository. This allows me to also push/pull to the main repository when I don't have network access. That extra remote might not migrate well across computers (due to different file paths), but since it's just there "in case of no network", I don't mind it too much.

Remark about submodule In the git world there has been a long-standing discussion whether submodule is a good thing or not. The alternative is to use git subtree, I don't use it, but I suppose the concept of this workflow can be adapted to use subtree instead of submodule.

| improve this answer | |
  • If I collaborate with a group of people in writing a paper, the Bibtex file will be changed not because of content changes but because of reformatting due to various bibtex software people are using. Like white space, line ending and comments from those bib software. Is there any way to filter out those changes and commit only the content changes on the submodule? – Xiaodong Qi Sep 29 '16 at 1:51
  • I would recommend all to use the same software (e.g. JabRef works well with source control). If that is not wanted, git has so-called smudge and clean filters that exist exactly for such a use-case. However, you will either have to search for good bibtex filters or implement them yourself. – Egon Sep 29 '16 at 8:53
  • Any recommendation on the good bibtex filters? Now I can confirm the main issue I have encountered is the serialization of fields in a bibtex entry. I have to work with different versions of JabRef (because the Docear I use only has JabRef v2.7 embeded), and the the ordering of entries have been changed dramatically in the newer versions without the ability to adjust them. I haven't found any filters that can solve this problem. Thank you. – Xiaodong Qi Sep 30 '16 at 6:50
  • Sorry, I don't have any good bibtex filters laying arounds. You might have to roll your own, but e.g. unix.stackexchange.com/questions/31266/… could help – Egon Sep 30 '16 at 6:56
4

Sure. Its almost what I do.

In answer to sub Q1: this is an excellent idea (even if I do say so myself)

In answer to sub Q2: ...

Put the .bib in your Local texmftree

What you would like is a local texmf tree, which in in one git repo. A local texmf tree is for putting various package-like-artifacts, that are not proper packages managed through your package manager (e.g. Miktex in windows).

The local texmf tree means that you never have to use absolute, or even relative paths for your bibliography. Its also very useful as many conferences and journals distribute their own templates and styles not through CTAN, but just as .sty and .bst files.

How to setup a local tex tree is beyond the scope of this answer but it isn't hard. See the following questions:

Once you have created that tree, add it to git. Put your master.bib file in localtexmf\bibtex\bib\master_bibliography\master.bib. (I also have my JabRef XML setting in this directory, and a .tex file to make an annotated bibliography)

Then when you go to work in a project you just add \bibliography{master} and it will find it. Better than an absolute or a relative path.

Then when ever you work on a new computer check out the repo from git, and tell your tex distribution to know about the local texmf tree.


Put the Local texmf tree in Git

Now where to put it.

One repo for all work, including the texmftree

I just have one git rep for all my PhD work, with a bunch of folders. Its not a great system, but it is simple. So my texmf tree is in phd\Resources\tools\localtexmf\. Where as my papers are in phd\documents_prepared\Journal Papers\PaperName. My experimental prototypes are in \phd\prototypes\ProjectName. Often I'll have the prototype dump out its data into a CSV, into the a data folder within the Paper folder, then I can produce plots with PGFPlots. (its taken a while for my tex foo to reach that level, but now that it has, it is good)

That's easy and simple. and since it is all on repo, you don't need to worry about committing multiple times.

Keep a git repo for your current work, and a separate for the texmf tree

This is not too complex. There is no need to nest one in the other, as you are not using relative paths or anything (There is also no need not to; see next section). So that is fine. You just need to remember to commit and pull both. You could script that.

Use Git sub-modules to nest them

You can have git repos in git repos. This is a sub-module (Documentation) . I don't particularly see the advantage, except that it does mean someone can just checkout your paper repo, and have it pull the texmf tree git repo as a submodule.

The disadvantage is now you have 1 texmf tree per paper repo, locally. But still only one in git. But keeping them both synced when you are working on multiple projects seems unfun.

git sub-modules don't work the way you might think they do; and for this I don't like them. They might not the least intuitive thing in git, but that is a very competitive playing field (:-P). This post goes into some details. I suggest doing a bit of research before doing this.


P.S. You can use the Bibtex Annote field to store annotations. If you are using a tool like Jabref, that helps to search and index and generally GUI your .bib file, you may need to enable Annote on all Bibentry types.

| improve this answer | |
  • Alternative to sub-modules: subtrees. – Paul Gaborit Mar 3 '16 at 10:33
  • 1
    Actually, isn't using a local texmf tree as unportable as using absolute paths? The same way I shall instruct latex where to find files, I have to instruct it where the texmf tree is, if there is one on the other computer, and if not, I would have to build it – fmonegaglia Mar 3 '16 at 10:38
  • No, because you don't have to edit your latex file to point to the path on each computer (which would result in having to reedit the absolute path each time you swapped computer). Just once per computer you checkout your local texmf tree, and tell you Miktex/TexLive (etc) where the local texmf tree you just created is. Absolute is per change in computer, but local texmf is once per computer ever, Does this clarify the difference? – Lyndon White Mar 3 '16 at 10:43
  • 2
    Well it could become an issue when you start sharing papers with other contributors. You don't really want them to have to use your texmf tree as they may have their own. in which case having the bibliography in the paper tree may be a better idea, but linking it to your main bibliography repo my be tricky. – ArTourter Mar 3 '16 at 12:05
  • Miktex lets you have as many texmf trees as you want, idk about tex live. But yes, if you were collaborating things can be come harder. (one option would be the actual, two independent repos in one folder approach) – Lyndon White Mar 3 '16 at 15:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.