5

My question is similar to How to sort a .bib file with biber?

I have a .bib file which is generated by Mendeley. Because it is generated, the entries are not formatted nor ordered. Also the order changes completley whenever I add something in Mendeley.

Because I use git to track changes, I want to have a sorted and formatted bib file. This would lead to nice diffs/commits where one could easily see what has changed.

I got the formatting part, but I am not able to make biber sort the file. For my task it is irrelevant what the sorting parameter is, as long as sorting two different files with the same entries leads to the same sorting. So lets assume we want to sort according to the citation key.

What I got so far is:

biber -tool C:\SomePath\unordererd.bib --output_indent=4 --output_fieldcase=lower --output-file=C:\AAAA\SomePath\output.bib

This gives me a nice format, but the entries are not sorted.

This is the output i get

INFO - This is Biber 2.3 running in TOOL mode
...
INFO - Overriding locale 'en_US' defaults 'variable = shifted' with 'variable = non-ignorable'
INFO - Overriding locale 'en_US' defaults 'normalization = NFD' with 'normalization = prenormalized'
INFO - Sorting list 'none' of type 'entry' with scheme 'none' and locale 'en_US'
INFO - No sort tailoring available for locale 'en_US'
...

Obviously, biber thinks about sorting, but does nothing about it. I also tried providing a biber.conf like in the other question, but with no success.

Solution

If someone has the same issue, this is my final config file. It also includes some filter to clean up the file:

  • it removes abstract, file, keywords and mendeley-tags fields from every entry
  • it removes url fields from some entries
<config>
  <sorting>
    <presort>mm</presort>
    <sort order="1">
      <sortitem order="1">title</sortitem>
    </sort>
    <sort order="2">
      <sortitem order="1">year</sortitem>
    </sort>
  </sorting>
  <sourcemap>
    <maps datatype="bibtex" map_overwrite="1">
        <map>
            <map_step map_field_set="ABSTRACT" map_null="1"/>
            <map_step map_field_set="FILE" map_null="1"/>
            <map_step map_field_set="keywords" map_null="1"/>
            <map_step map_field_set="mendeley-tags" map_null="1"/>
        </map>
        <map>
            <per_type>ARTICLE</per_type>
            <per_type>BOOK</per_type>
            <per_type>inproceedings</per_type>
            <per_type>incollection</per_type>
            <map_step map_field_set="URL" map_null="1"/>
        </map>    
    </maps>
  </sourcemap>  
</config>

I also added these two parameters to the commandline:

--output_indent=4 --output_fieldcase=lower
  • There are other tools, for example bibtool. Never used it myself, but from a quick look at the manual bibtool -s unordered.bib -o ordered.bib should sort by citation key I think. – Torbjørn T. Mar 18 '16 at 10:25
  • from the doc bibtool seems to be suitable, but only source files are provided and no binary is available. it is also not available in any larger latex distribution (Miktex or Tex Live). This would mean I have to provide it to every plattform I use. Also compiling it is not trivial since is has makefiles with many hardcoded paths (bad in windows) or it needs root rights in linux. .. on the other hand: biber is a quasi standard and available everywhere. – masgo Mar 18 '16 at 11:15
  • Yes, I see that is inconvenient. (I didn't really think about such problems at all.) – Torbjørn T. Mar 18 '16 at 11:25
  • when you mentioned it, I remembered that I had already tried it, but it prooved to be too inconvenient. Also I was only able to compile it in linux. But still, thank you for participating. – masgo Mar 18 '16 at 11:31
  • 1
    biber --tool should work, but there seems to have been a bug: github.com/plk/biber/issues/113 – Ulrike Fischer Mar 18 '16 at 21:53
6

Actually with biber 2.3, which is what you are using, it should work with a suitable <sorting> specification in your biber.conf. There was a bug in 2.4, fixed in the 2.5 dev version currently on Sourceforge. As your log says, the default sorting in tool mode is none because people don't always want to change the order of things in their data sources if they are nicely commented etc.

Try something like this in your .conf file (pass the file name with --configfile=<file>

<config>
  <sorting>
    <presort>mm</presort>
    <sort order="1">
      <sortitem order="1">author</sortitem>
    </sort>
    <sort order="2">
      <sortitem order="1">year</sortitem>
    </sort>
  </sorting>
</config>

This would be the full commandline:

biber --tool unsorted.bib --output-file=sorted.bib --configfile=biber-sorting.conf

If the files are not in the current folder, you need to specify the path correctly. If you name the config file biber.conf, biber will use it even without the --configfile parameter as this is the default.

  • Can you show me a suitable <sorting> specification? I tried the one from the other question without success. As stated, I do not really care about the sort key, so pick the simplest you can think of. – masgo Mar 24 '16 at 10:13
  • Thank you, this worked with slight changes. When using author as sortitem I got the following error: Can't use an undefined value as a HASH reference at [...]/Biber/Internals.pm line 1314. This might be because some Author names contain special characters like ö and ß. (Which, by the way are escaped correctly by Mendeley as \"{o} and {\ss} and are converted by biber to ö and ß). But when I use title as the first sortitem the sorting works. (it still works, even when I put special charaters in the title!?) – masgo Mar 24 '16 at 11:21
  • Oh, and althou the soring works I get this in the biber output: INFO - Sorting list 'none/global' of type 'entry' with scheme 'none' and locale 'en_US' INFO - No sort tailoring available for locale 'en_US' – masgo Mar 24 '16 at 11:23
  • Are you sure you are using biber 2.3? Special characters are no problem anywhere, biber handles full UTF-8. Those messages are also normal but (just 'INFO' level). – PLK Mar 24 '16 at 12:17
  • I think you are using 2.4 as that's the exact error reported in the issue linked above. You will probably need to get the 2.5 DEV version from Sourceforge, along with biblatex 3.4 DEV version. – PLK Mar 24 '16 at 12:43
4

For anyone using a more recent version of biber, the element names have changed slightly. As of biber 2.11, sorting is now sortingtemplate name="tool". This config file works for me:

<config>
  <sortingtemplate name="tool">
    <sort order="1">
      <sortitem order="1">year</sortitem>
    </sort>
    <sort order="2">
      <sortitem order="1">author</sortitem>
    </sort>
    <sort order="3">
      <sortitem order="1">title</sortitem>
    </sort>
  </sortingtemplate>
  <sourcemap>
    <maps datatype="bibtex" map_overwrite="1">
        <map>
            <map_step map_field_set="ABSTRACT" map_null="1"/>
            <map_step map_field_set="FILE" map_null="1"/>
            <map_step map_field_set="keywords" map_null="1"/>
            <map_step map_field_set="mendeley-tags" map_null="1"/>
        </map>
        <map>
            <per_type>ARTICLE</per_type>
            <per_type>BOOK</per_type>
            <per_type>inproceedings</per_type>
            <per_type>incollection</per_type>
            <map_step map_field_set="URL" map_null="1"/>
        </map>    
    </maps>
  </sourcemap>  
</config>

You can run with this command:

biber --tool --output-fieldcase=title --output-indent=4 --output-align --output-file=bibliography.bib --configfile=bib-cleanup.conf --validate-config MyCollection.bib

(--validate-config will assert that your config file passes the Relax NG schema for the config file.)

0

With biber 2.12 I needed to map the journal field to journaltitle. Otherwise the field completely disappeared in the output file. Here's my full config file, based on the one posted by arprice.

<config>
  <sortingtemplate name="tool">
    <sort order="1">
      <sortitem order="1">year</sortitem>
    </sort>
    <sort order="2">
      <sortitem order="1">author</sortitem>
    </sort>
    <sort order="3">
      <sortitem order="1">title</sortitem>
    </sort>
  </sortingtemplate>
  <sourcemap>
    <maps datatype="bibtex" map_overwrite="1">
        <!-- Remove fields -->
        <map>
            <map_step map_field_set="ABSTRACT" map_null="1"/>
            <map_step map_field_set="FILE" map_null="1"/>
            <map_step map_field_set="keywords" map_null="1"/>
            <map_step map_field_set="mendeley-tags" map_null="1"/>
           <map_step map_field_set="issn" map_null="1"/>
        </map>
        <!-- Map journal to journaltitle -->
        <map>
            <map_step map_field_source="journal" map_field_target="journaltitle"/>
        </map>
        <!-- Remove URL for specific types -->
        <map>
            <per_type>ARTICLE</per_type>
            <per_type>BOOK</per_type>
            <per_type>inproceedings</per_type>
            <per_type>incollection</per_type>
            <map_step map_field_set="URL" map_null="1"/>
        </map>
        <!-- Map annote to note -->
        <map>
            <map_step map_field_source="annote" map_field_target="note"/>
        </map>
    </maps>
  </sourcemap>  
</config>
New contributor
lvdgraaff is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.