5

I have exported the bibliography from Zotero in biblatex format. All the references contain a date field, but now year field.

Is there a simple way (a tool, a script etc.) to convert the date field or to append year and month fields?

Example:

@article{kattan_timeseries_2015,
  title = {Time-series event-based prediction: An unsupervised learning framework based on genetic programming},  
  journaltitle = {Information Sciences},
  author = {Kattan, Ahmed and Fatima, Shaheen and Arif, Muhammad},
  date = {2015-04}
}

and I want to be

@article{kattan_timeseries_2015,
  title = {Time-series event-based prediction: An unsupervised learning framework based on genetic programming},  
  journaltitle = {Information Sciences},
  author = {Kattan, Ahmed and Fatima, Shaheen and Arif, Muhammad},
  date = {2015-04},
  year = {2015},
  month = {04}
}
6
  • Welcome to TeX.SX! Please help us help you and add a minimal working example (MWE) that illustrates your problem. Reproducing the problem and finding out what the issue is will be much easier when we see compilable code, starting with \documentclass{...} and ending with \end{document}. Sep 5, 2015 at 10:01
  • You want a modification of an existing bib-file. From my point of view there is no relation to LaTeX/TeX but rather to perl Sep 5, 2015 at 10:15
  • Why do you want it? Biblatex/biber can/will extract the year from the date. Sep 5, 2015 at 10:20
  • 1
    biblatex can deal with the date field in YYYY-MM-DD format. If you actually use biblatex in the end, this will only confuse everyone involved. And if you don't you shouldn't export in biblatex format in the first place, I think.
    – moewe
    Sep 5, 2015 at 12:52
  • Interestingly, the contents of the date field always override the contents of the year and month field; the day field is not overwritten if it comes after the date field.
    – moewe
    Sep 5, 2015 at 12:58

5 Answers 5

7

This is something Biber can do in its tool mode.

Unfortunately, the solution originally suggested here ceased to work a while ago. Following https://github.com/plk/biber/issues/301 Biber was adjusted so things would work again. The fix will be available in Biber 2.15. For versions in between, I'm afraid I can not offer a simple Biber-based solution. (Maybe bibtool can help here.)

Biber version 2.15 and above

We need the following config file, called, say, biber-date.conf

<?xml version="1.0" encoding="UTF-8"?>
<config>
  <output_fieldcase>lower</output_fieldcase>
  <datamodel>
    <fields>
      <field fieldtype="field" datatype="literal">year</field>
      <field fieldtype="field" datatype="literal">month</field>
    </fields>
    <entryfields>
      <field>year</field>
      <field>month</field>
    </entryfields>
  </datamodel>
  <sourcemap>
    <maps datatype="bibtex">
      <map map_overwrite="1">
        <map_step map_field_source="date" map_match="[0-9]{4}-([0-9]{2})" map_final="1"/>
        <map_step map_field_set="montha" map_field_value="$1"/>
      </map>
      <map map_overwrite="1">
        <map_step map_field_source="date" map_match="([0-9]{4})" map_final="1"/>
        <map_step map_field_set="yeara" map_field_value="$1"/>
      </map>
    </maps>
  </sourcemap>
</config>

To convert the date fields in your .bib file run

biber --tool --configfile=biber-date.conf <yourbibfile>

For the example file augustus.bib

@article{kattan_timeseries_2015,
  title = {Time-series event-based prediction: An unsupervised learning framework based on genetic programming},  
  journaltitle = {Information Sciences},
  author = {Kattan, Ahmed and Fatima, Shaheen and Arif, Muhammad},
  date = {2015-04}
}
@book{lorem,
  title = {Lorem},  
  author = {Anne Uthor},
  date = {2015}
}
@book{ipsum,
  title = {Ipsum},  
  author = {Anne Uthor},
  date = {2015-08-07},
}

the output (which you can find in augustus_bibertools.bib after the biber --tool --configfile=biber-date.conf augustus.bib run) is

@article{kattan_timeseries_2015,
  author = {Kattan, Ahmed and Fatima, Shaheen and Arif, Muhammad},
  journaltitle = {Information Sciences},
  month = {4},
  title = {Time-series event-based prediction: An unsupervised learning framework based on genetic programming},
  year = {2015},
}

@book{lorem,
  author = {Uthor, Anne},
  title = {Lorem},
  year = {2015},
}

@book{ipsum,
  author = {Uthor, Anne},
  date = {2015-08-07},
  title = {Ipsum},
}

Old versions of Biber

We need the following config file, called say biber-date.conf

<?xml version="1.0" encoding="UTF-8"?>
<config>
  <output_fieldcase>lower</output_fieldcase>
  <sourcemap>
    <maps datatype="bibtex" map_overwrite="1">
      <map map_overwrite="1">
        <map_step map_field_source="date" map_match="[0-9]{4}?-([0-9]{2}?)" map_final="1"/>
        <map_step map_field_set="month" map_field_value="$1"/>
      </map>
      <map map_overwrite="1">
        <map_step map_field_source="date" map_match="([0-9]{4}?)" map_final="1"/>
        <map_step map_field_set="year" map_field_value="$1"/>
      </map>
    </maps>
  </sourcemap>
</config>

The regular expressions are not particularly elegant, but the first one copies the month position of the date field to the month field, the second maps the year position to the year field.

Now you just need to run biber --tool --configfile=biber-date.conf <yourbibfile>

On the example file augustus.bib

@article{kattan_timeseries_2015,
  title = {Time-series event-based prediction: An unsupervised learning framework based on genetic programming},  
  journaltitle = {Information Sciences},
  author = {Kattan, Ahmed and Fatima, Shaheen and Arif, Muhammad},
  date = {2015-04}
}
@book{lorem,
  title = {Lorem},  
  author = {Anne Uthor},
  date = {2015}
}
@book{ipsum,
  title = {Ipsum},  
  author = {Anne Uthor},
  date = {2015-08-07},
}

the output (which you can find in augustus_bibertools.bib after the biber --tool --configfile=biber-date.conf augustus.bib run) is

@article{kattan_timeseries_2015,
  author       = {Kattan, Ahmed and Fatima, Shaheen and Arif, Muhammad},
  date         = {2015-04},
  journaltitle = {Information Sciences},
  month        = {04},
  title        = {Time-series event-based prediction: An unsupervised learning framework based on genetic programming},
  year         = {2015},
}

@book{lorem,
  author = {Anne Uthor},
  date   = {2015},
  title  = {Lorem},
  year   = {2015},
}

@book{ipsum,
  author = {Anne Uthor},
  date   = {2015-08-07},
  month  = {08},
  title  = {Ipsum},
  year   = {2015},
}
5
  • If you also add <output_resolve>1</output_resolve> Biber will automatically resolve crossrefs/xrefs with inheritance, together with <output_safechars>1</output_safechars> you might come very close to creating a fully (BibTeX) downward-compatible version of your .bib file. (Though there is no such thing as "BibTeX compatible" as different styles support different fields.)
    – moewe
    Sep 5, 2015 at 19:22
  • But please note the caveat mentioned in my comment to the question. A year and month field together with a date field will confuse Biber quite a bit, that is something one should avoid. Note that in the solution above possibly existing month and years are overwritten (the map_overwrite="1" does that).
    – moewe
    Sep 5, 2015 at 19:53
  • I cannot reproduce your solution with biber 2.15. The only thing that changes in the bib-file is the order of fields. Oct 5, 2020 at 14:39
  • @LukasCB Sorry, I don't have time to look into this in more detail at the moment. I suggest you open an issue at github.com/plk/biber/issues there were some changes in the data model handling of --tool mode and some more changes are lined up for Biber 2.16.
    – moewe
    Oct 5, 2020 at 18:32
  • just did: github.com/plk/biber/issues/339 Oct 5, 2020 at 19:31
1

The biber solution does not work anymore. This seems to be by design since the manual (version 2.14) says

Dates are normalised into DATE fields. Legacy YEAR fields are never output in BibTeX format data output.

I solve this using a regexp solution for the year (the month can be extracted similarly):

sed -e 's/\(.*\)date\(.*\)\([[:digit:]]\{4\}\).*\(\}.*\)/\1year\2\3\4/g' augustus.bib
1
  • if you have an electronic entry with an urldate field, that will get turned into urlyear with this :)
    – emilBeBri
    Feb 10, 2022 at 17:04
1

After trying other solutions with no success, I ended up doing a Python script to process a list of files. The pre-requisite is that date is alone in a line, and is writen in the canonical forma YYYY-MM-DD. The program can process multipe files (FILES).

import os
import re

FILES = ["bibliografia.bib","standards.bib","bibdoslivrosesas.bib"]

TEMP = "temp.bib"

for arquivo in FILES:
    bak = arquivo+".bak"
    try:
        filew = open(TEMP,"w")
        with  open(arquivo,"r") as f:
            for linha in f:
                novalinha = linha.lstrip()
                if novalinha[:4] == "date" or novalinha[:4] == "DATE":
                    regexp = "\{[0-9\-]+\}"
                    m = re.search(regexp,novalinha).group(0)
                    # get year
                    print(m)
                    informacao = m.split("-")
                    if len(informacao) == 1:
                        filew.write("year = "+informacao[0]+",\n")
                    if len(informacao) == 2:
                        filew.write("year = "+informacao[0]+"},\n")
                        filew.write("month = {"+informacao[1]+",\n")
                    if len(informacao) == 3:
                        filew.write("year = "+informacao[0]+"},\n")
                        filew.write("month = {"+informacao[1]+"},\n")
                        filew.write("day = {"+informacao[2]+",\n")
                else:
                    filew.write(linha)
    finally:
        filew.close()
    os.rename(arquivo,bak)
    os.rename(TEMP,arquivo)
1

To anyone coming here and with Biber 2.15/2.16 and struggles to use the fixes provided here:

Try the solutions provided in https://github.com/plk/biber/issues/322 :

Both worked in my case, which is better depends whether you're using month/day as well -- see discussion in the issue.

-1

Here is another version, that makes sure that something like urldate does not get turned into urlyear (which happens with @Sebastians answer), and that does not replace date with year, but creates two new fields, one for year, one for month.

long story short:

sed -r 's/([ \t]+)(date)([ \t={]+)([[:digit:]]{4})([^0-9]{1})([[:digit:]]{2})(.*)/\1\2\3\4\5\6\7\n \1year\3\4\7\n \month\3\6\7/g' inputfile.bib | bibtool  -o outputfile.bib

Explanation:

  1. for this to work, there can't be a date with day included in the date-field. I don't have any in my own library, and have never seen it, but to check use ripgrep or grep to verify that there isn't one:

    rg -P '[ \t]date.*\d{4}-\d{2}-\d{2}' work-bibliotek.bib

  2. instead of

(.*)

in the beginning of the regex that captures all, i use

([ \t]+)

so that a field like urldate does that get replaced with urlyear. this version does require that there is actually a space before the field-name 'date' .

This can be easily secured by running bibtool on the bibfile before prior to the sed command:

  bibtool -i input.bib -o output.bib
  1. don't replace the field, but make two new ones, that include year and month.

  2. use sed -r where you don't have to use escape characters all the time, it's confusing

I pipe it to bibtool and output from there to align the month field with the others

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .