Biblatex: How to convert date field into year and month field?

Question

I have exported the bibliography from Zotero in biblatex format. All the references contain a date field, but now year field.

Is there a simple way (a tool, a script etc.) to convert the date field or to append year and month fields?

Example:

@article{kattan_timeseries_2015,
  title = {Time-series event-based prediction: An unsupervised learning framework based on genetic programming},  
  journaltitle = {Information Sciences},
  author = {Kattan, Ahmed and Fatima, Shaheen and Arif, Muhammad},
  date = {2015-04}
}

and I want to be

@article{kattan_timeseries_2015,
  title = {Time-series event-based prediction: An unsupervised learning framework based on genetic programming},  
  journaltitle = {Information Sciences},
  author = {Kattan, Ahmed and Fatima, Shaheen and Arif, Muhammad},
  date = {2015-04},
  year = {2015},
  month = {04}
}

Answer 1

This is something Biber can do in its tool mode.

Unfortunately, the solution originally suggested here ceased to work a while ago. Following https://github.com/plk/biber/issues/301 Biber was adjusted so things would work again. The fix will be available in Biber 2.15. For versions in between, I'm afraid I can not offer a simple Biber-based solution. (Maybe bibtool can help here.)

Biber version 2.15 and above

We need the following config file, called, say, biber-date.conf

<?xml version="1.0" encoding="UTF-8"?>
<config>
  <output_fieldcase>lower</output_fieldcase>
  <datamodel>
    <fields>
      <field fieldtype="field" datatype="literal">year</field>
      <field fieldtype="field" datatype="literal">month</field>
    </fields>
    <entryfields>
      <field>year</field>
      <field>month</field>
    </entryfields>
  </datamodel>
  <sourcemap>
    <maps datatype="bibtex">
      <map map_overwrite="1">
        <map_step map_field_source="date" map_match="[0-9]{4}-([0-9]{2})" map_final="1"/>
        <map_step map_field_set="montha" map_field_value="$1"/>
      </map>
      <map map_overwrite="1">
        <map_step map_field_source="date" map_match="([0-9]{4})" map_final="1"/>
        <map_step map_field_set="yeara" map_field_value="$1"/>
      </map>
    </maps>
  </sourcemap>
</config>

To convert the date fields in your .bib file run

biber --tool --configfile=biber-date.conf <yourbibfile>

For the example file augustus.bib

@article{kattan_timeseries_2015,
  title = {Time-series event-based prediction: An unsupervised learning framework based on genetic programming},  
  journaltitle = {Information Sciences},
  author = {Kattan, Ahmed and Fatima, Shaheen and Arif, Muhammad},
  date = {2015-04}
}
@book{lorem,
  title = {Lorem},  
  author = {Anne Uthor},
  date = {2015}
}
@book{ipsum,
  title = {Ipsum},  
  author = {Anne Uthor},
  date = {2015-08-07},
}

the output (which you can find in augustus_bibertools.bib after the biber --tool --configfile=biber-date.conf augustus.bib run) is

@article{kattan_timeseries_2015,
  author = {Kattan, Ahmed and Fatima, Shaheen and Arif, Muhammad},
  journaltitle = {Information Sciences},
  month = {4},
  title = {Time-series event-based prediction: An unsupervised learning framework based on genetic programming},
  year = {2015},
}

@book{lorem,
  author = {Uthor, Anne},
  title = {Lorem},
  year = {2015},
}

@book{ipsum,
  author = {Uthor, Anne},
  date = {2015-08-07},
  title = {Ipsum},
}

---

Old versions of Biber

We need the following config file, called say biber-date.conf

<?xml version="1.0" encoding="UTF-8"?>
<config>
  <output_fieldcase>lower</output_fieldcase>
  <sourcemap>
    <maps datatype="bibtex" map_overwrite="1">
      <map map_overwrite="1">
        <map_step map_field_source="date" map_match="[0-9]{4}?-([0-9]{2}?)" map_final="1"/>
        <map_step map_field_set="month" map_field_value="$1"/>
      </map>
      <map map_overwrite="1">
        <map_step map_field_source="date" map_match="([0-9]{4}?)" map_final="1"/>
        <map_step map_field_set="year" map_field_value="$1"/>
      </map>
    </maps>
  </sourcemap>
</config>

The regular expressions are not particularly elegant, but the first one copies the month position of the date field to the month field, the second maps the year position to the year field.

Now you just need to run biber --tool --configfile=biber-date.conf <yourbibfile>

On the example file augustus.bib

@article{kattan_timeseries_2015,
  title = {Time-series event-based prediction: An unsupervised learning framework based on genetic programming},  
  journaltitle = {Information Sciences},
  author = {Kattan, Ahmed and Fatima, Shaheen and Arif, Muhammad},
  date = {2015-04}
}
@book{lorem,
  title = {Lorem},  
  author = {Anne Uthor},
  date = {2015}
}
@book{ipsum,
  title = {Ipsum},  
  author = {Anne Uthor},
  date = {2015-08-07},
}

the output (which you can find in augustus_bibertools.bib after the biber --tool --configfile=biber-date.conf augustus.bib run) is

@article{kattan_timeseries_2015,
  author       = {Kattan, Ahmed and Fatima, Shaheen and Arif, Muhammad},
  date         = {2015-04},
  journaltitle = {Information Sciences},
  month        = {04},
  title        = {Time-series event-based prediction: An unsupervised learning framework based on genetic programming},
  year         = {2015},
}

@book{lorem,
  author = {Anne Uthor},
  date   = {2015},
  title  = {Lorem},
  year   = {2015},
}

@book{ipsum,
  author = {Anne Uthor},
  date   = {2015-08-07},
  month  = {08},
  title  = {Ipsum},
  year   = {2015},
}

Answer 2

The biber solution does not work anymore. This seems to be by design since the manual (version 2.14) says

Dates are normalised into DATE fields. Legacy YEAR fields are never output in BibTeX format data output.

I solve this using a regexp solution for the year (the month can be extracted similarly):

sed -e 's/\(.*\)date\(.*\)\([[:digit:]]\{4\}\).*\(\}.*\)/\1year\2\3\4/g' augustus.bib

Answer 3

After trying other solutions with no success, I ended up doing a Python script to process a list of files. The pre-requisite is that date is alone in a line, and is writen in the canonical forma YYYY-MM-DD. The program can process multipe files (FILES).

import os
import re

FILES = ["bibliografia.bib","standards.bib","bibdoslivrosesas.bib"]

TEMP = "temp.bib"

for arquivo in FILES:
    bak = arquivo+".bak"
    try:
        filew = open(TEMP,"w")
        with  open(arquivo,"r") as f:
            for linha in f:
                novalinha = linha.lstrip()
                if novalinha[:4] == "date" or novalinha[:4] == "DATE":
                    regexp = "\{[0-9\-]+\}"
                    m = re.search(regexp,novalinha).group(0)
                    # get year
                    print(m)
                    informacao = m.split("-")
                    if len(informacao) == 1:
                        filew.write("year = "+informacao[0]+",\n")
                    if len(informacao) == 2:
                        filew.write("year = "+informacao[0]+"},\n")
                        filew.write("month = {"+informacao[1]+",\n")
                    if len(informacao) == 3:
                        filew.write("year = "+informacao[0]+"},\n")
                        filew.write("month = {"+informacao[1]+"},\n")
                        filew.write("day = {"+informacao[2]+",\n")
                else:
                    filew.write(linha)
    finally:
        filew.close()
    os.rename(arquivo,bak)
    os.rename(TEMP,arquivo)

Answer 4

To anyone coming here and with Biber 2.15/2.16 and struggles to use the fixes provided here:

Try the solutions provided in https://github.com/plk/biber/issues/322 :

• --output-field-replace=date:year https://github.com/plk/biber/issues/322#issuecomment-657228997
• --output-legacy-dates https://github.com/plk/biber/issues/322#issuecomment-658273552

Both worked in my case, which is better depends whether you're using month/day as well -- see discussion in the issue.

Answer 5

Here is another version, that makes sure that something like urldate does not get turned into urlyear (which happens with @Sebastians answer), and that does not replace date with year, but creates two new fields, one for year, one for month.

long story short:

sed -r 's/([ \t]+)(date)([ \t={]+)([[:digit:]]{4})([^0-9]{1})([[:digit:]]{2})(.*)/\1\2\3\4\5\6\7\n \1year\3\4\7\n \month\3\6\7/g' inputfile.bib | bibtool  -o outputfile.bib

Explanation:

0. for this to work, there can't be a date with day included in the date-field. I don't have any in my own library, and have never seen it, but to check use ripgrep or grep to verify that there isn't one:

   rg -P '[ \t]date.*\d{4}-\d{2}-\d{2}' work-bibliotek.bib

1. instead of

(.*)

in the beginning of the regex that captures all, i use

([ \t]+)

so that a field like urldate does that get replaced with urlyear. this version does require that there is actually a space before the field-name 'date' .

This can be easily secured by running bibtool on the bibfile before prior to the sed command:

  bibtool -i input.bib -o output.bib

2. don't replace the field, but make two new ones, that include year and month.

3. use sed -r where you don't have to use escape characters all the time, it's confusing

I pipe it to bibtool and output from there to align the month field with the others