Problem running Biblatex with bib file exported from Zotero

I am trying since few days to solve a problem with Biblatex, which can't compile any of my references. I am running TeX Live 2013 (MacTeX) with Aquamacs as editor, biber 1.7 and biblatex 2.7. All packages are updated.

Strangely enough, Biblatex doesn't print any error while running (actually in the mini-buffer it says that the process was successful). The only line in the log where it shows any problem is the last one, when it says Biber abort trap: 6 at Wed Aug 14 09:33:22

The problem disappears if instead of Biblatex I switch to BibTex as backend: BibTex compiles correctly all the references (even though it gives two errors Your field is more than 20000 characters---line 1223 of file... and "{" immediately follows a field name---line 1698 of file...; in both cases BibTex concludes with I'm skipping whatever remains of this entry and finishes successfully the job).

Now I just tried to run Biblatex as backend on a different bib file on the same tex file: now Biblatex compiles successfully. I then guess Biblatex doesn't like my bib file exported from Zotero.

From where I start to get my bib compatible with Biblatex?

MWE (When I replace backend=biber with backend=biblatex the following works):

%test.tex

\documentclass[a4paper]{article}

% Create command for paragraph style (new line after)
\newcommand{\myparagraph}[1]{\paragraph{#1}\mbox{}\\}

% Set the values for the bibliography
\usepackage[
backend=biber,
isbn=false,
url=false,
doi=false,
eprint=false,
]{biblatex}

%Point to the bibliography db

\begin{document}

\include{test_chap}

\end{document}

%test_chap.tex


First bib entry (here posted first, problematic and final lines):

@book{rainie2012,
title = {Networked: The New Social Operating System},
isbn = {0262017199},
shorttitle = {Networked},
publisher = {{MIT} Press},
author = {Rainie, Lee and Wellman, Barry},
year = {2012},
annote = {Extracted Annotations {(Tue} Jun 25 20:12:42 2013)
{"Are} people huddling alone in front of their screens? If they are connecting with someone online, is it a vague simulacrum of real community with people they could have seen, smelled, heard, and touched in the {\textquotedblleft} g ood old days" {(Rainie} and Wellman 2012:17)
{"Our} research supports the notion that small, densely knit groups like families, villages, and small organizations have receded in recent generations. A different social order has" {(Rainie} and Wellman 2012:17)


... Missing lines here (It is very long)

"technology supported task coordination, in the sense that everyone knew what everyone else was working on" {(Rainie} and Wellman 2012:199)
{"ICTs} supported {\textquotedblleft} distributed cognition, {\textquotedblright} enabling team members to share and integrate their own diverse perspectives" {(Rainie} and Wellman 2012:199)
"variety of {ICTs} enabled the teams to interact in many ways, using the media they felt was most appropriate to the task at hand" {(Rainie} and Wellman 2012:199) %%this is line 1223 (starting with "variety...)
"appears that networked work and networked organizations are better at linking ideas {\textemdash} bits {\textemdash} t han at linking parts {\textemdash} atoms" {(Rainie} and Wellman 2012:199)
{"ICTs} convey fewer social cues than in-person contacts. That is why people travel to meet in person: to build trust, develop nuanced understanding, and exchange tacit knowledge" {(Rainie} and Wellman 2012:200)


... Missing lines

{"Rochelle} C \^{o} t \'{e} Gabriele Plickert, and Barry Wellman, {\textquotedblleft} Does the Golden Rule, Rule? {\textquotedblright} in Contexts of Social Capital ed. {Ray-May} Hsung, Nan Lin, and Ronald Breiger {(London:} Routledge, 2009), 49 {\textendash} 71" {(Rainie} and Wellman 2012:348)
{"Eli} Pariser, T he Filter Bubble: What the Internet Is Hiding from You {(London:} Penguin, 2011" {(Rainie} and Wellman 2012:351)}
},


Second bib entry:

@book{shirky2008, %This is problematic line 1698
title = {Here Comes Everybody: The Power of Organizing Without Organizations},
isbn = {9780713999891},
shorttitle = {Here Comes Everybody},
publisher = {Penguin Books},
author = {Shirky, Clay},
year = {2008},
keywords = {Francesco {PhD}}
},


EDIT: After removing the annotations (see "annote" field above) while exporting the bibliography with Zotero I partially solved the problem: now BibLatex complete the job (I am able to import references running Latex). Still I got warnings for the first line of each entry (even after changing text coding to UTF-8).

This is the output after running BibLatex (on a UTF-8 coded .bib file):

Running Biber' on myfile' with myfile''
INFO - This is Biber 1.7
INFO - Logfile is 'myfile.blg'
INFO - Found 1 citekeys in bib section 0
INFO - Found 9 citekeys in bib section 1
INFO - Found 1 citekeys in bib section 0
INFO - Found 7 citekeys in bib section 2
INFO - Found 1 citekeys in bib section 0
INFO - Found 15 citekeys in bib section 3
INFO - Found 1 citekeys in bib section 0
INFO - Processing section 0
INFO - Looking for bibtex format file '/my/path/ZoteroOutput.bib' for section 0
INFO - Found BibTeX data source '/my/path/ZoteroOutput.bib'
WARN - BibTeX subsystem: /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/H0EsS1omok/ZoteroOutput.bib_5763.utf8, line 13, warning: 1 characters of junk seen at toplevel
WARN - BibTeX subsystem: /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/H0EsS1omok/ZoteroOutput.bib_5763.utf8, line 28, warning: 1 characters of junk seen at toplevel
WARN - BibTeX subsystem: /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/H0EsS1omok


...MISSING TEXT...

/ZoteroOutput.bib_5763.utf8, line 3954, warning: 1 characters of junk seen at toplevel
WARN - BibTeX subsystem: /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/H0EsS1omok/ZoteroOutput.bib_5763.utf8, line 3970, warning: 1 characters of junk seen at toplevel
WARN - BibTeX subsystem: /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/H0EsS1omok/ZoteroOutput.bib_5763.utf8, line 3985, warning: 1 characters of junk seen at toplevel
WARN - BibTeX subsystem: /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/H0EsS1omok/ZoteroOutput.bib_5763.utf8, line 3997, warning: 1 characters of junk seen at toplevel
WARN - BibTeX subsystem: /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/H0EsS1omok/ZoteroOutput.bib_5763.utf8, line 4009, warning: 1 characters of junk seen at toplevel
INFO - Overriding locale 'en_US.UTF-8' default tailoring 'variable = shifted' with 'variable = non-ignorable'
INFO - Sorting 'entry' list 'nty' keys
INFO - No sort tailoring available for locale 'en_US.UTF-8'
INFO - Processing section 1
INFO - Looking for bibtex format file '/Users/francesco/Dropbox/Papers/BibTex/ZoteroOutput.bib' for section 1
INFO - Overriding locale 'en_US.UTF-8' default tailoring 'variable = shifted' with 'variable = non-ignorable'
INFO - Sorting 'entry' list 'nty' keys
INFO - No sort tailoring available for locale 'en_US.UTF-8'
INFO - Processing section 2
INFO - Looking for bibtex format file '/my/path/ZoteroOutput.bib' for section 2
INFO - Overriding locale 'en_US.UTF-8' default tailoring 'variable = shifted' with 'variable = non-ignorable'
INFO - Sorting 'entry' list 'nty' keys
INFO - No sort tailoring available for locale 'en_US.UTF-8'
INFO - Processing section 3
INFO - Looking for bibtex format file '/my/path/ZoteroOutput.bib' for section 3
INFO - Overriding locale 'en_US.UTF-8' default tailoring 'variable = shifted' with 'variable = non-ignorable'
INFO - Sorting 'entry' list 'nty' keys
INFO - No sort tailoring available for locale 'en_US.UTF-8'
INFO - Writing 'annotated_bibliography.bbl' with encoding 'ascii'
"\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
"\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
"\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
"\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
"\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
"\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
"\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
"\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
"\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
"\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
"\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
"\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
"\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
"\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
"\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
"\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
"\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
"\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
"\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
"\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
"\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
"\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
"\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
"\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
"\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
"\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
"\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
INFO - Output to annotated_bibliography.bbl
INFO - WARNINGS: 316

Biber finished at Wed Aug 21 18:46:40


I solved my issues of compatibility between the BibTex file exported from Zotero and BibLatex. I recap the solution here. Apparently BibLatex is much more sensible than BibTex when it comes to reading a BibTex file...

To export correctly the bibliography from Zotero and then into a Tex document via BibLatex I used this BibTeX Export Translator (modified by Robin Wilson) with two important tweaks:

1) On line 11 of the translator script you need to set "exportNotes": true, to "exportNotes": false, (of course just in case you have added notes to your Zotero items, e.g. you extract annotations from PDFs). This will avoid BibLatex crashing on too long fields.

2) On line 2256 you need to tell the translator to avoid adding a comma after each bibliography item (the comma will result in BibLatex giving a warning on the first line of each entry excluding the first...) changing this line: Zotero.write((first ? "" : ",\n\n") + "@"+type+"{"+citekey); with this line: Zotero.write((first ? "" : "\n\n") + "@"+type+"{"+citekey);

Another possible source of problems of the BibTeX Export Translator could be the character encoding. In my translator I kept as in source file "exportCharset": "ISO-8859-1", on line 10. I tried to change it to UTF-8 but it created another class of issues, not with BibLatex (it run without errors) but with Latex,

[1{/usr/local/texlive/2013/texmf-var/fonts/map/pdftex/updmap/pdftex.map}]
(./bib/price2012.tex [2]
! Undefined control sequence.
<to be read again> \edef \blx@tempa {193\x
{FFFD}\x {FFFD}\x {FFFD}219}


for each refsection. It also messed with the "pages" field of some items, outputting:

Andrea B. Hollingshead. “Information suppression and status persistence in group decision making the effects of communication media”. In: Human Communication Research 23.2 (1996), 193fffdfffdfffd219.

I then switched back to "exportCharset": "ISO-8859-1", and both BibLatex and Latex run without warnings or errors. The PDF output was correct:

Andrea B. Hollingshead. “Information suppression and status persistence in group decision making the effects of communication media”. In: Human Communication Research 23.2 (1996), 193–219.

and I also get correct output with the umlaut of Habermas:

EDIT: The problem with the page field was probably due by a odd "-" character that you sometimes get to separate the 2 page numbers when you download the citation from the Internet.

The included field annote and abstract in your showed bib entries are too long (see your cited error message: Your field is more than 20000 characters). I guess you do not want to print them in your bibliography so you can delete them.

With a little pretty printing and minor corrections I came to the following MWE running with MiKTeX 2.9 and Biber without errors (package filecontents is used to have only one MWE, including both .tex and .bib file):

\RequirePackage{filecontents}
\begin{filecontents*}{\jobname.bib}
@book{test1,
author     = {Rainie, Lee and Wellman, Barry},
publisher  = {{MIT} Press},
title      = {Networked: The New Social Operating System},
shorttitle = {short},
isbn       = {0262017199},
year       = {2012},
}
@book{test2,
title      = {Here Comes Everybody: The Power of Organizing Without Organizations},
isbn       = {9780713999891},
shorttitle = {Here Comes Everybody},
publisher  = {Penguin Books},
author     = {Shirky, Clay},
year       = {2008},
keywords   = {Francesco {PhD}},
}
\end{filecontents*}

\documentclass[a4paper]{article}

\usepackage[utf8]{inputenc}
%\usepackage[Latin9]{inputenc}

% Set the values for the bibliography
\usepackage[
backend=biber
% backend=bibtex8
,isbn=false
,url=false
,doi=false
,eprint=false
]{biblatex}

%Point to the bibliography db

\begin{document}

\cite{test1} \cite{test2} \fullcite{test1}

\printbibliography
\end{document}


It seems you have an encoding problem with your files, because you said in your comment you still have a message about a junk character. Please check the encoding you have used in Zotero, could be you have to change it ... Thats the reasion why I used new first lines for your bib entrys (@book{test1,) written in my editor to be sure to have no junk characters there ...

Another problems seems that Zotero adds a comma (,) after the closing bracket of an bib entry. I deleted this commas for my MWE. That should not be! Correct is:

@book{key,
name  = {value},
name1 = {value1},
}             <===== here no comma!


As you can read in this blog the commas are part of Zotero and they seem to be the jink character.

I just found an old question biblatex or biber bug that means in my opinion that Zotero is not able to create a bib file for biblatex/biber. So you should use BibTeX or change the bibliography program, for example jabref or something else.

• I use the AutoZotBib plugin to have Zotero exporting automatically to .bib after every update. I was able to set "exportNotes": false, in the .js script of the plugin so to solve my problem with too long fields. Still I have problems with junk chars. According to the .js script the coding should be "exportCharset": "ISO-8859-1" – Francesco Aug 21 '13 at 0:13
• Yes, I am using a Mac (10.8.4) and TeX Live 2013. The BibTeX keys were generated by [this] (rtwilson.com/academic/BibTeX_RobinTan.js) translator. It is not the default translator of Zotero. I installed it when installed the AutoZotBib plugin. Should I try to replace "exportCharset": "ISO-8859-1" with "exportCharset": "UTF8" (on line 10 of the translator)? – Francesco Aug 21 '13 at 5:23
• I edit the main question with the output from biblatex. After changing to utf8 I have the same problem. (I have checked the file encoding and now is ZoteroOutput.bib: text/plain; charset=utf-8) – Francesco Aug 21 '13 at 9:02

Confirming that

},

at the end of a record in your .bib file can cause this error msg. One way to fix this is to edit out those occurrences in your .bib file. Be careful, though, since you may have instances of that sequence record-internally. BibTeX (and I suppose LaBibTex) has a habit of converting upper case characters in titles etc. to lower case, on the assumption that nothing past the first content word of a title should be capitalized. (I think some librarian decided that.) But in fact we often have proper names in titles, and they need to retain their capitalization; so we often embed titles (and some other fields) in {}, e.g.

title={Evidentiality in South Asian languages},


and you definitely don't want to remove the comma after that closing }.

I obtained the same error message

Your field is more than 20000 characters


but found the solutions presented here were too complex for my situation (although I accept that they may have been necessary in this particular instance).

What I found worked for me was to paste my text into a word processor (such as Word, eek) and count back 20,000 characters from the point at which bibtex noted the error occurred. Be careful to include spaces in the character-count! I found that there was an unclosed "}" in a recent entry.