Problem running Biblatex with bib file exported from Zotero

Question

I am trying since few days to solve a problem with Biblatex, which can't compile any of my references. I am running TeX Live 2013 (MacTeX) with Aquamacs as editor, biber 1.7 and biblatex 2.7. All packages are updated.

Strangely enough, Biblatex doesn't print any error while running (actually in the mini-buffer it says that the process was successful). The only line in the log where it shows any problem is the last one, when it says Biber abort trap: 6 at Wed Aug 14 09:33:22

The problem disappears if instead of Biblatex I switch to BibTex as backend: BibTex compiles correctly all the references (even though it gives two errors Your field is more than 20000 characters---line 1223 of file... and "{" immediately follows a field name---line 1698 of file...; in both cases BibTex concludes with I'm skipping whatever remains of this entry and finishes successfully the job).

Now I just tried to run Biblatex as backend on a different bib file on the same tex file: now Biblatex compiles successfully. I then guess Biblatex doesn't like my bib file exported from Zotero.

From where I start to get my bib compatible with Biblatex?

MWE (When I replace backend=biber with backend=biblatex the following works):

%test.tex

\documentclass[a4paper]{article}

% Create command for paragraph style (new line after)
\newcommand{\myparagraph}[1]{\paragraph{#1}\mbox{}\\}


% Set the values for the bibliography
\usepackage[
    backend=biber,
    isbn=false,
    url=false,
    doi=false,
    eprint=false,
]{biblatex}

%Point to the bibliography db
\addbibresource{ZoteroOutput.bib}

\begin{document}

\fullcite{greenwade93}

\include{test_chap}

\end{document}

%test_chap.tex
\fullcite{greenwade93}

First bib entry (here posted first, problematic and final lines):

@book{rainie2012,
    address = {Cambridge, {MA}},
    title = {Networked: The New Social Operating System},
    isbn = {0262017199},
    shorttitle = {Networked},
    publisher = {{MIT} Press},
    author = {Rainie, Lee and Wellman, Barry},
    year = {2012},
    annote = {Extracted Annotations {(Tue} Jun 25 20:12:42 2013)
{"Are} people huddling alone in front of their screens? If they are connecting with someone online, is it a vague simulacrum of real community with people they could have seen, smelled, heard, and touched in the {\textquotedblleft} g ood old days" {(Rainie} and Wellman 2012:17)
{"Our} research supports the notion that small, densely knit groups like families, villages, and small organizations have receded in recent generations. A different social order has" {(Rainie} and Wellman 2012:17)

... Missing lines here (It is very long)

"technology supported task coordination, in the sense that everyone knew what everyone else was working on" {(Rainie} and Wellman 2012:199)
{"ICTs} supported {\textquotedblleft} distributed cognition, {\textquotedblright} enabling team members to share and integrate their own diverse perspectives" {(Rainie} and Wellman 2012:199)
"variety of {ICTs} enabled the teams to interact in many ways, using the media they felt was most appropriate to the task at hand" {(Rainie} and Wellman 2012:199) %%this is line 1223 (starting with "variety...)
"appears that networked work and networked organizations are better at linking ideas {\textemdash} bits {\textemdash} t han at linking parts {\textemdash} atoms" {(Rainie} and Wellman 2012:199)
{"ICTs} convey fewer social cues than in-person contacts. That is why people travel to meet in person: to build trust, develop nuanced understanding, and exchange tacit knowledge" {(Rainie} and Wellman 2012:200)

... Missing lines

{"Rochelle} C \^{o} t \'{e} Gabriele Plickert, and Barry Wellman, {\textquotedblleft} Does the Golden Rule, Rule? {\textquotedblright} in Contexts of Social Capital ed. {Ray-May} Hsung, Nan Lin, and Ronald Breiger {(London:} Routledge, 2009), 49 {\textendash} 71" {(Rainie} and Wellman 2012:348)
{"Eli} Pariser, T he Filter Bubble: What the Internet Is Hiding from You {(London:} Penguin, 2011" {(Rainie} and Wellman 2012:351)}
},

Second bib entry:

@book{shirky2008, %This is problematic line 1698
    address = {New York, {NY}},
    title = {Here Comes Everybody: The Power of Organizing Without Organizations},
    isbn = {9780713999891},
    shorttitle = {Here Comes Everybody},
    abstract = {Read Clay Shirky's posts on the Penguin Blog.    A revelatory examination of how the wildfirelike spread of new forms of social interaction enabled by technology is changing the way humans form groups and exist within them, with profound long-term economic and social effects-for good and for ill A handful of kite hobbyists scattered around the world find each other online and collaborate on the most radical improvement in kite design in decades. A midwestern professor of Middle Eastern history starts a blog after 9/11 that becomes essential reading for journalists covering the Iraq war. Activists use the Internet and e-mail to bring offensive comments made by Trent Lott and Don Imus to a wide public and hound them from their positions. A few people find that a world-class online encyclopedia created entirely by volunteers and open for editing by anyone, a wiki, is not an impractical idea. Jihadi groups trade inspiration and instruction and showcase terrorist atrocities to the world, entirely online. A wide group of unrelated people swarms to a Web site about the theft of a cell phone and ultimately goads the New York City police to take action, leading to the culprit's arrest.  With accelerating velocity, our age's new technologies of social networking are evolving, and evolving us, into new groups doing new things in new ways, and old and new groups alike doing the old things better and more easily. You don't have to have a {MySpace} page to know that the times they are a changin'. Hierarchical structures that exist to manage the work of groups are seeing their raisons d'tre swiftly eroded by the rising technological tide. Business models are being destroyed, transformed, born at dizzying speeds, and the larger social impact is profound.  One of the culture's wisest observers of the transformational power of the new forms of tech-enabled social interaction is Clay Shirky, {andHere} Comes Everybodyis his marvelous reckoning with the ramifications of all this on what we do and who we are. Like Lawrence Lessig on the effect of new technology on regimes of cultural creation, Shirky's assessment of the impact of new technology on the nature and use of groups is marvelously broad minded, lucid, and penetrating; it integrates the views of a number of other thinkers across a broad range of disciplines with his own pioneering work to provide a holistic framework for understanding the opportunities and the threats to the existing order that these new, spontaneous networks of social interaction represent. Wikinomics, yes, but also wikigovernment, wikiculture, wikievery imaginable interest group, including the far from savory. A revolution in social organization has commenced, and Clay Shirky is its brilliant chronicler.},
    publisher = {Penguin Books},
    author = {Shirky, Clay},
    year = {2008},
    keywords = {Francesco {PhD}}
},

EDIT: After removing the annotations (see "annote" field above) while exporting the bibliography with Zotero I partially solved the problem: now BibLatex complete the job (I am able to import references running Latex). Still I got warnings for the first line of each entry (even after changing text coding to UTF-8).

This is the output after running BibLatex (on a UTF-8 coded .bib file):

Running `Biber' on `myfile' with ``myfile''
INFO - This is Biber 1.7
INFO - Logfile is 'myfile.blg'
INFO - Reading 'myfile.bcf'
INFO - Found 1 citekeys in bib section 0
INFO - Found 9 citekeys in bib section 1
INFO - Found 1 citekeys in bib section 0
INFO - Found 7 citekeys in bib section 2
INFO - Found 1 citekeys in bib section 0
INFO - Found 15 citekeys in bib section 3
INFO - Found 1 citekeys in bib section 0
INFO - Processing section 0
INFO - Looking for bibtex format file '/my/path/ZoteroOutput.bib' for section 0
INFO - Found BibTeX data source '/my/path/ZoteroOutput.bib'
WARN - BibTeX subsystem: /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/H0EsS1omok/ZoteroOutput.bib_5763.utf8, line 13, warning: 1 characters of junk seen at toplevel
WARN - BibTeX subsystem: /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/H0EsS1omok/ZoteroOutput.bib_5763.utf8, line 28, warning: 1 characters of junk seen at toplevel
WARN - BibTeX subsystem: /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/H0EsS1omok

...MISSING TEXT...

/ZoteroOutput.bib_5763.utf8, line 3954, warning: 1 characters of junk seen at toplevel
    WARN - BibTeX subsystem: /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/H0EsS1omok/ZoteroOutput.bib_5763.utf8, line 3970, warning: 1 characters of junk seen at toplevel
    WARN - BibTeX subsystem: /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/H0EsS1omok/ZoteroOutput.bib_5763.utf8, line 3985, warning: 1 characters of junk seen at toplevel
    WARN - BibTeX subsystem: /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/H0EsS1omok/ZoteroOutput.bib_5763.utf8, line 3997, warning: 1 characters of junk seen at toplevel
    WARN - BibTeX subsystem: /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/H0EsS1omok/ZoteroOutput.bib_5763.utf8, line 4009, warning: 1 characters of junk seen at toplevel
    INFO - Overriding locale 'en_US.UTF-8' default tailoring 'variable = shifted' with 'variable = non-ignorable'
    INFO - Sorting 'entry' list 'nty' keys
    INFO - No sort tailoring available for locale 'en_US.UTF-8'
    INFO - Processing section 1
    INFO - Looking for bibtex format file '/Users/francesco/Dropbox/Papers/BibTex/ZoteroOutput.bib' for section 1
    INFO - Overriding locale 'en_US.UTF-8' default tailoring 'variable = shifted' with 'variable = non-ignorable'
    INFO - Sorting 'entry' list 'nty' keys
    INFO - No sort tailoring available for locale 'en_US.UTF-8'
    INFO - Processing section 2
    INFO - Looking for bibtex format file '/my/path/ZoteroOutput.bib' for section 2
    INFO - Overriding locale 'en_US.UTF-8' default tailoring 'variable = shifted' with 'variable = non-ignorable'
    INFO - Sorting 'entry' list 'nty' keys
    INFO - No sort tailoring available for locale 'en_US.UTF-8'
    INFO - Processing section 3
    INFO - Looking for bibtex format file '/my/path/ZoteroOutput.bib' for section 3
    INFO - Overriding locale 'en_US.UTF-8' default tailoring 'variable = shifted' with 'variable = non-ignorable'
    INFO - Sorting 'entry' list 'nty' keys
    INFO - No sort tailoring available for locale 'en_US.UTF-8'
    INFO - Writing 'annotated_bibliography.bbl' with encoding 'ascii'
    "\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
    "\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
    "\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
    "\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
    "\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
    "\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
    "\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
    "\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
    "\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
    "\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
    "\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
    "\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
    "\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
    "\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
    "\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
    "\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
    "\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
    "\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
    "\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
    "\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
    "\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
    "\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
    "\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
    "\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
    "\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
    "\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
    "\x{fffd}" does not map to ascii at /var/folders/4b/v7yn1_5h8xq1p3006s6jvcg00000gn/T/par-6672616e636573636f/cache-cdd483146f82a9655ce063f848d5139480fbf872/inc/lib/Biber/Utils.pm line 949.
    INFO - Output to annotated_bibliography.bbl
    INFO - WARNINGS: 316

    Biber finished at Wed Aug 21 18:46:40

Answer 1

I solved my issues of compatibility between the BibTex file exported from Zotero and BibLatex. I recap the solution here. Apparently BibLatex is much more sensible than BibTex when it comes to reading a BibTex file...

To export correctly the bibliography from Zotero and then into a Tex document via BibLatex I used this BibTeX Export Translator (modified by Robin Wilson) with two important tweaks:

1) On line 11 of the translator script you need to set "exportNotes": true, to "exportNotes": false, (of course just in case you have added notes to your Zotero items, e.g. you extract annotations from PDFs). This will avoid BibLatex crashing on too long fields.

2) On line 2256 you need to tell the translator to avoid adding a comma after each bibliography item (the comma will result in BibLatex giving a warning on the first line of each entry excluding the first...) changing this line: Zotero.write((first ? "" : ",\n\n") + "@"+type+"{"+citekey); with this line: Zotero.write((first ? "" : "\n\n") + "@"+type+"{"+citekey);

Another possible source of problems of the BibTeX Export Translator could be the character encoding. In my translator I kept as in source file "exportCharset": "ISO-8859-1", on line 10. I tried to change it to UTF-8 but it created another class of issues, not with BibLatex (it run without errors) but with Latex,

[1{/usr/local/texlive/2013/texmf-var/fonts/map/pdftex/updmap/pdftex.map}]
(./bib/price2012.tex [2]
! Undefined control sequence.
<to be read again> \edef \blx@tempa {193\x 
                                           {FFFD}\x {FFFD}\x {FFFD}219}
l.8 \printbibliography[heading=subbibliography]

for each refsection. It also messed with the "pages" field of some items, outputting:

Andrea B. Hollingshead. “Information suppression and status persistence in group decision making the effects of communication media”. In: Human Communication Research 23.2 (1996), 193fffdfffdfffd219.

I then switched back to "exportCharset": "ISO-8859-1", and both BibLatex and Latex run without warnings or errors. The PDF output was correct:

Andrea B. Hollingshead. “Information suppression and status persistence in group decision making the effects of communication media”. In: Human Communication Research 23.2 (1996), 193–219.

and I also get correct output with the umlaut of Habermas:

EDIT: The problem with the page field was probably due by a odd "-" character that you sometimes get to separate the 2 page numbers when you download the citation from the Internet.

Answer 2

The included field annote and abstract in your showed bib entries are too long (see your cited error message: Your field is more than 20000 characters). I guess you do not want to print them in your bibliography so you can delete them.

With a little pretty printing and minor corrections I came to the following MWE running with MiKTeX 2.9 and Biber without errors (package filecontents is used to have only one MWE, including both .tex and .bib file):

\RequirePackage{filecontents}
\begin{filecontents*}{\jobname.bib}
@book{test1,
  author     = {Rainie, Lee and Wellman, Barry},
  publisher  = {{MIT} Press},
  title      = {Networked: The New Social Operating System},
  shorttitle = {short},
  address    = {Cambridge, {MA}},
  isbn       = {0262017199},
  year       = {2012},
}
@book{test2,
  address = {New York, {NY}},
  title      = {Here Comes Everybody: The Power of Organizing Without Organizations},
  isbn       = {9780713999891},
  shorttitle = {Here Comes Everybody},
  publisher  = {Penguin Books},
  author     = {Shirky, Clay},
  year       = {2008},
  keywords   = {Francesco {PhD}},
}
\end{filecontents*}


\documentclass[a4paper]{article}

\usepackage[utf8]{inputenc}
%\usepackage[Latin9]{inputenc}

% Set the values for the bibliography
\usepackage[
  backend=biber
% backend=bibtex8
 ,isbn=false
 ,url=false
 ,doi=false
 ,eprint=false
]{biblatex}

%Point to the bibliography db
\addbibresource{\jobname.bib}

\begin{document}

\cite{test1} \cite{test2} \fullcite{test1}

\printbibliography
\end{document}

It seems you have an encoding problem with your files, because you said in your comment you still have a message about a junk character. Please check the encoding you have used in Zotero, could be you have to change it ... Thats the reasion why I used new first lines for your bib entrys (@book{test1,) written in my editor to be sure to have no junk characters there ...

Another problems seems that Zotero adds a comma (,) after the closing bracket of an bib entry. I deleted this commas for my MWE. That should not be! Correct is:

@book{key,
  name  = {value}, 
  name1 = {value1},
}             <===== here no comma! 

As you can read in this blog the commas are part of Zotero and they seem to be the jink character.

I just found an old question biblatex or biber bug that means in my opinion that Zotero is not able to create a bib file for biblatex/biber. So you should use BibTeX or change the bibliography program, for example jabref or something else.

Answer 3

Confirming that

},

at the end of a record in your .bib file can cause this error msg. One way to fix this is to edit out those occurrences in your .bib file. Be careful, though, since you may have instances of that sequence record-internally. BibTeX (and I suppose LaBibTex) has a habit of converting upper case characters in titles etc. to lower case, on the assumption that nothing past the first content word of a title should be capitalized. (I think some librarian decided that.) But in fact we often have proper names in titles, and they need to retain their capitalization; so we often embed titles (and some other fields) in {}, e.g.

title={Evidentiality in South Asian languages},

and you definitely don't want to remove the comma after that closing }.

Answer 4

I obtained the same error message

Your field is more than 20000 characters

but found the solutions presented here were too complex for my situation (although I accept that they may have been necessary in this particular instance).

What I found worked for me was to paste my text into a word processor (such as Word, eek) and count back 20,000 characters from the point at which bibtex noted the error occurred. Be careful to include spaces in the character-count! I found that there was an unclosed "}" in a recent entry.