18

I've developed a simple BibTeX parser. I want to test whether it can handle all possible nuances that can occur in BibTeX databases.

Is there a standard .bib file against which I can check my parser, so that I can be assured of its parsing capabilities?

I've already tried some .bib files off the Internet and they are being processed correctly.

4
  • 4
    The Collection of Computer Science Bibliographies might be another source for test cases. Commented Apr 24, 2011 at 12:23
  • 2
    Is your parser, by any chance, open source? I would be very interested in a good parser implementation, even better if it can convert to HTML Commented Apr 25, 2011 at 9:42
  • @Christian, that's an excellent choice, you may want to put that down as an answer.
    – user5018
    Commented Apr 26, 2011 at 3:58
  • @Federico, I'm sorry but my parser only stores the results in an SQLite DB for querying. I can share the source code if needed. You should surely check out bib2html (litech.org/~wkiri/bib2html) though.
    – user5018
    Commented Apr 26, 2011 at 4:01

3 Answers 3

14

The Collection of Science Bibliographies provides a huge number of test cases. It might be that the ones on site are generated by a pretty printer and thus would be quite uniform in their syntactical features. However, there are also links to the original files which should be more diverse.

For testing parsers in general, there typically is no one Gold standard. Compiler writers rely on individual test cases and test case generators. A test case generator is driven by the grammar of a language and can help to find corner cases that rarely occur in practice. Developing those requires substantial effort, though. If you are curious, look for fuzz testing. As an example for the C language, Finding and Understanding Bugs in C Compilers is a fascinating read about this subject.

9

If you have biblatex installed, then you have a nice example .bib file at $TEXMF\bibtex\bib\biblatex\biblatex-examples.bib. It has a lot of comments describing the possible pitfalls of each bibtex entry. Alternatively you can download the file here: http://mirror.ctan.org/macros/latex/contrib/biblatex/bibtex/bib/biblatex/biblatex-examples.bib

5

I was thinking on a BNF grammar of the BibTeX format, but I coudn't find one by myself. The closest I found was this one (from a TCL/TK wiki):

# A rough grammar (case-insensitive):
#
# Database  ::= (Junk '@' Entry)*
# Junk      ::= .*?
# Entry ::= Record
#       |   Comment
#       |   String
#       |   Preamble
# Comment   ::= "comment" [^\n]* \n     -- ignored
# String    ::= "string" '{' Field* '}'
# Preamble  ::= "preamble" '{' .* '}'   -- (balanced)
# Record    ::= Type '{' Key ',' Field* '}'
#       |   Type '(' Key ',' Field* ')' -- not handled
# Type  ::= Name
# Key   ::= Name
# Field ::= Name '=' Value
# Name      ::= [^\s\"#%'(){}]*
# Value ::= [0-9]+
#       |   '"' ([^'"']|\\'"')* '"'
#       |   '{' .* '}'          -- (balanced)

If your parser is handling well all the .bib files you tested, I think you already did a great job!

2
  • You don't need salutations or closings in your answers. That grammar is also wrong.
    – TH.
    Commented Apr 26, 2011 at 7:18
  • @TH.: Thanks for the hints. As I said, that grammar is the closest I found on the subject. Unfortunately, I'm not that used to BibTeX neither BNF to provide a better grammar. I just thought that'd be a nice starting point. Glad you pointed that. Commented Apr 26, 2011 at 12:45

You must log in to answer this question.