15

When I want to cite a research paper, I typically look up the respective BibTeX entry in some online catalogue (such as DBLP or the ACM and IEEE digital libraries for computer science). The quality of the entries is not always great, but at least a good start.

Now I am in the situation that I am writing a research article in which I also need to cite a couple of LaTeX packages. Of course, I could just enter all the respective BibTeX entries manually. However, given how many package documentations cite other packages, I hope that there is some BibTeX database of all CTAN packages available, somewhere?

I found it extremely hard to look for this. As you can imagine, googling for "latex package" AND bibliography or variants thereof does yield many, but no useful results.

  • 5
    I don't know of such a bibliography but you can query ctan and get xml-files: ctan.org/help/access. It shouldn't be to difficult to create bib-files from them. – Ulrike Fischer Sep 12 '14 at 9:39
18

Edit: The code is now hosted on Github.

Thanks to Ulrike, who pointed out existence of the xml interface on CTAN, I was able to create simple and dumb lua script, using wget command and my luaxml package.

The idea is that you pass the package name as argument for this script and biblatex entry is printed on the standard output. First whole script, ctanbib.lua:

#!/usr/bin/env texlua
kpse.set_program_name("luatex")
-- ctanbib.lua -- export ctan entries to bib format
--

if #arg < 1 or arg[1]=="--help" or arg[1]=="-h" then 
print [[ctanbib - convert ctan package information to bibtex format
Usage:
  texlua ctanbib <package name>

This command will bibtex entry to the terminal output
]]
os.exit(1)
end

local pkgname = arg[1]
local url = "https://www.ctan.org/xml/pkg/" .. pkgname

-- change that for different title scheme
local titleformat = "The %s package"

local bibtexformat = [[
@manual{$package,
title = {$title},
subtitle = {$subtitle},
author = {$author},
url = {$url},
urldate = {$urldate}, 
date = {$date},
version = {$version}
}
]]

local xml = require('luaxml-mod-xml')
local handler = require('luaxml-mod-handler')

local load_xml =  function(url)
    local command = io.popen("wget -qO- ".. url,"r")

    local info = command:read("*all")
    command:close()

    if string.len(info) == 0 then
        return false
    end
    --print(pkginfo)
    treehandler = handler.simpleTreeHandler()
    treehandler.options.noReduce = {authorref=true}
    x = xml.xmlParser(treehandler)
    x:parse(info)
    return treehandler.root
end

local get_authors = function(a)
    local authors = {}
    local retrieved_authors = {}
    -- fix LuaXML "feature" 
    if #a == 0 then a = {a} end
    for _, author in ipairs(a) do
      local current = {}
      current[#current+1] = author._attr.familyname
      current[#current+1] = author._attr.givenname
      table.insert(retrieved_authors, table.concat(current, ", "))
    end
    return table.concat(retrieved_authors," and ")
end

local get_title = function(title)
    local title = title:gsub("^(.)", function(a) return unicode.utf8.upper(a) end)
    return string.format(titleformat, title)
end


local get_url = function(home)
    local home = home or {}
    local attr = home["_attr"] or {}
    local href = attr.href or "http://www.ctan.org/pkg/"..pkgname
    return href
end

local get_version = function(version)
    local version = version or {}
    local attr = version["_attr"] or {}
    return attr["number"], attr["date"]
end

local bibtex_escape = function(a)
    local a = a or ""
    return a:gsub("([%$%{%}])", function(x) return '\\'..x end)
end

local compile = function(template, records)
  return template:gsub("$([a-z]+)", function(a) 
        return bibtex_escape(records[a]) or ""
    end)
end

local entry = load_xml(url)

if not entry then
    print("Cannot find entry for package "..pkgname)
    os.exit(1)
end

-- root element is also saved, so we use this trick 
local record = entry.entry

local e = {}

e.author = get_authors(record.authorref)
e.package = pkgname
e.title = get_title(record.name)
e.subtitle = record.caption
e.url = get_url(record.home)
e.version, e.date = get_version(record.version)
e.urldate = os.date("%Y-%m-%d")

local result = compile(bibtexformat, e)

print(result)

to understand this script we need to look at the xml files from ctan:

<entry id="pgf">
<name>pgf</name>
<caption>Create PostScript and PDF graphics in TeX</caption>
<authorref id="auth:tantau"/>
<authorref id="auth:feuersaenger"/>
<copyright owner="Till Tantau" year="2005-2014"/>
<license type="lppl1.3"/>
<version number="3.0.0" date="2013-12-20"/>
<description>PGF is a macro package for creating graphics.
        It is platform- and format-independent and works together
        with the most important TeX backend drivers, including pdftex and
        dvips. It comes with a user-friendly syntax layer called TikZ. 
        <p/>
        Its usage is similar to
        <ref refid="pstricks">pstricks</ref> and the standard picture
        environment.  PGF works with plain (pdf-)TeX, (pdf-)LaTeX, and
        ConTeXt.  Unlike <ref refid="pstricks">pstricks</ref>, it
        can produce either PostScript or PDF output.</description>
<documentation details="Readme" href="ctan:/graphics/pgf/base/README"/>
<documentation details="PGF Manual" href="ctan:/graphics/pgf/base/doc/pgfmanual.pdf"/>
<documentation details="Minimal introduction to TikZ" href="http://cremeronline.com/LaTeX/minimaltikz.pdf"/>
<home href="http://sourceforge.net/projects/pgf/"/>
<ctan path="/graphics/pgf/base" file="true"/>
<install path="/graphics/pgf/base/pgf.tds.zip"/>
<miktex location="pgf"/>
<texlive location="pgf"/>
<keyval key="topic" value="graphics-in-tex"/>
<keyval key="index" value=""/>
</entry>

note, that authors are saved in authorref elements, which doesn't contain names, but only reference to other file, which hold author info. Also note, that there is no usable title, only package name and caption, which could be used as subtitle.

xml file with author info looks like that:

<author key="tantau" givenname="Till" familyname="Tantau"/>

so for each author, we need to fetch also this file with name.

now we may look at some parts of this script:

local titleformat = "The %s package"

We need format for title because we get only package name.

local bibtexformat = [[
@manual{$package,
title = {$title},
subtitle = {$subtitle},
author = {$author},
url = {$url},
urldate = {$urldate}, 
date = {$date},
version = {$version}
}
]]

These are formats for printing the bibtex entry. $strings will be replaced with corresponding parsed fields.

local load_xml =  function(url)
    local command = io.popen("wget -qO- ".. url,"r")

    local info = command:read("*all")
    command:close()

    if string.len(info) == 0 then
        return false
    end
    --print(pkginfo)
    treehandler = handler.simpleTreeHandler()
    x = xml.xmlParser(treehandler)
    x:parse(info)
    return treehandler.root
end

this function fetch the url and returns parsed xml.

local get_authors = function(a)
    local authors = {}
    local retrieved_authors = {}
    if #a == 0 then 
        table.insert(authors,a) 
    else
        authors = a
    end
    for _, v in ipairs(authors) do
        local id = v["_attr"]["key"]
        local author_url = "http://www.ctan.org/xml/author/".. id
        local authorxml = load_xml(author_url)
        if not authorxml then
            print("Warning: cannot load author info for: " .. id)
        else
            local x = authorxml.author["_attr"]
            table.insert(retrieved_authors, x.familyname.. ", " .. x.givenname)
        end
    end
    return table.concat(retrieved_authors," and ")
end

function load_authors is the most complex, due to limitation of luaxml, because it uses different table structure when there is only one element and when two or more elements exists. after table normalization, xml files are processed for each author and author field is build.

local get_title = function(title)
    local title = title:gsub("^(.)", function(a) return unicode.utf8.upper(a) end)
    return string.format(titleformat, title)
end

the first letter of package name is upper cased (we use unicode library in order to support accented characters). Then titleformat is used to build a title.

local get_url = function(home)
    local home = home or {}
    local attr = home["_attr"] or {}
    local href = attr.href or "http://www.ctan.org/pkg/"..pkgname
    return href
end

some packages have homepage set, for these which doesn't, we use ctan path

local get_version = function(version)
    local version = version or {}
    local attr = version["_attr"] or {}
    return attr["number"], attr["date"]
end

parse version number and release date, not all packages have these!

local bibtex_escape = function(a)
    local a = a or ""
    return a:gsub("([%$%{%}])", function(x) return '\\'..x end)
end

escape special characters. this is very basic version. I assume biblatex and biber so utf8 encoding is used. If we want to encode accented characters to bibtex codes (like {\v{c}}), it would make the script much more complex.

local compile = function(template, records)
  return template:gsub("$([a-z]+)", function(a) 
        return bibtex_escape(records[a]) or ""
    end)
end

function to place values into the template. all fields are in the records table

local e = {}

e.author = get_authors(record.authorref)
e.package = pkgname
e.title = get_title(record.name)
e.subtitle = record.caption
e.url = get_url(record.home)
e.version, e.date = get_version(record.version)
e.urldate = os.date("%Y-%m-%d")

local result = compile(bibtexformat, e)

print(result)

there is a table with all fields built and the bibtex entry is printed.

Now we can try the code:

texlua ctanbib.lua tex4ht

will print:

@manual{tex4ht,
title = {The Tex4ht package},
subtitle = {Convert (La)TeX to HTML/XML},
author = {Gurari, Eitan M. and Berry, Karl and Radhakrishnan, C. V.},
url = {http://tug.org/tex4ht},
urldate = {2014-09-12}, 
date = {2008-06-16},
version = {}
}

we may use standard methods to pipe output to the file with

texlua ctanbib.lua tex4ht > sample.bib

and some more files

texlua ctanbib.lua luaxml >> sample.bib
texlua ctanbib.lua pgf >> sample.bib

we may test the result with some sample file:

\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{biblatex}
\addbibresource{sample.bib}
\begin{document}
\nocite{*}
\printbibliography
\end{document}

and the result:

enter image description here

  • If I just create the Lua file and run the command line shown, I get ctanbib.lua:32: module 'luaxml-mod-xml' not found:. And, yes, I have /usr/local/texlive/2014/texmf-dist/tex/luatex/luaxml/luaxml-mod-xml.lua – egreg Sep 12 '14 at 14:24
  • A line kpse.set_program_name("luatex") seems missing. – egreg Sep 12 '14 at 14:31
  • After adding the line mentioned by @egreg it works for me. Great work, I particularly like the attempt to use the lua interpreter provided by the the TeX distribution anyway :-) – Daniel Sep 12 '14 at 14:37
  • One caveat: The output of accented characters is UTF8. This of course is fine with biblatex and bibtex8, but many scientific venue require .bbl-files produced by archaic 7-bit bibtex. – Daniel Sep 12 '14 at 14:46
  • @egreg you are right. I created this answer on my work computer with TL 2012, so I had to manually copy luaxml to the working dir and haven't notice that this doesn't work even with newer versions – michal.h21 Sep 12 '14 at 15:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.