How can I get all TeX / LaTeX symbols?

Question

I am currently writing the practical part of my bachelors thesis which will be about on-line recognition of handwritten mathematical symbols (and eventually also formuae).

One problem I currently have with this is that I need to have the symbols in my database. I basically need

• The latex code for the symbol (e.g. A, $\alpha$, $\rightarrow$, $\mathbb{R}$, ...)
• The package name if I need one (e.g. \usepackage{amsmath})
• SVG

I can create the SVG with the other two pieces of information (although the SVGs I can create are far from optimal), but I currently don't see a better way than manually entering those symbols. This seems like a tremendous waste of time, hearing from the developer of detexify that he has about 2000 symbols (but he can't give me the database - I don't know why).

So: How can I get a list of all symbols LaTeX / TeX has?

(By the way, the site is http://write-math.com - but it still is a very early alpha release and I might truncate the database in the next few days, so please don't waste your time and generate data. The symbol recognition part doesn't exist at the moment.)

Answer 1

As mentioned in the comments, the source for detexify is available on GitHub at https://github.com/kirel/detexify.

The symbols database is provided at lib/latex/symbols.yaml in a YAML format (sample below) which is easily extensible and satisfies your requirements for information about the required package(s) and mode (text or math).

# Table 212: wasysym Biological Symbols
- package: wasysym
  bothmodes:
    - \female 
    - \male
- package: skull
  mathmode:
    - \skull
# Table 254: wasysym General Symbols
- package: wasysym
  textmode:
    - \LEFTarrow
    # <...>
    - \recorder
  bothmodes:
    - \checked #
    # <...>
    - \invdiameter #

SVG output can be created by adapting a rake task provided in lib/rake/symbol_task.rb. I present my initial adaptation of this code below. Disclaimer: I am a newbie to Ruby, and there are likely better ways to do this since I'm stripping away most of Daniel's code here. Take this as a starting point rather than the standard. :-)

require 'rake/tasklib'
require 'erb'
require 'latex/symbol'

class SymbolTask < Rake::TaskLib

  TEMPLATE = ERB.new <<-LATEX #open('template.tex.erb').read
    \\documentclass[10pt]{article}
    \\usepackage[utf8]{inputenc}

    <%= @packages %>

    \\pagestyle{empty}
    \\begin{document}

    <%= @command %>

    \\end{document}
  LATEX

  TMP = 'tmp'
  OUT = 'images/latex'

  attr_accessor :name, :tmp, :out

  # initialize sets the name and calls a block to get
  #   the rest of the options
  def initialize(name = :symbols)
      @name = name
      yield self if block_given?
      define
  end

  # define creates the new task(s)
  def define
    #desc "prepare necessary directories"
    #task :prepare do
      directory TMP
      directory OUT
    #end

    all_image_tasks = Latex::Symbol::List.map do |symbol|
      define_single_tex_task symbol
      define_single_dvi_task symbol      
      define_single_image_task symbol
    end

    desc "create svg images from all symbols"
    task :gen_svg => all_image_tasks
  end

  def define_single_image_task symbol
    file "#{File.join(OUT, symbol.filename)}.svg" => [OUT, "#{File.join(TMP, symbol.filename)}.dvi"] do |t|
      # Now convert to image
      puts "Creating image... #{t.name}"
      sh %|dvisvgm -n #{File.join(TMP, symbol.filename)}.dvi #{t.name}| do |ok, res|
        if ! ok
          puts "Major Failure creating image! (status = #{res.exitstatus})"
        end
      end

    end
    "#{File.join(OUT, symbol.filename)}.svg" # need the names
  end

  def define_single_dvi_task symbol
    file "#{File.join(TMP, symbol.filename)}.dvi" => [TMP, "#{File.join(TMP, symbol.filename)}.tex"] do
      puts "Generating dvi for #{symbol}..."
      sh %|latex -interaction=batchmode -output-directory=#{TMP} #{File.join(TMP, symbol.filename)}.tex| do |ok, res|
        if ! ok
          puts "Major Failure creating dvi! (status = #{res.exitstatus})"
        end
      end
    end
  end

  def define_single_tex_task symbol
    file "#{File.join(TMP, symbol.filename)}.tex" => TMP do |t|
      open(t.name, 'w+') do |texfile|
        # setup variables
        @packages = ''
        @packages << "\\usepackage{#{symbol[:package]}}\n" if symbol[:package]
        @packages << "\\usepackage[#{symbol[:fontenc]}]{fontenc}\n" if symbol[:fontenc]
        @command = symbol.mathmode ? "$#{symbol.command}$" : symbol.command
        # write symbol to tempfile
        puts "Generating latex for #{symbol}..."
        texfile.puts TEMPLATE.result(binding)
      end
    end
  end

end

I made these modifications on a clone of the detexify repository, but the only files required are lib/rake/symbol_task.rb, lib/latex/symbol.rb, and lib/latex/symbols.yaml.

To build all SVG outputs, execute rake gen_svg at a Ruby command prompt. dvisvgm is required; alternatively, the Ruby code could be modified to use pdflatex with pdf2svg (untested) or Inkscape (on the command line) to perform the conversion to SVG.