How do I extract the definition(s) of a Chinese word from the CC-EDICT dictionary?

Question

I want to create a command in LaTeX, like this:

\chinese{炒面}

And have it create a footnote with the definition given from the CC-EDICT dictionary.

炒麵 炒面 [chao3 mian4] /stir-fried noodles/"chow mein"/

The CC-EDICT dictionary...

The dictionary is a downloadable text file, with ~118,000 lines like

...
炒雞蛋 炒鸡蛋 [chao3 ji1 dan4] /scrambled eggs/
炒飯 炒饭 [chao3 fan4] /fried rice/(slang) (Tw) to have sex/
炒魷魚 炒鱿鱼 [chao3 you2 yu2] /(coll.) to fire sb/to sack sb/
炒麵 炒面 [chao3 mian4] /stir-fried noodles/"chow mein"/
...

The first word is the traditional Chinese, the second is the simplified Chinese. In brackets is the pronunciation. And between the slashes is the definition.

I tried \usepackage{datatool}, and it could manage the first 1000 lines or so, but nothing in the ballpark of ~118,000 lines. It seems it tries to load the whole file into memory.

Using grep I could use grep " 炒面 " cedict_ts.u8 to get the appropriate line(s) for simplified Chinese, but I don't know how to do this in LaTeX.

Multiple definitions...

Sometimes there are multiple definitions:

得 得 [de2] /to obtain/to get/to gain/to catch (a disease)/proper/suitable/proud/contented/to allow/to permit/ready/finished/
得 得 [de5] /structural particle: used after a verb (or adjective as main verb), linking it to following phrase indicating effect, degree, possibility etc/
得 得 [dei3] /to have to/must/ought to/to need to/

Ideally these would be all included, and separated in some way.

Question: How do I extract the definition(s) of a Chinese word from the CC-EDICT dictionary?

Answer 1

I've created a simple package for LuaLaTeX: ccedict.sty:

\ProvidesPackage{ccedict}
\RequirePackage{luacode}
\RequirePackage{luatexja-fontspec}
\setmainjfont{FandolSong}

\begin{luacode*}
local dict = {}

function load_ccedict(filename)
  for line in io.lines(filename) do
  local traditional, simplified, spelling, description = line:match("^(.+)%s+(.+)%s+%[(.-)%]%s*(.+)")
    if traditional then
      -- insert new record for the current header
      local rec = dict[simplified] or {}
      table.insert(rec, {spelling = spelling, description = description, simplified = simplified, traditional = traditional})
      dict[simplified] = rec
    end
  end
end

function ccedict_get_term(term)
  local rec = dict[term] or {{spelling="cannot find term", description = "", simplified = term, traditional=""}}
  local new = {}
  local traditional, simplified 
  for _, v in ipairs(rec) do
  -- this will be printed in the footnote
  new[#new+1] = " [" .. v.spelling  .. "] " .. v.description 
  -- save the simplified and traditional terms
  traditional, simplified = v.traditional, v.simplified
  end
  -- you may want to add some separator betweend traditional and simplified terms
  -- the multiple terms will be separated using semicolon
  return traditional .. ": " .. simplified .. " " ..table.concat(new, "; ")
end

\end{luacode*}

\newcommand\loadccedict[1]{\directlua{load_ccedict("\detokenize{#1}")}}

\loadccedict{cedict_ts.u8}

\newcommand\chinese[1]{#1\footnote{\directlua{tex.sprint(ccedict_get_term("\detokenize{#1}"))}}}


\endinput

There are two Lua functions - load_ccedict that loads the dictionary and makes a look up table, and ccedict_get_term that returns the term information. The command \chinese uses this information to print a footnote.

It uses the luatexja-fontspec package with the FandolSong font to get out of the box support for Chinese. You may want to use a different font, as I had found that it doesn't support all traditional Chinese characters.

Here is a sample document:

\documentclass{article}
\usepackage{ccedict}
\begin{document}

Hello \chinese{炒面}, \chinese{得}
\end{document}

And a result:

(you can see the missing character in the first footnote. I don't know anything about Chinese fonts, so I cannot tell which one would have a better support for traditional Chinese)