# Parsing files through LuaTeX

This question has two parts:

1. How would one generally parse a file with LuaTeX, assuming the file only holds data that is delimited either by spaces, commas or semicolons.

2. How would you parse the file (if it includes (all)TeX code as well as text code), for example an ini file where some of the fields are written in TeX.

This question aims at finding a solution that is also well documented in the post. As LuaTeX is still in heavy development, I find the documentation to be sparse and this is the first of a number of questions that I intend to post (with bounties) so that the answers can provide some form of documentation on the web (and on our favourite Site!).

Edit:

The first part is a simple CSV file. If you want you can use the example in my answer in
Adding a list of bios to the book class. As the answer included alpha sorting using TeX, it might be a good example, where LuaTeX should have an advantage over normal (all)TeX.

Ini files are normally used as configuration files, although I have used them for all sort of data capturing. If you wish you can also use a Yaml format.

[general]
languages=en,ngerman

;This is a comment
[article]
pagewidth=15cm
pageheight=20cm
paper=a4

[book]
includeparts=true

;defines chapter
[chapter]

;packages
[graphicx]
keys="keya,keyb,keyc"

For TeX code, add one command to the ini file.

texcode="\def\test#1#2{}"
-
An example of such input file should help. In the meantime general lua documentation is your friend as the issues seems fairly generic (sans the escaping of TeX code or feeding it back to TeX). –  Khaled Hosny Apr 10 '11 at 17:14
@Khaled Hosny added some clarifications as requested. –  Yiannis Lazarides Apr 10 '11 at 20:45
Do you want to parse it using TeX code or Lua code? The latter would be simpler. You only have to think you to return the result back to TeX. –  Martin Scharrer Apr 10 '11 at 20:52
@Martin Scharrer Parsing should be in Lua code. Lua code must be invoked from LaTeX, i.e., not a standalone program. –  Yiannis Lazarides Apr 10 '11 at 22:05
For .ini files, there are some pure lua modules that you can simply plug in. See e.g. luaforge.net/projects/inilazy (there is a usage example in the zip archive) –  Taco Hoekwater Apr 12 '11 at 13:45

Following the two comments for the actual cases I wrote above, I think the general approach goes like this:

• If the file format you want to parse is more or less standardized, there usually is already some lua code out there in the wild that can read it and produce a lua table from the file's contents. Otherwise, usually the best solution is to use LPEG to do the same (see links in Aditya's reply).

• Once that is done, you have to figure out how to feed the file name to the existing (or new lua) code that does the actual parsing, and what the structure is of the returns values.

• Chances are high that the result of parsing is a Lua table, and you just have to tex.sprint() the bits you want.

For example, here is how the \directlua input for the inilazy module could look:

\directlua { require ("inilazy")
local tt = assert (inilazy.get"try.ini")
for k,v in pairs(tt['graphicx']['keys']) do
tex.print(k)
end
}
-

One possibility is to use LPEG. ConTeXt uses LPEG (with some enhancements) to

These files in themselves are not documented. To understand the usage, you will need to read the corresponding *.mkiv file where the lua functions are called.

-