How to read a file with luatex

Question

I tried to read a file with lua and set the content as normal text in my document, but I didn't worked it out. Here my code:

\documentclass{scrartcl}
\usepackage{luatextra}
\usepackage{filecontents}

\begin{filecontents}{testdata.dat}
  A  B
  1.0 20
  1.1 21
  1.2 22
\end{filecontents}

\begin{luacode}
  function readtxt()
    file = io.open("testdata.dat", "r")
    text = file:read("*all")
    print(text)
    return tex.print(text) 
  end
\end{luacode}

\begin{document}
  \directlua{readtxt()}
\end{document}

But nothing is printed to the document although lua diplays the content of the file in the terminal.

Answer 1

Edit: a short version of readtxt() function.

\begin{luacode*}
  function readtxt()
    file = io.open("testdata.dat", "r")
    tex.print(string.split(file:read("*a"),"\n"))
  end
\end{luacode*}

First proposition: a solution consists to store lines in a table and to "tex.print" the lines one by one (each argument of tex.print is a single line of TeX).

\documentclass{scrartcl}
\usepackage{luatextra}
\usepackage{filecontents}

\begin{filecontents}{testdata.dat}
  A  B
  1.0 20
  1.1 21
  1.2 22
\end{filecontents}

\begin{luacode}
  function readtxt()
    file = io.open("testdata.dat", "r")
    for line in file:lines() do
      print(line)
      tex.print(line)
    end
  end
\end{luacode}

\begin{document}
  \directlua{readtxt()}
\end{document}

Answer 2

io.lines also closes the file and tex.sprint writes a new line in the output for every call. The reason why the %-lines are ignored by TeX automatically

\documentclass[a5paper]{article}
\usepackage{luacode,fancyvrb}
\pagestyle{empty}
\begin{filecontents}{testdata.dat}
  A  B
  1.0 20
  1.1 21
  1.2 22
\end{filecontents}

\begin{luacode*}
  function readtxt(filename,suffix)
    suffix = suffix or ""
    for line in io.lines(filename) do tex.sprint(line..suffix) end
  end
\end{luacode*}

\begin{document}
\VerbatimInput[frame=single]{testdata.dat}

\directlua{readtxt("testdata.dat")}

\directlua{readtxt("testdata.dat","\string\\par")}
\end{document}    

Answer 3

The file being written by filecontents looks like this:

%% LaTeX2e file `testdata.dat'
%% generated by the `filecontents' environment
%% from source `luareadfile' on 2013/02/12.
%%
  A  B
  1.0 20
  1.1 21
  1.2 22

When lua reads that in to the string, it does no processing. If TeX were to read that in, it would ignore the comment lines since everything from a % to a newline gets eaten up. Moreover, TeX would translate the newlines to space characters (and translate any double newlines to \par tokens).

The file then gets inserted into the TeX stream via tex.print. This, it would appear, is inserted as if the input parser has already done its job. So TeX assumes that double newlines have already been replaced by \pars and treats a newline character as simply a space token. However, it still sees the % character and is prepared to deal with it by ignoring anything after it up to a newline. But there are no newlines, so TeX keeps discarding stuff until it is stopped (at the end of the tex.print).

I don't know if it is possible to fix this "properly" by getting the new lines to be interpreted correctly, but if you have control over the file contents (as you appear to here) you could get lua to strip out the comments first using a simple pattern match before presenting the data to TeX.

\documentclass{scrartcl}
%\url{http://tex.stackexchange.com/q/97822/86}
\usepackage{luatextra}
\usepackage{filecontents}

\begin{filecontents}{testdata.dat}
  A  B
  1.0 20
  1.1 21
  1.2 22
\end{filecontents}

\begin{luacode}
  function readtxt()
    file = io.open("testdata.dat", "r")
    text = file:read("*all")
    clean = string.gsub(text,'%%[^\string\n]*',"")
    print(clean)
    return tex.print(clean) 
  end
\end{luacode}

\begin{document}
Reading in text:
  \directlua{readtxt()}
Finished reading in text.
\end{document}

Notice the \string! That's because your lua code is embedded in your document. It's usually a good idea to avoid that by having the lua code in a separate file.

This produces:

Reading in text: A B 1.0 20 1.1 21 1.2 22 Finished reading in text.

Answer 4

Another (besides @AndrewStacey's solution) possibility is to use string.gmatch() and insert the file line by line:

\documentclass{scrartcl}
\usepackage{luatextra}
\usepackage{filecontents}

\begin{filecontents}{testdata.dat}
  A  B
  1.0 20
  1.1 21
  1.2 22
\end{filecontents}

\begin{luacode*}
  function readtxt()
    file = io.open("testdata.dat", "r")
    text = file:read("*all")
    file:close()
    for x in string.gmatch(text,"[^\n]+") do
        tex.print(x)
    end
  end
\end{luacode*}

\begin{document}
  \directlua{readtxt()}
\end{document}

And if you want to insert your file verbatim, you can use the -2 catcode table:

tex.print(-2,text)

Which tells TeX not to interpret the contents of the string.