In which order is Lua Code in LuaTeX being processed

Question

Can someone help me understand the order in which LuaTeX documents are being processed?

From what I've understood, the order is the following:

• the code is being read, top to bottom
• when the compiler finds the start of some Lua code it swallows it, expands any TeX commands it can find and passes the result to a Lua interpreter

• the Lua interpreter interprets the code and puts its output back into the document

• is the output being parsed by again? Is it just put into the document as is?

To illustrate my question:

\newcommand{\compare}[2]{
#1
\directlua{
    testNumber=#2
    if #1<testNumber then
        tex.sprint("is bigger")
    else
        tex.sprint("is smaller")    
    end
    tex.sprint(" than "..testNumber..".")
    }
}

When \compare{1}{5} is called the Lua interpreter is being fed:

testNumber=5
if 1<testNumber then
    tex.sprint("is bigger")
else
    tex.sprint("is smaller")    
end
tex.sprint(" than "..testNumber..".")

And the output would be pasted into the document in the place where the directlua command was called.

Now suppose I add another command:

\newcommand{\iterate}[2]{
    \directlua{
        for i=1,#1,1 do
        tex.print(\\compare{i}{#2})
        end
    }
}

and call it via \iterate{5}{5}

From my limited understanding, this should be fed to the Lua interpreter as follows:

for i=1,10,1 do
tex.print(\compare{i}{5})
end

and the output would be:

\compare{1}{5}
\compare{2}{5}
\compare{3}{5}
\compare{4}{5}
\compare{5}{5}

Is this output being evaluated again? I can't seem to get my code to run, hence I was thinking that I've maybe misunderstood something.

Can this process be understood in terms of a preprocessor-processor relationship such as with, say, a PHP application which outputs JS code which itself is being interpreted by the client?

Answer 1

When you write

\newcommand{\iterate}[2]{
    \directlua{
        for i=1,#1,1 do
        tex.print(\\compare{i}{#2})
        end
    }
}

The Lua process sees

for i=1,#1,1 do
  tex.print(<the result of \compare>)
end

What you need to do is something like that:

\newcommand\iterate[2]{%
  \luaexec{
    for i=1,#1 do
        tex.print(string.format("\\compare{\%d}{#2}",i))
    end
 }
}

\iterate{3}{2}

(luaexec is from the luacode package) This prints "\compare{1}{2} \compare{2}{2} \compare{3}{2}" into TeX's input string and that gets evaluated. This works because \\ generates a backslash and it is separate from the word "compare". It's a hack and I don't recommend that. Please see How to do a 'printline' in LuaTeX for more info.

This is an experiment on "reevaluating":

For example: if you write

\documentclass[a4paper]{article}
\usepackage{luacode}
\begin{document}

\begin{luacode*}
tex.sprint("%hello")
\end{luacode*}

\end{document}

TeX will see %hello and interprets it as a comment and thus no output file will be generated.

When you write this instead (see the -2 as the first argument to tex.print())

\documentclass[a4paper]{article}
\usepackage{luacode}
\begin{document}

\begin{luacode*}
tex.sprint(-2,"%hello")
\end{luacode*}

\end{document}

tex sees %hello, but the % has a "safe" category code. If the first argument to tex.print is a number, it will be taken as a catcode table. What I want to show: TeX reads the result of the \directlua{} call.

Answer 2

FWIW, ConTeXt provides a lua table context that has access to all macros defined at the TeX level. So, you can use context.compare to call the macro \compare defined at the TeX end. Thus, one could translate your macro to ConTeXt as follows (I am following the same definition as your macros. The normal ConTeXt style of writing lua functions is different).

\def\compare#1#2%
    {\directlua{
        local testNumber=#2
        if #1<testNumber then
            context("#1 is bigger")
        else
            context("#1 is not bigger")    
        end
        context(" than "..testNumber..".")
    }}

\def\iterate#1#2%
    {\directlua{
        for i=1,#1,1 do
          context.compare(tostring(i) , #2)
        end}}

\starttext
\iterate{5}{2}
\stoptext

This works using some Lua magic that I don't understand. Internally it is similar to the answer given by rdhs; just that you don't have to manually do the plumbing. See ConTeXt Lua Documents manual for more details.

Answer 3

First, here's a working version of your code:

\newcommand{\compare}[2]{
#1
\directlua{
    testNumber=#2
    if #1<testNumber then
        tex.sprint("is bigger")
    else
        tex.sprint("is not bigger")    
    end
    tex.sprint(" than "..testNumber..".")
    }
}
\newcommand{\iterate}[2]{
    \directlua{
        for i=1,#1,1 do
        tex.print("\noexpand\\compare{" .. i .. "}{#2}")
        end
    }
}
\iterate{5}{5}

Now, the basic sequence of events:

1. \newcommand stores your code in the definition of \iterate. Nothing is executed so far.
2. At some point, \iterate is encountered and your definition is substituted. TeX sees \directlua and tries to expand it.
3. TeX (via \directlua) expands everything within the braces. This means the arguments (#1) but also \\, since \\ is a control sequence. So when you say "\\compare", TeX tries to pass the definition of \\ to Lua. To prevent this, you need to use "\noexpand\\compare". You could also use [[\@backslashchar]] .. "compare" or for extreme brain damage try "\@backslashchar\noexpand\compare".
4. \directlua passes the expanded code to Lua., which makes "\\compare" into \compare.
5. TeX now sees that \directlua expanded to five \compare sequences (and their arguments). It substitutes your definition and sees another \directlua, which it tries to expand exactly as in step 2.

You can "turn off" step 3 by putting the Lua code in a separate file or by using the facilities provided by the luacode package. You can do the same for step 5 with tex.print(-2,"") since that makes Lua's output into a string that TeX won't try to keep expanding.

FWIW, this is the cleanest way to implement \iterate:

\begin{filecontents*}{myiterate.lua}
function mycompare(num1,num2)
    tex.sprint(num1 .. " ")
    if num1<num2 then
        tex.sprint("is bigger")
    else
        tex.sprint("is not bigger")    
    end
    tex.sprint(" than " .. num2 .. ".")
end

function myiterate(reps,targ)
    for i=1,reps,1 do
        tex.sprint([[\compare{]] .. i .. "}{" .. targ .. [[}\\]])
    end
end
\end{filecontents*}

...

\directlua{require("myiterate")}
\newcommand\compare[2]{\directlua{mycompare(#1,#2)}}
\newcommand\iterate[2]{\directlua{myiterate(#1,#2)}}
\iterate{5}{5}