What is the best practice to pass spaces to Lua from expl3?

Question

In LuaTeX, using the \directlua primitive (within a standard LaTeX2e setup)

\directlua{tex.print("Hello World!")}

will produce "Hello World!" to the output PDF.

On the other hand, in expl3 code

\ExplSyntaxOn
\lua_now:e { tex.print("Hello~World!") }
\ExplSyntaxOff

left "HelloWorld!" (no space between Hello and World!) to the input stream.

What is the best practice to pass spaces to Lua interpreter from expl3 code?

Answer 1

In general, we recommend that Lua chunks are defined separately from expl3 (or indeed standard LaTeX) code, so they can simply be loaded using require(). Where you do want a short piece of Lua 'inside' expl3, using ~ is the correct action. However, when you use tex.print or similar to return tokens to TeX, LuaTeX tokenizes them using the current catcode regime. As such, your space gets ignored. However, there is an optional argument to tex.print() which takes a category code table number: this can be used to apply any catcode setup one likes.

At present, we are finalising category code table support for expl3. The 'Lua side' is perhaps not quite right just yet, but it is workable:

\documentclass{article}
\usepackage{expl3}
\usepackage{l3cctab}
\begin{document}
\ExplSyntaxOn
\lua_now:e { tex.print(\int_use:N \c_document_cctab, "Hello~World!") }
\ExplSyntaxOn
\end{document}

(Eventually, l3cctab will be part of the expl3 core.)

Category code tables are supported by LaTeX nowadays via ltluatex: you could avoid loading l3cctab entirely:

\documentclass{article}
\usepackage{expl3}
\begin{document}
\ExplSyntaxOn
\makeatletter
\lua_now:e { tex.print(\number\catcodetable@latex, "Hello~World!") }
\makeatother
\ExplSyntaxOn
\end{document}

Answer 2

As Joseph said, the default tex.print will tokenize by the catcode current at that point so rather than letting ~ making a space then printing with a normal catcode regime, you can print with the expl3 catcode regime which means that you nneed to print a ~ so need to avoid the ~ being tokenised as a space as it's passed to Lua:

\documentclass{article}
\usepackage{expl3}

\begin{document}
\ExplSyntaxOn
\lua_now:e { tex.print("Hello \c_tilde_str World!") }
\ExplSyntaxOn
\end{document}