How to manipulate a multi-line string in luatex?

Question

I want to write a Lua function that processes some multi-line string.

Thus, I created a new file test.lua which contains

userdata = userdata or {}
function userdata.surroundwithdashes(str)
  str = str:gsub("\n", " (linebreak)\\\\ ")
  tex.print("--" .. str .. "--")
end

and added \directlua{dofile('test.lua')} to my latex preamble. To call that function, I created a macro \newcommand{\dash}[1]{\directlua{userdata.surroundwithdashes([[#1]])}}.

This works well for regular text like

\dash{This is some test text}

However, It does not recognize line breaks

\dash{This is some test text
      with more than one line}

And throws errors (undefined control sequence) if I insert them manually

\dash{This is some test text \\
      with more than one line}

Also, this produces an additional "m" after the "TeX" for

\dash{This is some \TeX text}

How can I parse the input string reliably and detect newlines in it?

Answer 1

The \\ gets expanded before it is passed to the Lua function userdata.surroundwithbraces. Because the expansion is an expansion like \edef and not LaTeX's \protected@edef, macro \\ breaks.

Expansion can be prevented by e-TeX's \detokenize.

Also I prefer \luaescapestring (or LuaLaTeX's \luatexluaescapestring) for passing string arguments, because the primitive takes care of the necessary escapings:

\RequirePackage{filecontents}
\begin{filecontents*}{dash.lua}
userdata = userdata or {}
function userdata.surroundwithdashes(str)  
  str = str:gsub("\n", " (linebreak)\\\\ ")
  tex.print("--" .. str .. "--")
end
\end{filecontents*}
\directlua{dofile('dash.lua')}

\documentclass{article}

\newcommand*{\dash}[1]{%
  \directlua{userdata.surroundwithdashes(%
    "\luatexluaescapestring{\detokenize{#1}}")}%
}

\begin{document}
  \dash{This is some test text}

  \dash{This is some test text \\
    with more than one line}
\end{document}

Answer 2

except within verbatim or obeylines etc end of line is reported to the macro layer as a space (or as \par if a blank line) so the macro \dash has an argument with a single space token between text and with