# LuaTeX gets stuck when “.” is added to pattern-expression

today on SE I got inspired to try LuaTeX.

I'm facing a problem where the compilation process stalls at some point if I add a single "." to a pattern matching expression.

In short, this works:

line = string.gsub ( line , "^(%s*)"    ,   "%1" )


this doesn't:

line = string.gsub ( line , "^(%s*)."   ,   "%1" )


(I know this pattern doesnt make toooo much sense, but it's a reduced version of a longer expression)

Compilation stalls at:

This is LuaTeX, Version beta-0.80.0 (TeX Live 2015/Debian) (rev 5238)
restricted \write18 enabled.
(./mwe4.tex
LaTeX2e <2015/01/01> patch level 2
[...]
(/usr/share/texlive/texmf-dist/tex/luatex/luatexbase/cctb.lua))
(/usr/share/texlive/texmf-dist/tex/luatex/luatexbase/luatexbase-mcb.sty
(/usr/share/texlive/texmf-dist/tex/luatex/luatexbase/mcb.lua))))  (./mwe4.aux))
*


A full MWE is here (the critical line being #6):

\documentclass{article}
\usepackage{luacode,luatexbase}

\begin{luacode}
function replacetext2 ( line )
line = string.gsub ( line , "^(%s*)"    ,   "%1" )
return ( line )
end
\end{luacode}

\newenvironment{haystack}{%
"process_input_buffer",
replacetext2, "replacetext2")}}{%
\directlua{luatexbase.remove_from_callback(
"process_input_buffer",
"replacetext2")}}

\begin{document}
\begin{haystack}
a
b
\end{haystack}
\end{document}

• I can reproduce the exact same behaviour. – moewe Sep 5 '15 at 14:18

The pattern ^(%s*). in

line = string.gsub ( line , "^(%s*)."    ,   "%1" )


looks for any white space (can be empty) at the begin of line, followed by one character (.). This is replaced by the white space (%1) and the matched non-white-space character is removed. Therefore LuaTeX will no longer see \end{haystack}, but end{haystack} and keeps looking for the end of the environment.

The example with some debug output to show the line in brackets, before and after the substitution:

\documentclass{article}
\usepackage{luacode,luatexbase}

\begin{luacode}
function replacetext2 ( line )
texio.write_nl('')
texio.write('[', line, '] --> [')
line = string.gsub ( line , "^(%s*)."    ,   "%1" )
texio.write(line, "]")
texio.write_nl('')
return ( line )
end
\end{luacode}

\newenvironment{haystack}{%
"process_input_buffer",
replacetext2, "replacetext2")}}{%
\directlua{luatexbase.remove_from_callback(
"process_input_buffer",
"replacetext2")}}

\begin{document}
\begin{haystack}
a
b
\end{haystack}
\end{document}


Result:

[a] --> []
[ b] --> [ ]
[\end{haystack}] --> [end{haystack}]
[\end{document}] --> [end{document}]


If you only want to remove a real dot at the beginning of a line, then the dot needs to be escaped as magic character %., since the pure dot . means any character.

• Thanks for the detailed analysis. I wasn't even aware that the \end{..} and \begin{..} were also passed on to replacetext2 – sheß Sep 5 '15 at 14:44
• @sheß - The process_input_buffer callback operates at a very early stage, before TeX does much of its usual work. Once replacetext2 is activated (via \begin{haystack}), it continues to operate until the end of the haystack environment is encountered. However, because of the way you wrote replacetext2, the  (backslash) in \end{haystack} got removed (unintentionally, of course). Because end{haystack} (without the backslash) does not terminate the haystack environment, replacetext2 was not removed from the process_input_buffer callback, leading to all sorts of havoc. – Mico Sep 5 '15 at 20:17