How to suppress a line completely from a LuaTeX process_input_buffer callback?

Question

I'm using a process_input_buffer callback to look at lines read from LuaLaTeX input before passing them on to TeX, to be processed as »verbatim« text (code and command examples). There are some lines in the input that I don't want to pass to TeX at all. If I return »nil«, then TeX gets the original line (as the documentation says). If I return an empty string, an empty line results in the TeX output, which I don't want. How can I arrange for the callback to throw the line away altogether without passing anything on to TeX?

(One obvious approach would be to read all of the example into a buffer and then simply cut out the undesired stuff before passing the complete buffer on to TeX, en bloc. This doesn't work for me due to other considerations.)

The idea is that in an environment like

\begin{example}
  # foo
  bar
  baz
\end{example}

the »#« line should not be passed to TeX but the other lines should. (In the real world, the »#« line is somewhat more complicated but is being processed on the Lua side of things.)

(In the meantime, I found a workaround such that I'm basically inserting a control-B character in front of the »#« on the Lua side, and declaring ^^B to be the fancyvrb »commentchar«, which causes anything after a ^^B, including the newline, to be ignored. I'd still be interested to see whether there is a simpler method.)

Answer 1

You can return an empty string, and if you are in vertical mode, no line will be created.

Another approach is to set \endlinechar to -1 and return the empty string for "don't pass to TeX" or the buffer plus ^^M for the unfiltered line.

Answer 2

Since it's possible to set endlinechar from Lua itself, it's possible to do this:

%! TEX program = lualatex
\documentclass{article}
\usepackage{luacode}
\begin{document}

\begin{luacode*}

luatexbase.add_to_callback("process_input_buffer", function(line)
    tex.endlinechar=13
    if line=="delete this" then
        tex.endlinechar=-1
        return ""
    end
    return line
end, "delete some lines")

\end{luacode*}

\begin{verbatim}
123
456
delete this
789
012
\end{verbatim}

\end{document}

Output is as you expect.

Disadvantage,

• if some special TeX code is placed around that line, it might be able to detect the difference in the value of \endlinechar.

  (probably this can only happen when some code peeks forward for the tokens inside \everyeof and the line is the last line of the file -- because there's no token at all on that line? Not tested)

• it assumes the normal value of endlinechar is 13. (it could be made more sophisticated by remembering the old value)

Answer 3

I don't know what you mean by buffer in LuaLaTeX. In ConTeXt, buffers are best way to pass data from TeX to lua and back. Below is an example (in ConTeXt) that removes lines starting with # and typesets the result of the environment with verbatim catcodes.

\def\startfoo
    {\dostartbuffer[foo][startfoo][stopfoo]}

\def\stopfoo
    {\ctxlua{thirddata.foo.clean_lines(buffers.getcontent('foo'))}}

\startluacode
  thirddata = thirddata or {}
  thirddata.foo = {}
  function thirddata.foo.clean_lines(content)
    local result = {}
    local lines = string.splitlines(content)
    for i = 1, #lines do
      -- Change the test with a more complicated test.
      if not string.find(lines[i], '^#') then
          table.insert(result, lines[i])
      end
    end
    result = table.concat(result, '\n')
    tex.sprint("\\starttyping ")
    tex.sprint(result)
    tex.sprint("\\stoptyping ")
  end
\stopluacode

\starttext
\startfoo
# foo
bar %verbatim # environment
bar
\stopfoo
\stoptext