8

This seems to have come up a couple times, but I cannot glean an answer to the generic question of how one might protect an arbitrary string in Lua to ensure its characters are printed.

In other words, given some code in a .lua file:

local i_can_be_anything = '&\\%*_'
context(i_can_be_anything)

This balks for any number of reasons, because the characters have special meaning in TeX when ConTeXt pushes them out to the TeX compiler.

The most apparent option to me would be to convert the string in Lua so that it contains the proper escaping or substitutions that can be passed to TeX to achieve the desired outcome, which would be in this case along the lines of:

\&\textbackslash{}\%\&\_

I believe the full list of replacements one would be looking for would be along the lines of (and this is from some Python I wrote):

tex_replacements = [
    (u'{', ur'\{'),
    (u'}', ur'\}'),
    (u'[', ur'{[}'),
    (u']', ur'{]}'),
    (u'\\', ur'\textbackslash{}'),
    (u'$', ur'\$'),
    (u'%', ur'\%'),
    (u'&', ur'\&'),
    (u'#', ur'\#'),
    (u'_', ur'\_'),
    (u'~', ur'\textasciitilde{}'),
    (u'^', ur'\textasciicircum{}'),
]

tex_mapping =  {ord(char):rep for char, rep in tex_replacements}
def escape(s):
    return unicode(s).translate(tex_mapping)

Is there no function in Lua or LuaTeX to achieve this set of substitutions (or otherwise) so that an arbitrary string could be printed?

A straightforward iteration over the characters with a substitution along the lines of the tex_replacements above could do the trick, but I would expect that someone had already written a solution to this problem.


EDIT

It is worth noting that @phg posed a plausible solution of changing the catcodes to vrbcatcode (i.e. \verbatim, or context.sprint(catcodes.numbers.vrbcatcodes, i_can_be_anything)). This does perform the escaping desired, however it seems to have serious issues with line breaking that make it essentially unusable.

7
  • 1
    The solution you are looking for is switching the catcode table. vrbcatcodes should suffice for most literal printing, but if you like you can skim the list for a more appropriate one: git.contextgarden.net/context/context/blob/master/tex/context/… (Btw. there is no such thing as a “TeX compiler”.) Jul 23, 2014 at 21:21
  • Thanks @phg. I am afraid it is not apparent from the link how one employs said catcodes. Incidentally, and aside, if there is no such thing as a TeX compiler, what is the mechanism by which the TeX source is converted to machine instructions? Jul 23, 2014 at 21:30
  • You can use \pushcatcodetable \setcatcodetable \somecatcodetable to set the table and \popcatcodetable to drop it again (that’s context.pushcatcodetable() context.setcatcodetable(catcodes.numbers.vrbcatcodes) / context.popcatcodetable() at the Lua end, or use context.sprint(n, ...) with the catcode table as the first argument). As for the terminology: TeX is interpreted by a TeX engine. Jul 23, 2014 at 21:52
  • (Untested) context(catcodes.numbers.vrbcatcodes, i_can_be_anything) should also work.
    – Aditya
    Jul 24, 2014 at 4:14
  • 1
    @Aditya Are you sure? It calls tex.sprint() explicitly with the currently active catcode table. There is context.verbatim(), though, which sets the vrbcatcodes table. Jul 24, 2014 at 5:24

2 Answers 2

5

This answer is based on an email from Hans Hagen. One way to achieve the desired output is using formatters. For example:

\starttext
\startluacode
  local text='&\%*_'
  context("%!tex!", text)
\stopluacode
\stoptext

The %!tex! is plug to the formatters and tex is one of the pre-defined formatters. Other pre-defined formatters are lua and xml (which escape strings for Lua and XML, respectively).

The above code is equivalent to

context(string.formatters["%!tex!"](text))

which is equivalent to the rather low-level

context(lpeg.match(lpeg.patterns.texescape,text))

The corresponding lpeg patterns for lua and xml formatters are lpeg.patterns.luaescape and lpeg.patterns.xmlescape. These are defined in util-str.lua as:

patterns.xmlescape = Cs((P("<")/"&lt;" + P(">")/"&gt;" + P("&")/"&amp;" + P('"')/"&quot;" + P(1))^0)
patterns.texescape = Cs((C(S("#$%\\{}"))/"\\%1" + P(1))^0)
patterns.luaescape = Cs(((1-S('"\n'))^1 + P('"')/'\\"' + P('\n')/'\\n"')^0) -- maybe also \0
patterns.luaquoted = Cs(Cc('"') * ((1-S('"\n'))^1 + P('"')/'\\"' + P('\n')/'\\n"')^0 * Cc('"'))

The luaquoted pattern is not used in the core.

3
  • Thanks Aditya - why do you think one would prefer these over the discussed catcodes? other than Hans saying so, which ought to be compelling enough :-) —just curious. Jul 29, 2014 at 10:36
  • One note, context("%!tex!", text) seems to handle "\" in an unintuitive way. For example, the string "\escape" results in "<LINEBREAK>escape". This may not be the desired behaviour. The catcodes solution does not have this particular caveat. Jul 29, 2014 at 13:12
  • It converts \ to \\ , which I guess is mapped to \crlf.
    – Aditya
    Jul 29, 2014 at 14:26
1

I am summarizing the comments here, but the gist is:

@phg suggested that one can escape by using catcodes. There is the verbatim catcode, but it has issues with linebreaks.

An alternative, albeit undocumented but seemingly preferable (i.e. escapes without wrapping issues), catcode is \notcatcodes.

One can employ this in Lua to escape any string with the following, for example:

context.sprint(catcodes.numbers.notcatcodes, str)
-- or --
context.pushcatcodetable()
context.setcatcodetable(catcodes.numbers.vrbcatcodes)
context(str)
context.popcatcodetable()

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .