# Hyphenating at active characters

Let's say I want to make certain letters active and do some devious things with them, like

\catcodey=13
\defy{\leavevmode\raise.1ex\hbox{\chary}}


(I know this is terrible in that it prohibits using control sequences containing a "y".)
Now I would like TeX to still consider hyphenating at the "y" normally (poly-ethy-lene). I know that TeX won't break a word containing an \hbox, but is it possible to trick TeX into thinking that there is just an innocent letter? Why I think it might be possible:

• TeX doesn't care per se whether or not the characters in a word are active when looking for a suitable breakpoint;
• at the time the active char occurs, it still knows which letter should be there.

A solution as general as possible is encouraged (i.e., regardless of what the active character does), but if that's not feasible, here is what I might want to make such an active character do:

• print itself normally
• insert itself into an \hbox and \raise it
• add some kerns before or after it
• manipulate its appearance with \pdfliterals
• A \special inhibits search for hyphenation points past it, just like \raise or explicit kerns (not an exhaustive list). Hyphenation is tried well after macro expansion has finished. For this job you can make a virtual font. – egreg Mar 27 '20 at 11:01
• @egreg The parameters with which the character is to be modified are determined only at runtime which makes it difficult to create a virtual font I suppose. Could LuaTeX do such a thing? – wave Mar 27 '20 at 11:08
• As far as I know, LuaTeX can build virtual fonts on the fly, see section 6.3 in the manual. – egreg Mar 27 '20 at 11:11
• with luatex you can change glyphs after hyphenation, see e.g. the chickenize package. – Ulrike Fischer Mar 27 '20 at 11:29
• @UlrikeFischer nice recommendation, the documentation of that package is very helpful given that I don't know much about LuaTeX yet – wave Mar 27 '20 at 11:44

Using LuaTeX, it's possible to avoid making the letters active and just manipulate them after hyphenation, as Ulrike Fischer pointed out in the comments.

Following is an implementation of this approach, inspired by the chickenize package. Since this is the first time I'm writing code in Lua, any suggestions are welcome.

transform.lua

First, I define a function iterating through the glyph nodes in a list and checking if the character has an entry in a table called chartbl, in which case it calls a function transform_char that uses the values in the table to manipulate the glyph node. This function is then registered as a post_linebreak_filter, so that it will be applied to a paragraph list after it was broken into lines (therefore obeying hyphenation patterns):

transform_chars = function(head)
chr = n.char
if chartbl[chr] ~= nil then
transformed = transform_char(n)
end
end
end
end

callback.register("post_linebreak_filter",transform_chars)


Now transform_char(n) can be adapted to the specific needs. In this case, we add a kern and a pdfliteral before and after the character and virtually shift the character:

transform_char = function(c)
kbfn = node.new(node.id("kern")) -- additional kern before char
pdfbfn = node.new(node.id("whatsit"),node.subtype("pdf_literal")) -- pdf literal before
cn = node.new(node.id("glyph")) -- char
cn = node.copy(c)
pdfan = node.new(node.id("whatsit"),node.subtype("pdf_literal")) -- pdf literal after
kan = node.new(node.id("kern")) -- additional kern after char

tbl = chartbl[c.char]

kbfn.kern = tex.sp(tbl["kbf"])
pdfbfn.data = tbl["pdfbf"]
cn.xoffset = tex.sp(tbl["xoff"])
cn.yoffset = tex.sp(tbl["yoff"])
pdfan.data = tbl["pdfa"]
kan.kern = tex.sp(tbl["ka"])

kbfn.next = pdfbfn
pdfbfn.next = cn
cn.next = pdfan
pdfan.next = kan
t = node.hpack(kbfn)
return t
end


The values for each of the operations are stored in chartbl:

chartbl = {
[string.byte("y")] = {
["kbf"] = "-0.1em",
["pdfbf"] = "-1 0 0 1 5.5 0 cm",
["xoff"] = "0ex",
["yoff"] = "0.5ex",
["pdfa"] = "-1 0 0 1 -5.5 0 cm",
["ka"] = "0.2em"
}
}


Example:

\directlua{dofile("transform.lua")}
\hsize1cm
\noindent polyethylene


For documentation: In principle, this approach is a general solution as, as far as I understood, everything TeX might want to do with a character can also be done in Lua, with potential changes to the transform_char function. However, for more complex tasks, this seems to be more difficult to do than having TeX typeset it.

So what I initially tried to do was having the post_linebreak_filter call a TeX macro on every character which puts the result of the desired transformation in a \box register and then having Lua replace the node by that box.

I think this is impossible. Any code called by tex.print is only executed after the Lua code, and the approaches to a concurrent interaction discussed in this question are, as it seems to me, not applicable to this situation:

1. Putting the code into a coroutine co and having it call
tex.print("\\directlua{coroutine.resume(co)}")
coroutine.yield()


whenever TeX should execute some macro before continuing works for 14 characters, after which I get
! TeX capacity exceeded, sorry [text input levels=15]

1. In the answer to the question linked above, a \loop is used in the TeX code to repeatedly resume the coroutine, which obviously doesn't work if the Lua code is to be used in a callback.

As a last resort, I only see the possibility of making the characters active, saving the paragraph hlist and temporarily replacing the boxes built by the active character with a glyph node containing the original character in pre_linebreak_filter, and changing them back in the post_linebreak_filter using the saved list. But that's a task for another day ...