# Minimal Testing Environment For LuaMetaTeX

The new LuaMetaTeX typesetting engine praises itself as being "lean and mean", but the ConTeXt package that it comes shipped with is actually pretty obese (around 270 MiB when installed). To get some basic understanding of how it works, I would like to run it with the minimal subset of files required for a kind of "hello world" test, converting into PDF the famous "short story" example from the TeXbook ("Once upon a time, in a distant galaxy called Ööç, there lived a computer named R. J. Drofnats...").

Using pdfTeX, this is quite simple (see manual https://ftp.tu-chemnitz.de/pub/tex/systems/doc/pdftex/manual/pdftex-a.pdf). Create one single folder C:\TeX without any subdirectories and copy the following files into it:

1. Binaries: pdftex.exe, pdftex.dll, kpathsealib.dll (from http://mirrors.ctan.org/systems/win32/w32tex/pdftex-w32.tar.xz).

2. Font metrics and outlines: cm*.tfm and cm*.pfb (from https://ctan.org/tex-archive/fonts/cm/tfm and https://ctan.org/tex-archive/fonts/cm/ps-type1/bakoma/pfb).

3. TeX files: plain.tex, hyphen.tex, story.tex (from https://ctan.org/tex-archive/macros/plain/base).

4. Font map file pdftex.map (created manually):

 cmbx10 CMBX10 <cmbx10.pfb
cmr10 CMR10 <cmr10.pfb
cmsl10 CMSL10 <cmsl10.pfb
[...]

5. Open a command prompt window, change directory and set environment variables to the current folder:

 cd C:\TeX
set t1fonts=.
set texfontmaps=.
set texfonts=.
set texformats=.
set texinputs=.
set texmf=.
set texmfcnf=.
set tfmfonts=.

6. Create memory dump format file plain.fmt:

 pdftex --ini ./plain.tex \dump

7. Typeset document story.pdf:

 pdftex --fmt=plain --output-format=pdf ./story.tex \end


Now do the very same thing with LuaTex (see manual http://www.pragma-ade.com/general/manuals/luatex.pdf):

1. Download and extract the archive http://lmtx.pragma-ade.nl/install-lmtx/context-win64.zip to another folder and run the included install.bat script. Copy luatex.exe and luametatex.exe to the C:\TeX test folder.

2. Create memory dump format file plain.fmt:

 luatex --ini ./plain.tex \dump

3. Typeset document story.pdf:

luatex --fmt=plain --output-format=pdf ./story.tex \end


The problem comes with the new LuaMetaTeX engine (see manual http://www.pragma-ade.com/general/manuals/luametatex.pdf). The command

    luametatex --ini ./plain.tex \dump


gives error messages

    startup error : missing find_log_file callback


and

    startup error : missing open_data_file callback


unless a Lua initialization file lmtx-ini.lua defining callback functions is also loaded, but obviously these ones are not sufficient:

    callback.register ("find_log_file", function () return "plain.log" end)
callback.register ("open_data_file", function () return true end)


because now, the command

    luametatex --lua=lmtx-ini.lua --ini ./plain.tex \dump


yields a really fundamental failure:

    This is LuaMetaTeX, Version 2.09.33  (INITEX)


So how do I typeset my story.pdf using LuaMetaTeX in this minimal environment?

 As @Marcel points out, essential input-output functionality from TeX and a PDF backend must be rebuilt with Lua. To awaken the LuaMetaTeX engine to life somehow, my question modifies to: How to get his extracted LuaMetaLaTeX script collection (https://github.com/zauguin/luametalatex) running in the above standalone folder? This command fails:

    luametatex --lua=./luametaplain-init.lua --ini ./plain.tex \dump

• There are some efforts by Marcel to port the LaTeX format to LuaMetaTeX, but last I heard about it it was plagued by quite a bit of frustration due to poor documentation of internals and the source of the engine not being available: github.com/zauguin/luametalatex Dec 25, 2021 at 18:43
• I just browsed the repository a little bit and saw that Marcel also developed a luametaplain format which is configured together with luametalatex. Dec 25, 2021 at 18:47
• Well, yes, @Henri, I had looked into Marcel's maze of scripts before asking my question, and at least they had helped me formulate these two callbacks to get past the startup errors, though not any further, as in the LuaMetaTeX manual, page 171, Hans just harshly states: "The callbacks are a moving target. Don't bother me with questions about them." Dec 27, 2021 at 5:07
• I find it somewhat mind-boggling that Hans has re-constructed an engine that by itself apparently can neither write to a log nor even input a TeX macro file to dump. Knuth created TeX such that it can produce unaltered results typesetting his manuscripts with plain.tex, even after 50 years. That's as important as passing the trip test. Hans would not be allowed to christen his ship "-Tex" as long as it only sails under the winds of ConTeXt while sinking here. Dec 27, 2021 at 5:08
• When you want to create a PDF, you will need a PDF backend. You either have to write that yourself, use the ConTeXt one or mine. IFAICT the ConTeXt one is not designed to be used outside of a full ConTeXt installation. My LuaMetaLaTeX (and in extension LuaMetaPlain) is designed under the assumption that it runs with a full TeX Live installation, but you might be able to use it in a more minimal setting. Using something significantly smaller would probably be hard. Dec 27, 2021 at 6:17

LuaMetaTeX aggressively removed many parts which can be implemented in Lua, therefore making the engine itself much smaller and shifting responsibilities from the engine to the macro/Lua format. This allows significantly more cutomization without wasting no space with no longer used engine functionality. But this also means that a lot more components are required to actually use the engine. After all, many of the things which are no longer present in the engine itself still have to be provided by someone.

Some examples:

• You have to implement all the callbacks for finding and reading files correctly. E.g. find_log_file has to return a valid path to a log file, open_data_file has to return a table implement the file reader interface. (Of course, if you return true or another invalid value from a callback like open_data_file which is used to open input files then this gets treated as an error, in this case the engine things that the file is missing) This allows the engine itself to longer contain the distribution specific file finding routines like TeX Live's kpathsea.
• Instead of including PDF, DVI, or any other backends, any attempts to shipout the current page are ignored and the loaded macro package has to implement an output driver itself.
• The engine does not include code to read fonts from TFM files, if you want to load such fonts you have to write a TFM file parser.
• Some primitives, including rather important ones like the file i/o support (\openin/\openout/\read/\readline/\write/\ifeof/\closein/\closeout) are no longer included and have to be implemented by the format if needed.
• Hyphenation patterns are no longer part of the dumped format and the Lua and/or TeX code is responsible for loading them if necessary.
• ...

In general, if you want an engine which can be used without much supporting code, then LuaMetaTeX is not the right engine for you. If you are looking for a more or less LuaTeX compatible implementation of most of these independent form the ConTeXt version, you can look at my LuaMetaLaTeX repo (which also includes a luametaplain format which is using the same code for plain TeX instead of LaTeX).

• I understand we have Web sources for DVI backend and TFM parser, and TeX 3.X - which is meant to be frozen - in terms of primitives defines a device-independent interface for handling input and output of text files. You are always free to re-define those primitives, but they must be present if you don't; that's why they're called primitives. Would they really blow up the binary? Dec 27, 2021 at 18:50
• Contemplating outsourcing of so much functionality from the binary: Why not go full Web2Lua and get rid of LuaMetaTeX, backporting PDF libraries to Web sources, which are meant to keep the full interwoven documentation, and generating one single tex.lua script that just needs a Lua interpreter to run, whatever slow, on any system? Dec 27, 2021 at 18:51
• I’m still looking for a direct download link to the single-binary luatex.exe from ConTeXt. A renamed luametatex.exe is already contained in the installer archive. Dec 27, 2021 at 18:51
• LuaTeXhasn't been written in Web for quite some time (and having worked with both TeX's web and LuaTeX's C code, I'm not surprised that they are not willing to go back). Noone is stopping you from porting TeX to Lua, but running a "whatever slow" TeX system doe not seem to be anyones priority. These things were not moved to Lua at random, there was a desire to get more control and sometimes speed in the backend/font loading etc. Dec 27, 2021 at 19:36
• @Gerolf If you want to learn more about the thoughts of the ConTeXt team about LuaMetaTeX (and all kinds of other things), read ConTeXt's evenmore manual, especially section 9: "Is LUAMETATeX still TeX?" Dec 27, 2021 at 19:41