6

This is perhaps a soft question, and I realise it has a chance of being closed, but here we go...

I am currently getting to grips with the internals of Tex/Latex and have found the learning curve to be quite steep. I've been mulling over the mathematical nature of the macro system and wondering how it relates to the actual typesetting and layout of a document.

To premise my question, here's my brief analysis of why Tex/Latex is so... unforgiving. This is not intended to be a rant, please bear with me if it seems that way. I'll blockquote the bit which might seem ranty so you can skim it if this is old news to you.

Tex is what I would call an aggressively imperative language, but it is intended to be used in a mostly declarative fashion. You "markup" your document with sections, captions, etc, in a way which is superficially analogous to something like HTML. However, if you want to write your own macros to tweak anything, you have to deal with a lot of hairy implementation details.

The core logic for parsing tex seems to be "examine the next token, do the thing it says to do (which may arbitrarily change pretty much anything in the execution environment), then consume it and go onto the next token". This seems attractively simple, but the action of any token can have far-reaching effects which are not evident in the immediate scope in which it is used. This stands in stark contrast to the conventional wisdom for other imperative languages such as C++ where good design is things like "don't use global variables, variables should be const wherever possible, use RAII to establish invariants, and make functions idempotent where possible". Such practices make functions easier to understand by making them easily composable.

Tex/Latex functions are not easily composable. Spooky-action-at-a-distance and unrestrained interference between macros seems to be idiomatic. It resembles a raw Turing machine more than almost every other language I have seen. It is certainly the most fragile language I have ever used.

It strikes me that this situation is heightened by the fact that, although it is normal to write and use macros that have highly non-local effects, paradoxically, the core language itself does not empower itself to easily inspect or control its environment beyond that which is immediately local. Hence we get tangled messes involving \expandafter, multiple ways of defining macros, nothing I would call a type system, and so on.

So that's what Tex looks like to me. So what -- shrug, right? It's just how the language is.

The thing is, nothing I've seen about the design of macro system so far seems to have anything to do with typesetting and layout.

I have no idea what the typesetting and layout actually is or how it works. Here is my question: does the typesetting and layout algorithm actually require the kind of code transformations afforded by the macro language, or could the same typesetting and layout process be easily implemented by something radically different?

I say "easily implemented" because, of course, Turing completeness means it is of course possible to get the same results in any other language. That's not the point. Some languages make it easy to work with matrices, for example, some languages make it easy to work with graphs, some with strings, some with predicates, etc etc. One way of looking at my question is: does the Tex macro system actually have an intimate connection with the mathematics and logic of typesetting and layout?

|improve this question
  • 2
    We've had over time several questions in a similar area, to which there is perhaps not a simple answer. If one were starting with a blank slate, once perhaps would use a radically different approach to TeX, but one isn't starting from such a position and thus any competitor needs to deal with not only the technical challenges but also the fact that TeX exists and works 'well enough' for many users. At the same time, there is the issue that perhaps 99% of the problems can be solved with a variety of approaches, but it's the of 1% 'awkward cases' that prove to be an issue ... – Joseph Wright♦ Mar 11 '17 at 12:18
  • You could look to luatex where the entire typesetting is (in principle) accessible from lua and doesn't need tex macros at all. just needs a couple of lines of tex syntax to bootstrap. Of course since only two or three people use it that way you don't have thousands of off-the-shelf packages from ctan but... – David Carlisle Mar 11 '17 at 12:19
  • 3
    Perhaps also worth noting is that Knuth wrote TeX for a particular purpose, and in plain TeX the amount of programming typically carried out is small. LaTeX is different but this is in part as most users don't seem to want to get involved in the detail. – Joseph Wright♦ Mar 11 '17 at 12:22
  • If you're looking for a better understanding of TeX as a typesetting system, I strongly recommend these two: (1) Knuth & Plass, Breaking Paragraphs into Lines (the latest version is in the 2012 printing of Digital Typography but the original paper is ok too). (2) Seroul & Levy, A Beginner's Book of TeX — this focuses on actually using (plain) TeX for typesetting, rather than for writing macros. At the end of the book you'll be able to typeset most things, and your mental model will match that of the TeX program. – ShreevatsaR Mar 23 '17 at 15:59
7

the interface to using luatex without using any tex macros is discussed in several places, notably

http://wiki.luatex.org/index.php?title=TeX_without_TeX

However your question is a little vague. In that you ask if the macro system has an intimate relation to the layout. Different parts of a document layout are implemented by different things.

Math for example is handled by the core engine code, so in a basic \[...\] there need not be any complicated macros, however alignment and number placement logic used in align is largely implemented using tex macros (with some support for \halign.

Similarly the latex figure placement algorithm is entirely implemented in the tex macro layer. Of course it doesn't have to be, and on the at tug2016 Frank gave a talk discussing a different float placement algorithm implemented in Lua. However the current float placement is entirely a macro layer construct.

The paragraph line breaking on the other hand is totally not controllable from TeX macros and in fact the TeX line breaking algorithm has been used by now in several other systems (including a javascript version for web pages).

|improve this answer
6

Maybe a look into the history of TeX can help to answer the question.

As Knuth once stated (Digital Typography, p. 648):

In some sense I put in many of TeX's programming features only after kicking and screaming

So one can conclude that most programming features are not needed for the typesetting or the layout algorithm in Knuth's original design. Knuth continues:

I know how Leslie [Lamport] went about writing LaTeX---first he would write the algorithm out in a high-level programming language ... then he would pretty much mechanically convert the high-level code into TeX macros. If I had suspected that such a style was going to be the most common use of TeX, I probably would have worried a lot about efficiency these days.

(See also an early article of Lamport in TUGboat.)

The initial thoughts of Knuth about the required macro system can be found in Chapters 24 and 25 of Digital Typography (or look up the entry "macros in \TeX, history of" in its index). For example, he wrote:

I have had some notion of adding an interpretive system to TEX and a mini-programming language so that such extensions are easier to make without penetrating into TEX's innards, but on second thought it seems much better to assume that TEX's code will be sufficiently clean that it will be best extended using SAIL code.

If the macro system consists of the control sequences that are placed in the macro family by David Bausum then except for \begingroup/\endgroup, \def/\edef/\gdef, \global/\long/\outer, and \relax all other commands are added after the initial design. The file errorlog.tex (see CTAN) shows the dates when the features are coded:

  1. \afterassignment (added 27.05.1983),
  2. \aftergroup (added 16.07.1983),
  3. \csname/\endcsname (added 13.11.1982),
  4. \expandafter (added 12.09.1982),
  5. \futurelet (added 02.12.1982),
  6. \globaldefs (added 20.01.1983),
  7. \let (added 26.03.1980; so this was part of the original TeX82),
  8. \noexpand (25.05.1983),
  9. \the (extended 12.07.1983), and
  10. \xdef (added 28.11.1978; again part of TeX82).

Several if tests of the logic family have been added late in the process too, for example,

  1. \ifeof (added 12.09.1982),
  2. \iffalse (added 03.02.1983),
  3. \ifhbox (added 27.08.1983),
  4. \iftrue (added 03.02.1983),
  5. \ifvbox (added 27.08.1983), and
  6. \ifx (added 13.07.1981; known in TeX82 from the beginning).

And there are a lot of other things that are not available in the design of TeX82, for example, token registers. But even without these commands the number of TeX primitives, i.e., commands implemented by the TeX program, is around 300, so that the task of typesetting is a challenge compared to a programming language like C, for example. This large set of primitive commands was necessary to make TeX a flexible typesetting language. TeX's typesetting model box-glue-penalty (described in Chapter 3 of Digital Typography, i.e., the paper by Knuth and Plass mentioned in the comments after the question) need at most a few aspects of the macro system to typeset text: definitions to combine the primitives to high-level commands and a few tests, for example, catcodes, modes, etc. to make TeX a usable typesetting language.

The line breaking algorithm was implemented in other systems (non-TeX based) too (see p. 105, left column, of Interview: Donald E. Knuth). So the macro system of TeX is not needed for this procedure. But you might need indents, justification, interword spaces, ligatures, hyphenation, ...

Please don't expect to understand this mess now, TEX is really very simple; trust me.
--- Donald E. Knuth, Digital Typography, p.512

|improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.