4

In TeX there are a lot of parameters which you can adjust for changing the behavior of TeX.

Example 1:

By assigning \baselineskip=... you change distance between lines.
Why TeX's breaking-paragraphs-into-lines-machinery does not break when you redefine the control sequence \baselineskip to do something totally different?
For example why does TeX not print error message but still create lines with a distance of 1.5cm with following example when \baselineskip is redefined not to denote glue at all?

\begingroup
\baselineskip=1.5cm\relax
\def\baselineskip{Hello, world!\hfil\break}
\noindent
\baselineskip
\baselineskip
\baselineskip
\par
\endgroup

\bye

enter image description here

Example 2:

By assigning \escapechar=... you change what chararacter is in front of the name of a control sequence when for example \string is applied to it.
Why TeX's \string primitive is not disturbed when you redefine the control sequence \escapechar to do something totally different?
For example why does TeX not print error message but still put / in front of name of control sequence when \escapechar is defined not to deliver internal encoding number of character at all?

\escapechar=`\/
\def\escapechar{Hello, world!\hfil\break}
\noindent
{\tt\string\escapechar} expands to: \escapechar
\par

\bye

enter image description here

How can TeX still access the last values of such parameters when control sequences which formerly served for accessing them are redefined?

Are these control sequences only "entrances" for users for accessing something that by TeX is accessed without these "entrances"?

Are there (besides \pdfprimitive) "backdoors" for accessing things when all control sequences which formerly served for accessing them are redefined?

  • 1
    Yes: These control sequences are "entrances" for users for accessing something that by TeX is accessed without these "entrances". ;-) By the way: You can use \let for creating more "entrances" for accessing the same thing. E.g.,you can do \let\escapecharcopy=\escapechar and then use \escapecharcopy=... for assigning values to the same parameter of TeX that usually can have values assigned via \escapechar=.... – Ulrich Diez Jun 14 '19 at 18:24
2

Redefining \baselineskip does not change the value stored in the internal skip register. The same for \escapechar, that points to an internal numeric register.

You're only giving up the possibility to change those values.

When TeX splits paragraphs into lines, it needs to know the current value stored in the register which \baselineskip points to, but it doesn't perform \the\baselineskip, because the paragraphing happens at a much deeper level than the one where tokens live (frequently called TeX's stomach).

The internal instruction is “look at what's stored in memory location baselineskip and use it for the needed calculations” (that's just a fictional name). When you do

\baselineskip=12pt

the instruction that's performed in TeX's stomach is “store in memory location baselineskip the number 786432 (the given value in scaled points).

With Knuth's TeX (no extensions by third parties) once you have redefined a primitive you're on your own and there's no method for recovering it. Of course, locally redefining the primitive in a group will resurrect the previous meaning at group end. With pdfTeX/epTeX one can use \pdfprimitive; with LuaTeX/XeTeX it's called \primitive.

If, in all engines, you “save” the primitive with another name before redefining, say

\let\primitivebaselineskip\baselineskip
\def\baselineskip{whatever}

...

\let\baselineskip\primitivebaselineskip

the original meaning will be restored. A control sequence points to a well defined memory location and a \let instruction makes the “new” control sequence point to the same place as the “old” one.

| improve this answer | |
2

Yes exactly: As you deduced, the control sequences provided as primitives are merely “entrances” for the user.

To elaborate,

  • [internal parameters] TeX internally stores a table of all these parameters. For example, (say) table entry 2883 always contains the glue used as the baseline skip, table entry 5308 always contains the escape character (as an integer), etc. During the operation of TeX, whenever these values need to be used (e.g. the escape character is needed to print a control sequence to the user, as in \string or error messages), the corresponding table entry is looked up.

  • [control sequences] Separately, TeX also stores a table (a different part of the same table actually) mapping control sequences to their equivalents. Both primitive control sequences and user-defined ones are treated identically; the only difference is that primitives are initialized by the program itself during startup (initex), and so their equivalents can happen to be of types (e.g. “show or modify baseline skip”) that the user cannot set. The only thing that happens when you redefine a control sequence (say \baselineskip or \escapechar) “away” from the primitive is that you're changing this mapping; TeX itself doesn't care about the meaning of the control sequence except when it encounters it as input.

Apart from \pdfprimitive or \primitive added by pdfTeX etc., another way to get at these internal parameters is with LuaTeX which provides Lua interfaces (like tex.baselineskip, tex.escapechar) for most of them directly, that do not care about the control sequences:

print()
function showVars()
    width, stretch, shrink, stretch_order, shrink_order = node.getglue(tex.baselineskip)
    mychar = tex.escapechar
    print(string.format('baseline skip: %s plus %s minus %s etc', width, stretch, shrink))
    print(string.format('escape char: %d = %c', mychar, mychar))

    cs_baselineskip = token.get_macro('baselineskip') or 'not a macro'
    cs_escapechar = token.get_macro('escapechar') or 'not a macro'
    print(string.format([[\baselineskip: %s]], cs_baselineskip))
    print(string.format([[\escapechar: %s]], cs_escapechar))
    print()
end
\directlua{dofile('showvars.lua')}
\directlua{showVars()}

\baselineskip=1pt
\escapechar=`/
\directlua{showVars()}

\def\baselineskip{boo}
\def\escapechar{whatever}
\directlua{showVars()}

\bye

Result:

baseline skip: 786432 plus 0 minus 0 etc
escape char: 92 = \
\baselineskip: not a macro
\escapechar: not a macro

baseline skip: 65536 plus 0 minus 0 etc
escape char: 47 = /
\baselineskip: not a macro
\escapechar: not a macro

baseline skip: 65536 plus 0 minus 0 etc
escape char: 47 = /
\baselineskip: boo
\escapechar: whatever

(If this doesn't answer your question and you're still curious about why TeX does not break and how TeX achieves this internally, the low-level details are in another answer.)

| improve this answer | |
2

(This is a low-level sequel to another answer. Posting this as a separate answer because it would make the other one very long, and this is probably more detail than anyone wants to know. But if your “why” and “how” are not answered by the high-level description and you're further interested in all the gory details of the implementation…)

TeX is a program. Internally, this program maintains a table called the “table of equivalents” — a large global variable of array type, called eqtb, defined in section 253 of the program:

program TEX; { ... }
var
  { ... }
  eqtb: array [1 .. eqtb_size] of memory_word; {§253}
  { ... }

where memory_word is basically a 4-byte blob of memory, with interpretation as given below.

As documented in section 220, this table of equivalents stores:

  • the equivalents for control sequences (in regions 1 and 2),
  • the values of various glue parameters (region 3),
  • the values of various other integer parameters (regions 4 and 5), and
  • the values of various dimension (length) parameters (region 6).

Each of these equivalents is stored (§221) as a level (used for saving and restoring inside groups), a eq_type (a command code, §207–210), and the actual equiv, an integer whose interpretation depends on the type (a table index, a pointer to a token list…).

Whenever the program needs to access the value of one of its internal parameters, it uses the corresponding entry from the table. For example, suppose the baseline skip is stored in table entry 2883. When appending to a vertical list the program may need the baseline skip (§679), in which case it uses (essentially) eqtb[2883].equiv.

Additionally, the program provides some “primitives” like \baselineskip or \escapechar for setting some of these table entries: these simply map a control sequence name to a command that sets that table entry. More precisely, in the equivalent for that name, the eq_type is set to be a command code like assign_int or assign_glue, with equiv holding which table entry is to be assigned. For example, if you don't redefine the primitive, the equivalent for the control sequence baselineskip (stored in region 2) would have an eq_type of assign_glue, and an equiv of 2883.

When the program encounters that corresponding control sequence in the input (as a command by itself), it follows the equivalent and does the corresponding assignment.

Here is a miniature version of the TeX program, with only the relevant parts, and some with some liberties taken for simplicity:

program TEX; { §4 ... }
var {§13}
    { ... }
    cur_cmd: int; {§297}
    cur_chr: int;
    eqtb: array [1 .. eqtb_size] of memory_word; {§253}
{ ... }

procedure primitive(s: string, eq_type: int, equiv: int); {§264}
begin
   { ... set eqtb[hash(s)] to (level 1 and) the eq_type and equiv passed here ...}
end

procedure init_prim; {§1336}
begin {...}
    primitive("baselineskip", assign_glue, 2883); {§226}
    {...}
    primitive("escapechar", assign_int, 5308); {§238}
    {...}
end

procedure prefixed_command; {§1211}
begin {...}
    case cur_cmd of
        set_font: { ... §1217 ...}
        def: {... §1218 ...}
        { ... }
        assign_int: {§1228}
            begin 
                p ← cur_chr; {e.g., 5308 in case of the primitive \escapechar}
                scan_optional_equals;
                scan_int;
                word_define(p, cur_val); {...sets eqtb[p], accounting for level...}
            end;
        assign_glue: {...similar...}
        {...}
    endcases;
end; 

procedure main_control; {§1030}
begin
    while true do
    begin
        {... read a token, i.e. set cur_cmd and cur_chr ...}
        case cur_cmd of
            {...}
            assign_int: prefixed_command; {§1210}
            assign_glue: prefixed_command;
            {...}
        endcases;
    end;
end;

{...}
begin {§1332}
    {...}
    init_prim;
    {...}
    main_control;
    {... cleanup ... }
end.
| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.