38

A new version of TeX has been released by Donald Knuth earlier this year (January/February 2021), and its version number is 3.141592653 now. What has changed since the previous release, version 3.14159265, from seven years ago (January 2014)?

For those who don't know, Knuth writes on his website that:

I still take full responsibility for the master sources of TeX, METAFONT, and Computer Modern. Therefore I periodically take a few days off from my current projects and look at all of the accumulated bug reports. This happened most recently in 1992, 1993, 1995, 1998, 2002, 2007, 2013, and 2020; following this pattern, I intend to check on purported bugs again at the end of years 2028, 2037, 2047, etc. The intervals between such maintenance periods are increasing, because the systems have been converging to an error-free state.

and, accordingly, that:

This is the year when I promised to do the seven-year cleanup of TeX and METAFONT (last updated at the beginning of 2014). It took me three weeks to wade through several hundred issues of many different kinds, and five actual bugs were discovered(!). But I'm happy to report that the newly-tuned-up systems don't “break” any of the things that used to work, and the bugs weren't likely to bite. Everyone can therefore upgrade or not, at their convenience.

So what are the "five actual bugs"? Under what (rare) circumstances do they matter?


(Some notes on this question itself:

  • With the previous two previous updates Knuth has published a summary of the changes in TUGboat, namely The TeX tuneup of 2014 and The TeX tuneup of 2008. I expect there will similarly be an article "The TeX tuneup of 2021", but last time, the question and answer were posted on this website even before that article was published…)
8
  • 2
    Not urgent to answer this question of course; I just noticed the update and jumped the gun a bit I guess. :-) – ShreevatsaR Jan 30 at 8:46
  • 28
    Would I be heavily downvoted if I posted an answer of " 0.000000003" ? – David Carlisle Jan 30 at 9:15
  • 4
    Well, in 2047 he will be 109 years old. Not impossible, but ... – Pieter van Oostrum Feb 2 at 20:10
  • 1
    Overall, the changes are those you can see in the commit you linked, but they are not final (there were updates, for example this just came in today, and there may be more), so the final changeset should be made official in a few days – Phelype Oleinik Feb 6 at 2:04
  • 2
    Here is a draft of the 2021 tuneup report by Knuth: tug.org/TUGboat/tb42-1/tb130knuth-tuneup21.pdf – texdr.aft Feb 7 at 23:25
35
+100

The total changes made by Knuth for the 2021 tuneup were huge (there were several people scrutinizing his work this time around). There were several small changes from typo corrections in older errata to rewording of the copying conditions in tex.web and even some changes to the plain format itself. Here are the entries added to errorlog.tex and a small1 explanation to each. (The “five bugs” Knuth mentions are I948, S949, S950, B952, and R953. There are other two that were not added to errorlog.tex; they are at the bottom of this answer.) A shorter version of this summary is also now available at the TUG website.

* 15 January 2021
I948. Don't pause on errors when tracing paragraphs (Udo Wermuth). @826
S949. Don't try to interact when in |\batchmode| (Xiaosa Zhang). @83
S950. Don't try to edit when no file is active (Xiaosa Zhang). @84
R951. Take date and time sometimes from system, not user (Udo Wermuth). @241,536
B952. Don't allow implicit left brace after |#| (Udo Wermuth). @476
R953. After nine parameters, must delete offending tokens (Bruno Le Floch). @476
D954. Garbage visible in buffer after file ends prematurely (DRF). @486
R955. Force nonexistent characters to have null specs (DRF). @722
C956. Don't mark fraction noads as temporarily Inner (DRF). @761
Q957. Reset |\newlinechar| before logging the stats (Udo Wermuth). @1333,1335

[1]: I wrote the “small explanation” in the intro before actually writing the explanations, so I didn't lie... technically :)


I948. Don't pause on errors when tracing paragraphs (Udo Wermuth). @826

This bug would cause TeX to apparently hang when \tracingparagraphs was on (> 0), and the Infinite glue shrinkage error occurred while the paragraph trace info was being printed. When \tracingparagraphs, is on, TeX is writing to log_only (unless \tracingonline=1), and in case of that error it would prompt the user for interaction, but since the write selector was redirecting the output to the the .log, the user would not see that, and TeX would be apparently stuck.

For example, if you run on a file that contains this line:

\tracingparagraphs=1 Press\hss return.\end

the terminal will show

This is TeX, Version 3.14159265 (TeX Live 2020) (preloaded format=tex)
 restricted \write18 enabled.
entering extended mode
(./test.tex

and hang there, waiting for user interaction (for example, typing x then <RETURN> to end the run, or just <RETURN> to ignore the offending \hss). This will not work if you run

$ tex '\tracingparagraphs=1 Press\hss return.\end'

because in that case there is no .log open yet, so the output will be to the terminal.

After the tuneup, TeX will treat this error as it treats any other error: if in \errorstopmode it ask the user for interaction, otherwise it will scroll past it:

This is TeX, Version 3.141592653 (TeX Live 2021/dev) (preloaded format=tex)
(./test.tex
! Infinite glue shrinkage found in a paragraph.
<inserted text> \par 
                     
<to be read again> 
                   \end 
l.1 \tracingparagraphs=1 Press\hss return.\end
                                              
? 

The relevant change entry is:

429. Don't echo error message to terminal when tracing paragraphs
(Udo Wermuth, 15 January 2017)
@x module 826
  begin no_shrink_error_yet:=false;
@y
  begin no_shrink_error_yet:=false;
  @!stat if tracing_paragraphs>0 then end_diagnostic(true);@+tats@;
@z
@x
  error;
@y
  error;
  @!stat if tracing_paragraphs>0 then begin_diagnostic;@+tats@;
@z

S949. Don't try to interact when in |\batchmode| (Xiaosa Zhang). @83

This bug, initially reported here, was causing TeX to ask for user interaction (TeX's ? prompt) while in \batchmode, thus trying to write to a closed \write stream, and this would cause a segmentation fault. From Karl's answer to the original report, you can reproduce that error in TeX up to 2020 by running tex -ini then typing these lines, ending them with <RETURN>:

\catcode`\^=7 \catcode`\^^?=15 \s^^?E
1
q
v

With the new TeX, after typing q to enter \batchmode, TeX won't try to ask for user interaction if it can't, so that won't break anymore.

The relevant change entry is:

430. Defeat interactions during batch mode (Xiaosa Zhang, 27 June 2020)
@x module 83
@ @<Get user's advice...@>=
loop@+begin continue: clear_for_error_prompt; prompt_input("? ");
@y
@ @<Get user's advice...@>=
loop@+begin continue: if interaction<>error_stop_mode then return;
  clear_for_error_prompt; prompt_input("? ");
@z

S950. Don't try to edit when no file is active (Xiaosa Zhang). @84

This bug, initially reported here was triggered when you tried to open the editor (using TeX's E option) when an error happened in an input given interactively. Suppose you have a file called h.tex with a single line (supposing that \ERROR is undefined, or is anything that would cause an error):

% h.tex
\ERROR

then, when TeX complains about ! Undefined control sequence \ERROR you reply:

I\MISTAKE V

which will insert \MISTAKE V for TeX to process, and it will once again complain, due to the undefined \MISTAKE, and now you reply:

E

and TeX will segfault.

Here's the transcript of the interactive session:

$ tex h
This is TeX, Version 3.14159265 (TeX Live 2020) (preloaded format=tex)
(./h.tex
! Undefined control sequence.
l.1 \ERROR
          
? I\MISTAKE V
! Undefined control sequence.
<insert>   \MISTAKE
                    V
l.1 \ERROR
          
? E
No pages of output.
Transcript written on h.log.
Segmentation fault (core dumped)

This error would happen because TeX would try to tell you the name of the input file in which the error occurred, but since the error was on an interactively input command, there is no associated file. After the tuneup TeX knows that in that case it is not reading from a file, so it won't try to give you a file name.

The relevant change entry is:

431. Don't exit to editor if no input file is at the bottom line
(Xiaosa Zhang, 03 July 2020)
@x module 84
"E": if base_ptr>0 then
@y
"E": if base_ptr>0 then if input_stack[base_ptr].name_field>=256 then
@z
@x module 85
if base_ptr>0 then print("E to edit your file,");
@y
if base_ptr>0 then if input_stack[base_ptr].name_field>=256 then
  print("E to edit your file,");
@z

R951. Take date and time sometimes from system, not user (Udo Wermuth). @241,536

Before setting \jobname (more precisely before starting the .log file) you could change the value of \year, \month, \day and \time, and that would be written in the header line of the .log. If you did

$ tex '\day=99 \end'

the first line of the .log would say something like

This is TeX, Version 3.14159265 (TeX Live 2020) (INITEX)  99 FEB 2021 22:18

(note the Feb 99th :) or, if you were feeling really devious, you could print any three bytes from TeX's executable by setting a bogus value of \month, like (with the build I have here)

$ tex -ini "\month=-54 \end"

to get a month called TeX:

This is TeX, Version 3.14159265 (TeX Live 2020) (INITEX)  2 TeX 2021 22:20

or with an extreme enough value you could make TeX crash with a segmentation fault, for example:

$ tex '\month=-100000 \end'

With the new version, the value printed in the header is an internal sys_(time|day|month|year), that can't be changed by changing the primitive registers.

This is a rather long (in number of lines) change, and not so interesting as reading material here (basically declare new variables sys_<thing>, initialise the primitives to those, and use sys_<thing> instead of <thing> to print the banner), so I will omit the change entry, but you can find by searching for its header

432. Keep date and time in system variables, use them in opening banner
(Udo Wermuth, 11 December 2020)

in tex82.bug.


B952. Don't allow implicit left brace after |#| (Udo Wermuth). @476

This bug (which I was surprised it wasn't found before) allowed you, when the last token of the <parameter text> of a definition was #6, to use an implicit begin-group character (like \bgroup) in place of the explicit begin-group character that marks the end of the <parameter text> of a definition, such that

\def\foo#1#\bgroup(#1)}
\show\foo

was valid, and would show

> \foo=macro:
#1\bgroup ->(#1)\bgroup .

on the terminal, meaning that the parameter #1 of \foo was delimited by \bgroup, and that \bgroup would be reinserted after the <replacement text> of the macro, exactly how TeX does with an explicit begin-group character. After the tuneup, you will get an error from the definition above:

! Parameters must be numbered consecutively.
<to be read again> 
                   \bgroup 
l.1 \def\foo#1#\bgroup
                      (#1)}
?

and further errors that will followed due to the malformed definition (the macro that will be defined with the input above will be, after some errors, \foo=macro:#1#2\bgroup (#31)->.).

The relevant change entry is:

434. Don't accept an implicit left brace after # in macro head
(Udo Wermuth, 20 May 2020)
@x module 476
if cur_cmd=left_brace then
@y
if cur_tok<left_brace_limit then
@z

R953. After nine parameters, must delete offending tokens (Bruno Le Floch). @476

With this bug you could have TeX do some real funny things. When scanning the <parameter text> of a macro, after the nine allowed parameters, any # will raise an error, but the token following that # would be left in the <parameter text>. Suppose you had a macro with 9 parameters, and tried to add a tenth parameter #0:

\def\foo#1#2#3#4#5#6#7#8#9#0{}
\show\foo

TeX would complain to you that

! You already have nine parameters.
l.1 \def\foo#1#2#3#4#5#6#7#8#9#0
                                {}
? h
I'm going to ignore the # sign you just used.

? 

and the macro definition would have the # ignored, but the 0 would remain there:

> \foo=macro:
#1#2#3#4#5#6#7#8#90->.
l.2 \show\foo
             
? 

So far, nothing exciting. But now suppose a day you were feeling extra naughty and used ## instead of #0, like \def\foo#1#2#3#4#5#6#7#8#9##{}, then \show\foo would say

> \foo=macro:
#1#2#3#4#5#6#7#8#9##->.
l.2 \show\foo
             
? 

and would you look at that! #9 is now delimited by a parameter token, so if you called \foo 12345678hello#, #9 would be hello!

Even worse, you could trick TeX's scanner into grabbing a } as the argument of a macro without errors (after the two You already have nine parameters errors, of course). This example from the original bug report shows that:

\def\foo#1#2#3#4#5#6#7#8#9#}##{\show#9}
\show\foo
\foo12345678} }#
\end %        ^^ delimiter

In the example you have a macro delimited by }# (the tokens left after TeX removed the two extra #), and as part of the scanning, the first } would not go through the Argument of \foo has an extra } error, so it would be added to the current parameter, then the \show#9 will say:

> end-group character }.
<argument> }
             
\foo #1#2#3#4#5#6#7#8#9}##->\show #9
                                    
l.39 \foo12345678} }#
                     
? 

After the 2021 tuneup, TeX now understands that you meant for #0 to be a parameter, so naturally the 0 should be removed as well, so now the error message says a little more:

! You already have nine parameters.
l.1 \def\foo#1#2#3#4#5#6#7#8#9(#0
                                 ){}
? h
I'm going to ignore the # sign you just used,
as well as the token that followed it.

?

and the definition will contain no trace of your tenth parameter:

> \foo=macro:
#1#2#3#4#5#6#7#8#9()->.
l.3 \show\foo
             
?

The relevant change entry is:

433. After nine parameters, delete both # and the token that follows
(Bruno Le Floch, 22 October 2020)
@x module 473
label found,done,done1,done2;
@y
label found,continue,done,done1,done2;
@z
@x module 474
begin loop begin get_token; {set |cur_cmd|, |cur_chr|, |cur_tok|}
@y
begin loop begin continue: get_token; {set |cur_cmd|, |cur_chr|, |cur_tok|}
@z
@x module 476
  help1("I'm going to ignore the # sign you just used."); error;
@y
  help2("I'm going to ignore the # sign you just used,")@/
    ("as well as the token that followed it."); error; goto continue;
@z

D954. Garbage visible in buffer after file ends prematurely (DRF). @486

With this bug, the error message File ended within \read could be followed by garbage context, if the circumstances were right. Before the tuneup, if you were \reading from a file with one { too many, you could see the error message. Suppose a file unbal.tex with the single line:

{

and then you run the following document:

\catcode`{=1 \catcode`}=2 \catcode`#=6
\openin1 unbal
\def\A#1#2#3#4#5#6#7#8#9{\read1to \x}
\def\B#1#2#3#4#5#6#7#8#9{\A#1#2#3#4#5#6#7#8#9 \relax}
\def\C#1#2#3#4#5#6#7#8#9{\B#1#2#3#4#5#6#7#8#9 \relax}
\def\D#1#2#3#4#5#6#7#8#9{\C#1#2#3#4#5#6#7#8#9 \relax}
\def\E#1#2#3#4#5#6#7#8#9{\D#1#2#3#4#5#6#7#8#9 \relax}
\E123456789 \end

the error message would start with

Runaway definition?
->{ 
! File ended within \read.
<read 1> {^^M7#8#9{\D

where ^^M7#8#9{\D are the leftovers in the buffer variable.

After the tuneup, TeX now cleans up the buffer and the error context is correct:

Runaway definition?
->{ 
! File ended within \read.
<read 1> 

The relevant change entry is:

435. Keep garbage out of the buffer if a |\read| end unexpectedly
(DRF, 17 February 2018)
@x module 486
    align_state:=1000000; error;
@y
    align_state:=1000000; limit:=0; error;
@z

R955. Force nonexistent characters to have null specs (DRF). @722

This one didn't have a visible effect on normal usage of TeX, so no compilable example for this one (mostly because I failed to produce a bad .tfm file to trigger this bug).

In a .tfm file, a non-existent character is marked by its width index being zero, and TeX assumes that if that is true, all other metrics of said character are zero as well, but nothing was enforced.

If that weren't the case, though, when reading a character from a font, TeX would only look at its width, and assume everything else is zero, without enforcing. But if a .tfm was made so that the width was zero, but for example the italic correction were not, that index would not be zeroed and the wrong italic correction would be used.

After the tuneup, if the width of a character is zero, TeX will nullify the entire character to make sure. The relevant change entry is:

436. Zero out nonexistent chars, to prevent rogue TFM files
(DRF, 06 October 2020)
@x module 722
    math_type(a):=empty;
@y
    math_type(a):=empty; cur_i:=null_character;
@z

C956. Don't mark fraction noads as temporarily Inner (DRF). @761

This bug had a fix in tex.web but it was more of a bug in The TeXbook. In short, some places in The TeXbook, for example the last paragraph on page 155 used to say:

There’s also an eighth classification, \mathinner, which is not normally used for individual symbols; fractions and \left...\right constructions are treated as “inner” subformulas [...]

but now fractions were removed from that statement. The relevant change entry in tex.web is:

437. Don't classify fraction noads as inner noads (DRF, 25 March 2019)
@x module 761
fraction_noad: begin t:=inner_noad; s:=fraction_noad_size;
  end;
@y
fraction_noad: s:=fraction_noad_size;
@z

which now doesn't make a fraction an Inner atom any longer. Though that won't have any change in math typesetting because a fraction was usually written as {1\over2}, and the extra braces to enclose the subformula would make that fraction an Ord atom for all purposes.

The only way to get the fraction as an actual Inner atom was if either the formula was only a fraction, like $1\over2$, in which case it wouldn't make a difference, because of the math boundaries, or if the fraction was enclosed in a \left...\right pair, but then it would become an Inner atom anyway because of \left...\right. All other uses of a fraction would result in an Ord atom due to the braces that delimit the subformula, so this classification was dropped altogether to avoid confusion.

Proof of that is can be seen from this example:

\nopagenumbers \loggingall \tracingonline=1
Punct: ${1\over2}.$\par
Ord: ${1\over2}x$
\end

If the fractions were actually an Inner atom, according to the math spacing table on page 170 of The TeXbook, you should have a \thinmuskip after the fractions in both cases (followed by a Punct and followerd by an ord), but if you look at the produced lists, you see that neither have the space:

.......\sevenrm 2
.....\hbox(0.0+0.0)x1.2, shifted -2.5
...\teni :

and

.......\sevenrm 2
.....\hbox(0.0+0.0)x1.2, shifted -2.5
...\teni x

Q957. Reset |\newlinechar| before logging the stats (Udo Wermuth). @1333,1335

Udo Wermuth reported that you could get some weird terminal output from TeX depending on how you set the \newlinechar parameter. For example running

$ tex '\newlinechar=32 \end'

would print

$ tex '\newlinechar=32 \end'
This is TeX, Version 3.14159265 (TeX Live 2020) (preloaded format=tex)
No
pages
of
output.
Transcript
written
on
texput.log.

because it would make TeX use the space character (ASCII 32) as a newline character when writing to a file, so all spaces would be converted. Equally interesting outputs could be achieved by ussing different ASCII codes. This is now corrected, and the command above will generate the much boring

$ tex '\newlinechar=32 \end'
This is TeX, Version 3.141592653 (TeX Live 2021/dev) (INITEX)
No pages of output.
Transcript written on texput.log.

The relevant change entry is:

440. Normalize newlinechar when printing the final stats
(Udo Wermuth, 29 November 2020)
@x module 1333
begin @<Finish the extensions@>;
@y
begin @<Finish the extensions@>; new_line_char:=-1;
@z
@x module 1335
begin c:=cur_chr;
@y
begin c:=cur_chr; if c<>1 then new_line_char:=-1;
@z

Missing (\tabskip) indication in Underfull message

Another bug, found by Igor Liferenko was a missing \tabskip glue indication in the Underfull box message of an alignment, that was otherwise documented in The TeXbook. In this example:

% \catcode`\{=1 \catcode`\}=2 \catcode`\&=4 \catcode`\#=6
\showboxdepth=1 \tracingonline=1
\tabskip=0pt plus10pt \halign to200pt{&#\hfil\cr
  \hbox to50pt{}&\hbox to60pt{}\cr}
\end

the terminal would show

\hbox(0.0+0.0)x200.0, glue set 3.0
.\glue(\tabskip) 0.0 plus 10.0
.\unsetbox(0.0+0.0)x50.0
.\glue(\tabskip) 0.0 plus 10.0
.\unsetbox(0.0+0.0)x60.0
.\glue 0.0 plus 10.0

whereas the last line should be

.\glue(\tabskip) 0.0 plus 10.0

Now the final (\tabskip) indication now shows properly, as documented.

The relevant change entry is:

438. Properly identify tabskip glue when tracing repeated templates
(Igor Liferenko, 10 January 2020)
@x module 793
link(p):=new_glue(glue_ptr(cur_loop));
@y
link(p):=new_glue(glue_ptr(cur_loop));
subtype(link(p)):=tab_skip_code+1;
@z

Internal variable overflow in \hyphenation

This is probably the least exciting bug found, mainly because with your everyday TeX you can't spot it. It was found by David Fuchs with a special version of TeX that he crafted to check memory boundary violations.

When declaring a \hyphenation, the variable hn, declared as a small_number (within the range 0..63) would be maxed out, but then in module §930, <Look for the word |hc[1..hn]|...>, that looks for a given word in TeX's exception table, would call incr(hn), making it larger than the declared size. This document does that:

\lefthyphenmin=0
\righthyphenmin=0
\hyphenation{-a-b-c-d-e-f-g-h-i-j-k-l-m-n-o-p-q-r-s-t-u-v-w-x-y-z-a-b-%
  c-d-e-f-g-h-i-j-k-l-m-n-o-p-q-r-s-t-u-v-w-x-y-z-a-b-c-d-e-f-g-h-i-j-%
  k-l-m-n-o-p-q-r-s-t-u-v-w-x-y-z}
\showhyphens{abcdefghijklmnopqrstuvwxyzabcdefg%
  hijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz}
\end

but it's not noticeable when running TeX because the variable overflow doesn't appear. What exactly happens is a bit implementation dependent, as it relies on what the compiler translates a small_number to. If it becomes a variable that holds more than 0..63, nothing bad will happen.

The relevant change entry is:

439. Use the correct range for local variable hn (DRF, 31 October 2020)
@x module 892
@!hn:small_number; {the number of positions occupied in |hc|}
@y
@!hn:0..64; {the number of positions occupied in |hc|;
                                  not always a |small_number|}
@z

Conclusion

Of course, the one change (and very likely the only one) people will see is:

-@d banner=='This is TeX, Version 3.14159265' {printed when \TeX\ starts}
+@d banner=='This is TeX, Version 3.141592653' {printed when \TeX\ starts}

:-)

As usual, there were several other changes to all of Knuth's distribution including, but not limited to, Metafont (some changes that were ported from TeX due to similar parts of the code), changes to The TeX and Metafont books, and changes to the plain format (everything really minimal and unlikely to bite any reasonable user document).

There is also a TUGboat article by Don describing the major changes to TeX and Metafont, available in the TUGboat web page.


Disclaimer

Most of the code examples in this answer are not my own, but taken from the original bug reports (some slightly modified), so thanks to the authors of those bug reports, and thanks also to the dedicated people that searched for bugs, and to those who read each and every one of the (probably thousands of) bug reports in the last 7 years. Let's prepare for another 8 of those!

17
  • Nice overview, you certainly put a lot of work into it. But I have a few comments to some statements. In I948, first line, \tracingparagraphs is on if it is >0. And note, there is only one error situation that shows the bug. All other problems are detected by TeX before it starts to break lines. That probably delayed the detection of this error; \tracingparagraphs is used only by experts, infinite shrinkage in a paragraph only by beginners until they read the excellent help message for this error. – Udo Wermuth Feb 14 at 16:23
  • 1
    @frougon Oops. Thanks for noticing! I don't offer checks though, sorry ;-) – Phelype Oleinik Feb 15 at 23:19
  • 2
    @UlrichDiez Given the bug is classified as B (A Blunder or Botch), I think the answer is probably “because Don didn't mean for an implicit brace to delimit the definition", but only Don (maybe Udo as well?) can answer you that – Phelype Oleinik Feb 28 at 9:03
  • 1
    @texdr.aft Then it could as well be a bug in the TeXbook, not in the program. The question arises how to decide whether the bug is in the program or in the user's manual. Especially as in this case in the preface of the user's manual/the TeXbook the "technique of deliberate lying" is praised as something that makes understanding ideas more easy (which I doubt, btw), which makes it difficult to decide whether one found a bug or just uncovered one of those acts of deliberate lying which might also manifest in not mentioning everything, e.g., about #<begin-group-character>-notation. – Ulrich Diez Feb 28 at 9:18
  • 1
    @UlrichDiez ...than the earlier ones do.'' So if you read about a topic and that description isn't an informal introduction in an early chapter then it is a reliable information. The information on page 275 about <definition text> and <left brace> and <right brace> in its syntax rules must be obeyed by \TeX's behavior as it is certainly not an informal description. And here it is stated clearly that <left brace> must be an explicit character. If the behavior of \TeX\ doesn't obey the written specification in the \TeX book and/or Vol. B then it must be considered a bug. – Udo Wermuth Feb 28 at 11:16
8

To complement the excellent answer by Phelype Oleinik, here is another version, with shorter/sloppier descriptions of the circumstances under which you can be affected by each change….

Background

Firstly, to avoid confusion about "first new version of TeX since 2014", note that:

  • LaTeX (not TeX), which is a system of macros on top of TeX, has much more frequent releases (LaTeX news).

  • Typical usage of TeX, especially as LaTeX, involves many packages that get updated at various intervals; each day many are updated (see ctan-ann).

So this "new version of TeX" is an update only to the underlying (and very stable) TeX engine, that too the "base" Knuth TeX (rather than say pdfTeX, XeTeX or LuaTeX—though these systems are derived from TeX and I imagine some or all of these changes will be or have been applied there too).

Unlike the previous "tune-up" (2014) where there was only one real change, this update (2021) has a whopping ten changes (the most in any single year since 1990, the year it was declared "done"), as counted by entries in errorlog.

What is this errorlog? This is a detailed record, manually maintained by Knuth, of all changes (not just "errors" technically) made to TeX since a very early state (entries go back to 1978). In 1989 he wrote an article called The Errors of TeX. (Reprinted with updates as Chapters 10–11 of the book Literate Programming and also further updated as the last chapter—"The Final Errors of TeX"—of the book Digital Typography.) That article describes the history of TeX's development and performs an analysis on this log, characterizing each entry (change made) into one of several categories: A for "algorithm awry", B for "blunder or botch", E for "efficiency enhancement", etc.

The categorization is subjective and personal to Knuth (sometimes about how he felt while making the change or the original error, or what led him to do it), and it's not always clear why he's put something in one category rather than another, but it's informative.

So for instance, the 2014 update had only one change:

  1. Space after \csname\endcsname to match other spaces.

and this was categorized under "C" for "a clean-up for consistency or clarity".

The changes

With that background, the following are the ten changes of this 2021 update (changes 948–957) by category. (My short descriptions here are sloppy and a function of my incomplete understanding; please see the accepted answer for details.)

Type B, "blunder or botch" ("sort of a mental typo")

952. Don't allow implicit left brace after #

Rules out an unnatural (well, even more unnatural than typical TeX-macro perversity) way of defining a macro.

Type C, "a clean-up for consistency or clarity"

956. Don't mark fraction noads as temporarily Inner

AFAICT, an internal detail with no visible effect, but makes the description in The TeXbook cleaner.

Type D, "a data structure debacle"

954. Garbage visible in buffer after file ends prematurely.

Prevents the "File ended within \read" error message (itself rare) from, under certain rare circumstances, showing additional garbage.

Type I, "an interactive improvement"

948. Don't pause on errors when tracing paragraphs

If you had set \tracingparagraphs and while logging the paragraph information to the log file there was an error, …

Type Q, "a quest for quality".

957. Reset \newlinechar before logging the stats

Error messages used to break lines depending on the value that was set by the user for \newlinechar at the time.

Type R, "a reinforcement of robustness"

951. Take date and time sometimes from system, not user

Similar: in places like the banner, don't use user-provided \date and \month but instead use the actual system time/date.

953. After nine parameters, must delete offending tokens

If you define a macro with more than nine parameters, and you ignore the error message "You already have nine parameters" thus continuing with TeX's error recovery, things are now left in a less surprising state.

955. Force nonexistent characters to have null specs

If you somehow managed to create a very particular kind of corrupted .tfm file, you could get the wrong italic correction, which you now don't.

Type S, "a surprising scenario"

949. Don't try to interact when in \batchmode

A devious way to make TeX actually crash, now fixed. (I haven't fully understood yet all circumstances under which this could happen…)

950. Don't try to edit when no file is active

If at TeX's error prompt you insert tokens interactively, and these inserted tokens themselves cause another error, and then at the next error prompt you ask to edit the file, it could have crashed earlier.

Summary

No typical user, with reasonable documents, is likely to be affected by any of these changes; they just close a few holes in corner cases. Nevertheless, they represent extremely close scrutiny of this program by many people over several years—and, at the same time, a remarkable enterprise towards bug-free software, over three decades after development ended.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.