Pros and cons of the LaTeX3 syntax?

Question

The programming language introduced by expl3 in LaTeX3 is a bit unusual compared to other TeX macro languages in that it uses both _ and : for function names. Could someone provide some technical insight on the pros and cons of the syntax?

Answer 1

(Note: the answer covers things that were in various versions of the question!)

The question asks about two main issues, the 'primitives renamed' idea and the 'argument specification'. Just as background, the expl3 environment makes _ and : 'letters' for inclusion in control sequences, and also ignores whitespace. Using @ as a 'letter' has, at least in LaTeX2e, been somewhat unsystematic. The idea of using _ and : is that their role is more defined, with _ used to divide up function and variable names into readable 'blocks' and : used to show the arguments taken by a function.

On 'primitives renamed', in the main directly using the primitives is not the idea of the expl3 language. In the question, the case

\ifXXX:Nno
  ...
\fi:

was given. To put things on a more concrete footing, the current expl3 renames all of the primitive TeX \if... statements as, for example

\if_mode_horizontal: % \ifhmode
  ....
\else:
  ...
\fi:

The idea is, however, that the primitives should not really be used in most cases (outside of the kernel itself). So instead the recommended test would be

 \mode_if_horizontal:TF { true-code } { false-code }

for two branches or

\mode_if_horizontal:T { true-code }
\mode_if_horizontal:F { false-code }

for a single branch. I've argued that the documentation shouldn't even mention the 'renamed primitives' where there is a better alternative.

On the 'argument specification' idea, I should first say what it is. The idea of the expl3 approach to naming functions is that letters and _ are used to generate the name, and that : is then used to end the name and start a list of arguments. This can be used in a couple of ways. First, I've shown above how we can have three related functions

\mode_if_horizontal:T
\mode_if_horizontal:F
\mode_if_horizontal:TF

where the name is the same but the arguments are different. So the idea is that I can read the above an quickly see what the test is supposed to achieve.

The second part of the 'argument specification' idea is that it allows a systematic way to show expansion. An example is \cs_set_nopar:Npn, which is the expl3 name for \def. In LaTeX2e we also have \@namedef as a shortcut for

\expandafter\def\csname #1\endcsname

but it is hard to extend this to all of the possibilities. The same idea in expl3 is called \cs_set_nopar:cpn. Notice that the only difference is the argument specification:

\cs_set_nopar:Npn \ControlSequence    #1#2 { stuff here } 
\cs_set_nopar:cpn { ControlSequence } #1#2 { stuff here } 

I can then go further, as \edef is specified in the above by using an 'x' in the last position rather than an 'n':

\cs_set_nopar:Npx \ControlSequence    #1#2 { stuff here } % \edef
\cs_set_nopar:cpx { ControlSequence } #1#2 { stuff here } % `\@nameedef`

Obviously there are more example that can be given for this, but that is best studied in something like 'expl3.pdf' or recent TUGBoat articles I've written.

The question asked if the community likes these ideas. On that, I can offer no real answer other than a personal perspective. There is of course a learning curve to the syntax, and that will put some people off. However, I think that it does address at least some of the issues with writing code in TeX.

The question also asked about the downsides. One obvious issue is that _ is used by TeX for subscripts, and so there are some issues in coding subscripts with expl3. That can be got around using the normal TeX approach to generating tokens with awkward category codes: \lowercase (in expl3 \tl_to_lowercase:n, at least at the moment). Obviously, this requires some knowledge of how to program TeX, and it would be best if an alternative was available.

There is also the point that expl3 is still be altered as new things come up. While that means that useful changes can happen it also means you need to be prepared to keep up to date. Usually the changes are flagged up in the CTAN uploads but there is still a need to do testing with the current expl3 release if you want to be sure things work.

(LuaTeX offers the possibility to program in Lua, which is obviously an alternative approach to expl3. That seems such a big difference that I'm not going to attempt to cover it in this answer!)

Answer 2

Disclaimer: originally the question was phrased as like/dislike instead of pros/cons, and that is the only way in which I can provide an answer. I do not have enough practical experience in expl3 to comprehend the technical (de)merits of the language. So this answer is an (outside) opinion piece only.

To readers that know my background as one of the luatex and ConTeXt developers it should come as no surprise that I do not like the new syntax.

My main objection is that it adds additional complexity to a task (macro programming) that is already highly complex.

Taking the definition of \mode_if_horizontal:TF as an example:

\prg_set_conditional:Npnn \mode_if_horizontal: {p,TF,T,F}{
   \if_mode_horizontal: 
   \prg_return_true: \else: \prg_return_false: \fi:
}

This looks like I would have to learn a completely new programming language that looks similar to, but is clearly not the same as, TeX macros. Taking into account that it took me more than a year at a full-time job to learn how to write TeX macros properly, this scares me quite a bit.

In the example, the naming of the internal macros and the use of a comma-separated list suggest that the intent is to make macros behave more like functions in other languages would. But looking at expl3 as a beginner while already having a lot of knowledge of the underlying engine, I do not truly believe that the new syntax will succeed in shielding off the underlying engine. The problem is that if it fails to do that then it is not a replacement language, but an additional language that needs to be learned and used on top of the already existing macro language. And, at least to me, it is an additional language with very unfamiliar syntax and semantics.

All in all, I much prefer traditional TeX style macros with lua for more complex tasks.

Answer 3

Whitespace

When writing expl3 code, whitespace is ignored. This produces more readable code because you don't have comment chars littering the ends of most lines. Consider

\cs_set:Nn \foo: {
  \mode_if_math:TF {
    \foo_math:
  }{
   \foo_text:                
  }
}

And now:

\cs_set:Nn \foo: {%
  \mode_if_math:TF {%
    \foo_math:
  }{%
   \foo_text:                
  }%
}

It might not seem much, but it takes away the cognitive burden of dealing with whitespace problems. You can change your indentation/brace style without worrying if you need to add %, and more C-like brace syntax is possible without messes of % everywhere.

By the way, if you need to insert a space in expl3 code, you use the ~ char instead, which has catcode 10 for this purpose.

Answer 4

Argument specifications

One of the fundamental improvements of the expl3 syntax revolves around its use of argument specifiers; that is, what comes after the : in the function names.

These argument specifiers signify how the arguments to a function will be pre-processed before being executed upon. For example, TeX provides the \let primitive to copy the meaning of one control sequence to another.

\let \foo \bar

In expl3, this is written as

\cs_set_eq:NN \foo \bar

The "N" means an argument of a single token, and \cs_set_eq:NN in this case is the base function from which variants are created. For example, this is equivalent to the above:

\cs_set_eq:cc {foo} {bar}

What the "cc" is saying is that before being executing \cs_set_eq:NN, first take the two arguments and turn them into control sequences. Other variants in this case are :cN and :Nc. To write this in TeX syntax would be

\expandafter \let \csname foo\expandafter \endcsname \csname bar\endcsname

I hope you'll agree the expl3 version is more readable and maintainable.

Another argument specifier is "V", whose meaning is "take the next token and replace it with the data stored by that token". For example,

\def\baz{\foo}
\cs_set_eq:VN \baz \bar

The "V" arg spec works not only with token list variables (or macros) as above, but also with token registers, integers, and so on.

Now, it happens that \cs_set_eq:VN isn't defined out of the box in expl3. No problem, there is a generic method for creating new variants:

\cs_generate_variant:Nn \cs_set_eq:NN {VN}

In brief, the full list of specifiers are

N n c V v x f o T F

The "T" and "F" specifiers don't do any expansion; they exist solely as a shorthand to signify true and false branches of a conditional. Please read the expl3 docs for more information.

While other programming tools for LaTeX convey similar ideas, the advantage in my eyes to expl3's approach is the generic and extensibility of the argument specifier syntax. Your package has a command that takes nine arguments? No problem at all to write

\cs_set:Nn \foo_something:nnnnnnnnn {...}    
\cs_generate_variant:Nn \foo_something:nnnnnnnnn {cxVnfcxcv}    

Obviously ridiculous, but in practise very powerful and convenient.

Answer 5

(Answering the question as revised.)

TeX does not provide any namespacing, and so there is only convention to keep the macros/variables of one package separate from another. Knuth introduced the idea of using an extra 'letter' in the plain TeX format, where he used @ in names which were not intended for use in typesetting. Both LaTeX and ConTeXt have continued to use the idea of one or more additional 'letters' for internal macros. At a technical level there is no reason to enforce such a separation: one could simply use variation of case to make the origin of macros obvious. For example, for a package 'foo' the usual LaTeX2e approach would be to use

\foo@internal@macro

while for expl3 we might have

\foo_internal_macro:

but there is no reason not to simply use

\FooInternalMacro

from a purely technical point of view.

LaTeX2e uses @ as a 'letter'. This is unfortunately not done systematically, with internal names often containing arbitrary numbers of @ symbols. This does not make for clear code. However, this has nothing to do with the choice of symbol: @ could be applied systematically. ConTeXt uses a few more 'letters', for example !, in code blocks and is more systematic.

The choice of _ and : for expl3 is to some extent arbitrary, although both symbols are seen in other programming languages. To my knowledge, @ is much less used in this way. As TeX only relies on category code, the choice of _ and : can be regarded as being one about readability. This is of course a highly subjective area!

One clear issue with using _ is that it is used by TeX to indicate a subscript. The overlap between code (which expl3 is intended for) and typesetting math (where _ is needed for subscripts) is small. This means that an approach using the \lowercase primitive is possible when a _ is needed inside a code block to indicate a subscript. However, you can argue that an alternative choice may have been better here.

Answer 6

I see two potential problems with expl3. The first is one of readability. Not only are there a whole new set of macros to learn for the ones we already know, their names seem to have gotten longer and are more opaque. \cs_set_nopar:Npn is far less clear than \def. (For a better example, look at the definition of \ExplSyntaxOff and compare that to its second definition (whose purpose I don't follow) in the expansion of \ExplSyntaxOn. On the plus side, the second definition is a lot happier :D) Worse, \cs_set_nopar:cpn and \cs_set_nopar:Npx look extremely similar to \cs_set_nopar:Npn. Maybe this is easy to get passed with a little practice.

Of course, LaTeX 2e doesn't do so well with that last point either. \@namedef and \@nameedef look extremely similar.

The changes to \if... also seem strange, beyond just the name expansion and rearrangement. Maybe it's easy to work around, but the two arms of the if seem to get tokenized completely. This seems potentially bad.

The second potential problem is debugging. As we all know, debugging LaTeX is painful. Often, the best way is to use TeX's tracing ability and see what it is actually doing. Consider the following if statement.

\ifhmode T\else F\fi

It's a very simple piece of TeX code and if we look at the tracing output, that is reflected.

{\ifhmode}
{true}
{the letter T}
{\else}

(The false branch is similar except it says the letter F and \fi, but it also enters hmode so there is some text in the log about that.)

Now consider the corresponding piece of expl3.

\mode_if_horizontal:TF{T}{F}

Apart from the if not being at the beginning of the name (this name makes me think it is going to set some mode if something else is horizontal), it is as straight-forward as the first snippet. Unfortunately, the log output is far less clear.

\mode_if_horizontal:TF ->\if_mode_horizontal: \prg_return_true: \else: \prg_ret
urn_false: \fi: \c_zero \exp_after:wN \use_i:nn \else: \exp_after:wN \use_ii:nn
 \fi:
{\ifhmode}
{true}

\prg_return_true: ->\exp_after:wN \if_true: \tex_romannumeral:D
{\expandafter}
{\romannumeral}
{\else}
{\iftrue}
{true}
{\expandafter}
{\else}

\use_i:nn #1#2->#1
#1<-T
#2<-F
{the letter T}

I'm trying to imagine what this would look like for a complicated piece of code. The most confusing part is this could have been written very simply as

\def\mode_if_horizontal:TF{\ifhmode\expandafter\@firstoftwo\else\expandafter\@secondoftwo\fi}

(substitute \use_i:nn and \use_ii:nn as necessary). It still wouldn't be quite as short, but at least it wouldn't be using \romannumerial.

Of course, it could be that it's just so much easier to write readable code that no one will ever have to look at TeX's execution.

Either way, I'm looking forward to seeing what LaTeX3 has to offer.

Answer 7

I've done a couple of Google searches.

1. web search for mode_if_horizontal produces 3 hits (not including this page, which hasn't been indexed yet).

2. open source code search for mode_if_horizontal produces no hits.