# Pros and cons of the LaTeX3 syntax?

The programming language introduced by expl3 in LaTeX3 is a bit unusual compared to other TeX macro languages in that it uses both _ and : for function names. Could someone provide some technical insight on the pros and cons of the syntax?

-
I don’t think this question is appropriate for this site (and no, I don’t have anything to do with the LaTeX3 project). –  Caramdir Aug 31 '10 at 19:30
Not even as community wiki? –  Leo Liu Aug 31 '10 at 19:31
No, because I think it is an invitation for trolling. –  Caramdir Aug 31 '10 at 19:35
A few comments from someone who is involved. (1) If you use \fi: in your own stuff then you are probably missing the point of expl3: I'm pushing for a lot less 'exposure' of the renamed primitives in the documentation. (2) I think this is a discussion best had elsewhere, for example the LaTeX-L list. (3) I suspect that in any case this is likely to be subjective. CW isn't meant to be a way to 'get around' that type of thing. (4) Obviously, I'm not going to vote to close this question as it would be very questionable of me to do so! –  Joseph Wright Aug 31 '10 at 19:38
@Caramdir and Joseph: I would like some insight from people having intimate knowledge of TeX. Do you see a way to rephrase the question? –  Leo Liu Aug 31 '10 at 19:43

(Note: the answer covers things that were in various versions of the question!)

The question asks about two main issues, the 'primitives renamed' idea and the 'argument specification'. Just as background, the expl3 environment makes _ and : 'letters' for inclusion in control sequences, and also ignores whitespace. Using @ as a 'letter' has, at least in LaTeX2e, been somewhat unsystematic. The idea of using _ and : is that their role is more defined, with _ used to divide up function and variable names into readable 'blocks' and : used to show the arguments taken by a function.

On 'primitives renamed', in the main directly using the primitives is not the idea of the expl3 language. In the question, the case

\ifXXX:Nno
...
\fi:


was given. To put things on a more concrete footing, the current expl3 renames all of the primitive TeX \if... statements as, for example

\if_mode_horizontal: % \ifhmode
....
\else:
...
\fi:


The idea is, however, that the primitives should not really be used in most cases (outside of the kernel itself). So instead the recommended test would be

 \mode_if_horizontal:TF { true-code } { false-code }


for two branches or

\mode_if_horizontal:T { true-code }
\mode_if_horizontal:F { false-code }


for a single branch. I've argued that the documentation shouldn't even mention the 'renamed primitives' where there is a better alternative.

On the 'argument specification' idea, I should first say what it is. The idea of the expl3 approach to naming functions is that letters and _ are used to generate the name, and that : is then used to end the name and start a list of arguments. This can be used in a couple of ways. First, I've shown above how we can have three related functions

\mode_if_horizontal:T
\mode_if_horizontal:F
\mode_if_horizontal:TF


where the name is the same but the arguments are different. So the idea is that I can read the above an quickly see what the test is supposed to achieve.

The second part of the 'argument specification' idea is that it allows a systematic way to show expansion. An example is \cs_set_nopar:Npn, which is the expl3 name for \def. In LaTeX2e we also have \@namedef as a shortcut for

\expandafter\def\csname #1\endcsname


but it is hard to extend this to all of the possibilities. The same idea in expl3 is called \cs_set_nopar:cpn. Notice that the only difference is the argument specification:

\cs_set_nopar:Npn \ControlSequence    #1#2 { stuff here }
\cs_set_nopar:cpn { ControlSequence } #1#2 { stuff here }


I can then go further, as \edef is specified in the above by using an 'x' in the last position rather than an 'n':

\cs_set_nopar:Npx \ControlSequence    #1#2 { stuff here } % \edef
\cs_set_nopar:cpx { ControlSequence } #1#2 { stuff here } % \@nameedef


Obviously there are more example that can be given for this, but that is best studied in something like 'expl3.pdf' or recent TUGBoat articles I've written.

The question asked if the community likes these ideas. On that, I can offer no real answer other than a personal perspective. There is of course a learning curve to the syntax, and that will put some people off. However, I think that it does address at least some of the issues with writing code in TeX.

The question also asked about the downsides. One obvious issue is that _ is used by TeX for subscripts, and so there are some issues in coding subscripts with expl3. That can be got around using the normal TeX approach to generating tokens with awkward category codes: \lowercase (in expl3 \tl_to_lowercase:n, at least at the moment). Obviously, this requires some knowledge of how to program TeX, and it would be best if an alternative was available.

There is also the point that expl3 is still be altered as new things come up. While that means that useful changes can happen it also means you need to be prepared to keep up to date. Usually the changes are flagged up in the CTAN uploads but there is still a need to do testing with the current expl3 release if you want to be sure things work.

(LuaTeX offers the possibility to program in Lua, which is obviously an alternative approach to expl3. That seems such a big difference that I'm not going to attempt to cover it in this answer!)

-
Thank you for the answer and examples. Sorry to change the question statement before your answer. Feel free to modify it to be in sync with your answer. –  Leo Liu Aug 31 '10 at 20:21
I think I'll leave it as-is, and see if there are any other comments on the particular choice of 'letters'. –  Joseph Wright Aug 31 '10 at 20:56
Can always use \sb to get subscripts :-) –  Will Robertson Sep 1 '10 at 1:24
Too late, practically speaking, to change, I'm afraid :) –  Will Robertson Sep 1 '10 at 12:42
Will's right. I get the feeling we could talk for 10 more years about expl3 and not satisfy everyone, or even those people who like the basic idea at all. At some stage things do have to be decided, for better or worse. As I said somewhere, this is to some extent a matter of preference rather than a clear technical decision. –  Joseph Wright Sep 1 '10 at 16:24

Disclaimer: originally the question was phrased as like/dislike instead of pros/cons, and that is the only way in which I can provide an answer. I do not have enough practical experience in expl3 to comprehend the technical (de)merits of the language. So this answer is an (outside) opinion piece only.

To readers that know my background as one of the luatex and ConTeXt developers it should come as no surprise that I do not like the new syntax.

Taking the definition of \mode_if_horizontal:TF as an example:

\prg_set_conditional:Npnn \mode_if_horizontal: {p,TF,T,F}{
\if_mode_horizontal:
\prg_return_true: \else: \prg_return_false: \fi:
}


This looks like I would have to learn a completely new programming language that looks similar to, but is clearly not the same as, TeX macros. Taking into account that it took me more than a year at a full-time job to learn how to write TeX macros properly, this scares me quite a bit.

In the example, the naming of the internal macros and the use of a comma-separated list suggest that the intent is to make macros behave more like functions in other languages would. But looking at expl3 as a beginner while already having a lot of knowledge of the underlying engine, I do not truly believe that the new syntax will succeed in shielding off the underlying engine. The problem is that if it fails to do that then it is not a replacement language, but an additional language that needs to be learned and used on top of the already existing macro language. And, at least to me, it is an additional language with very unfamiliar syntax and semantics.

All in all, I much prefer traditional TeX style macros with lua for more complex tasks.

-
You might be right that expl3 can only really be learned by those who already know how to program with TeX macros. The other side of expl3 is that it is essentially a library of higher level tools that what LaTeX gives you (which is higher level again than TeX), so any system along the same lines (etoolbox, etc.) has its own learning curve. –  Will Robertson Sep 1 '10 at 5:41
P.S. I don't take huge pains to turn my code into expl3-only syntax. There's no problem at all with using \def, \expandafter, and others while also using expl3's data structures and conditionals. –  Will Robertson Sep 1 '10 at 5:50
Yes, I understand. But it is a library of higher level tools using different language conventions so on top of having to learn the semantics of the new tool set, one also has to learn the syntax of the new tool set. –  Taco Hoekwater Sep 1 '10 at 5:53
Every time this sort of discussion arises I'm left wondering if a better approach would be to go the way of something like etoolbox. You can imagine writing a 'better LaTeX kernel' in a more systematic way using e-TeX and providing thee tools while mainly using TeX primitives. But then I wonder if it really would be clear enough. In the end I'm left not knowing what to think! –  Joseph Wright Sep 1 '10 at 6:59
@Taco: LaTeXe programming is so ad hoc it's hard to argue that it even has language conventions. I get that expl3 looks like it has a steep learning curve at first blush, but I don't think it's as bad as it looks. But I'm certainly going to admit it's not for everyone. @Joseph: I think expl3 is the right way to go :) –  Will Robertson Sep 1 '10 at 8:04

### Whitespace

When writing expl3 code, whitespace is ignored. This produces more readable code because you don't have comment chars littering the ends of most lines. Consider

\cs_set:Nn \foo: {
\mode_if_math:TF {
\foo_math:
}{
\foo_text:
}
}


And now:

\cs_set:Nn \foo: {%
\mode_if_math:TF {%
\foo_math:
}{%
\foo_text:
}%
}


It might not seem much, but it takes away the cognitive burden of dealing with whitespace problems. You can change your indentation/brace style without worrying if you need to add %, and more C-like brace syntax is possible without messes of % everywhere.

By the way, if you need to insert a space in expl3 code, you use the ~ char instead, which has catcode 10 for this purpose.

-
Catcode 10 means that ~~ is not two spaces? –  Charles Stewart Sep 1 '10 at 9:00
That's right. But remember this is the programmer's interface -- it does not change the behaviour of ~ on the document/user level. expl3 has commands such as \msg_four_spaces: if you want more than one space in an error message or whatever. –  Will Robertson Sep 1 '10 at 9:03
Can you write \cs_set: Nn for \cs_set:Nn? –  Leo Liu Sep 1 '10 at 11:22
No, for efficiency the arg spec is "baked in" as part of the macro name. –  Will Robertson Sep 1 '10 at 12:41

### Argument specifications

One of the fundamental improvements of the expl3 syntax revolves around its use of argument specifiers; that is, what comes after the : in the function names.

These argument specifiers signify how the arguments to a function will be pre-processed before being executed upon. For example, TeX provides the \let primitive to copy the meaning of one control sequence to another.

\let \foo \bar


In expl3, this is written as

\cs_set_eq:NN \foo \bar


The "N" means an argument of a single token, and \cs_set_eq:NN in this case is the base function from which variants are created. For example, this is equivalent to the above:

\cs_set_eq:cc {foo} {bar}


What the "cc" is saying is that before being executing \cs_set_eq:NN, first take the two arguments and turn them into control sequences. Other variants in this case are :cN and :Nc. To write this in TeX syntax would be

\expandafter \let \csname foo\expandafter \endcsname \csname bar\endcsname


I hope you'll agree the expl3 version is more readable and maintainable.

Another argument specifier is "V", whose meaning is "take the next token and replace it with the data stored by that token". For example,

\def\baz{\foo}
\cs_set_eq:VN \baz \bar


The "V" arg spec works not only with token list variables (or macros) as above, but also with token registers, integers, and so on.

Now, it happens that \cs_set_eq:VN isn't defined out of the box in expl3. No problem, there is a generic method for creating new variants:

\cs_generate_variant:Nn \cs_set_eq:NN {VN}


In brief, the full list of specifiers are

N n c V v x f o T F


The "T" and "F" specifiers don't do any expansion; they exist solely as a shorthand to signify true and false branches of a conditional. Please read the expl3 docs for more information.

While other programming tools for LaTeX convey similar ideas, the advantage in my eyes to expl3's approach is the generic and extensibility of the argument specifier syntax. Your package has a command that takes nine arguments? No problem at all to write

\cs_set:Nn \foo_something:nnnnnnnnn {...}
\cs_generate_variant:Nn \foo_something:nnnnnnnnn {cxVnfcxcv}


Obviously ridiculous, but in practise very powerful and convenient.

-

TeX does not provide any namespacing, and so there is only convention to keep the macros/variables of one package separate from another. Knuth introduced the idea of using an extra 'letter' in the plain TeX format, where he used @ in names which were not intended for use in typesetting. Both LaTeX and ConTeXt have continued to use the idea of one or more additional 'letters' for internal macros. At a technical level there is no reason to enforce such a separation: one could simply use variation of case to make the origin of macros obvious. For example, for a package 'foo' the usual LaTeX2e approach would be to use

\foo@internal@macro


while for expl3 we might have

\foo_internal_macro:


but there is no reason not to simply use

\FooInternalMacro


from a purely technical point of view.

LaTeX2e uses @ as a 'letter'. This is unfortunately not done systematically, with internal names often containing arbitrary numbers of @ symbols. This does not make for clear code. However, this has nothing to do with the choice of symbol: @ could be applied systematically. ConTeXt uses a few more 'letters', for example !, in code blocks and is more systematic.

The choice of _ and : for expl3 is to some extent arbitrary, although both symbols are seen in other programming languages. To my knowledge, @ is much less used in this way. As TeX only relies on category code, the choice of _ and : can be regarded as being one about readability. This is of course a highly subjective area!

One clear issue with using _ is that it is used by TeX to indicate a subscript. The overlap between code (which expl3 is intended for) and typesetting math (where _ is needed for subscripts) is small. This means that an approach using the \lowercase primitive is possible when a _ is needed inside a code block to indicate a subscript. However, you can argue that an alternative choice may have been better here.

-

I see two potential problems with expl3. The first is one of readability. Not only are there a whole new set of macros to learn for the ones we already know, their names seem to have gotten longer and are more opaque. \cs_set_nopar:Npn is far less clear than \def. (For a better example, look at the definition of \ExplSyntaxOff and compare that to its second definition (whose purpose I don't follow) in the expansion of \ExplSyntaxOn. On the plus side, the second definition is a lot happier :D) Worse, \cs_set_nopar:cpn and \cs_set_nopar:Npx look extremely similar to \cs_set_nopar:Npn. Maybe this is easy to get passed with a little practice.

Of course, LaTeX 2e doesn't do so well with that last point either. \@namedef and \@nameedef look extremely similar.

The changes to \if... also seem strange, beyond just the name expansion and rearrangement. Maybe it's easy to work around, but the two arms of the if seem to get tokenized completely. This seems potentially bad.

The second potential problem is debugging. As we all know, debugging LaTeX is painful. Often, the best way is to use TeX's tracing ability and see what it is actually doing. Consider the following if statement.

\ifhmode T\else F\fi


It's a very simple piece of TeX code and if we look at the tracing output, that is reflected.

{\ifhmode}
{true}
{the letter T}
{\else}


(The false branch is similar except it says the letter F and \fi, but it also enters hmode so there is some text in the log about that.)

Now consider the corresponding piece of expl3.

\mode_if_horizontal:TF{T}{F}


Apart from the if not being at the beginning of the name (this name makes me think it is going to set some mode if something else is horizontal), it is as straight-forward as the first snippet. Unfortunately, the log output is far less clear.

\mode_if_horizontal:TF ->\if_mode_horizontal: \prg_return_true: \else: \prg_ret
urn_false: \fi: \c_zero \exp_after:wN \use_i:nn \else: \exp_after:wN \use_ii:nn
\fi:
{\ifhmode}
{true}

\prg_return_true: ->\exp_after:wN \if_true: \tex_romannumeral:D
{\expandafter}
{\romannumeral}
{\else}
{\iftrue}
{true}
{\expandafter}
{\else}

\use_i:nn #1#2->#1
#1<-T
#2<-F
{the letter T}


I'm trying to imagine what this would look like for a complicated piece of code. The most confusing part is this could have been written very simply as

\def\mode_if_horizontal:TF{\ifhmode\expandafter\@firstoftwo\else\expandafter\@secondoftwo\fi}


(substitute \use_i:nn and \use_ii:nn as necessary). It still wouldn't be quite as short, but at least it wouldn't be using \romannumerial.

Of course, it could be that it's just so much easier to write readable code that no one will ever have to look at TeX's execution.

Either way, I'm looking forward to seeing what LaTeX3 has to offer.

-
\cs_set_nopar is a special case of ugly because its name was chosen specifically because at the time we believed that the non-long version really didn't have much place in internal code. Use/compare \cs_set:Npn instead. With :Npx being too similar, I disagree -- you learn to "spot" these argument specifiers quite well due to the systematic nature of the language. –  Will Robertson Aug 31 '10 at 23:58
You're right that tracing is made more complex, but ive never had much luck tracing complex code anyway. (extending the trace package to simplify expl3 tracing would be a great idea.) The expl3 conditionals are quite neat, and you can use Boolean logic with when the tests are expandable. –  Will Robertson Sep 1 '10 at 0:01
I think the real problem I have with the argument specifiers is that unlike standard practice in programming languages which allow function overloading like C++ or Java, these macros are doing fundamentally different things based on their arguments. For example, if foo is an int and bar is a float, then one would expect Set(foo) and Set(bar) to behave roughly the same: setting something to the argument. In contrast, \cs_set:Npn \cs_set:Npx treat their arguments extremely differently, despite having the same name. –  TH. Sep 1 '10 at 0:24
To be clear, I don't have a problem with \cs_set:Npn and \cs_set:cpn. They do the same thing. But maybe this is just something one gets used to. –  TH. Sep 1 '10 at 0:28
I think you might have a slight misunderstanding of how the argument specifiers work. (something for us to improve in the docs.) not only do the argument specifiers say how many arguments there are, they also specify how they are to be expanded before being passed to the underlying functions. So \foo_something:c means take the argument, turn it into a csname, and then apply the base function \foo_something:N to it. –  Will Robertson Sep 1 '10 at 0:42

I've done a couple of Google searches.

1. web search for mode_if_horizontal produces 3 hits (not including this page, which hasn't been indexed yet).

2. open source code search for mode_if_horizontal produces no hits.

-
If you'd searched for \cs_set_nopar:Npn you'd only have found the Project stuff, I think :-) \mode_if_horizontal:TF was possibly not the best choice of 'useful' functions: perhaps I should revise it. (EDIT: I see that all of the hits are back to the Project anyway.) –  Joseph Wright Aug 31 '10 at 20:42
Now this page is the first hit for the former, and I see 9 in total for it. The latter gets a grand total of one hit -- and it's in Emacs lisp code (something to do with auctex, naturally). –  SamB Dec 1 '10 at 5:56