140

I'm fairly new to TeX and LaTeX, having been drawn in by the Big Idea — creating a document programmatically. However, my experience of TeX (and its libraries — LaTeX, ConTeXt, …) has been almost entirely unpleasant.

My impression is that TeX (etc.) is essentially a pile of undocumented macros (where even are functions?) in an unparsable syntax (I am told it's Turing-complete), that very often collapses (most error messages have a mysterious origin), with shaky support for modern frameworks (I have to read a paper on font naming schemes to find how to apply Helvetica?), and no serious opportunity for actual programming (as opposed to braindead markup), with a confusingly fragmented codebase (which binary, distribution, library should I use, and how do they interact?).

Given that the Big Idea is a laudable one, and that TeX and friends are far from an ideal implementation of it, I'd like to know:

Why it is that, within the problem domain of programmatic document formatting, TeX is the only serious player? Does no-one have an idea of how do to this better? What would TeX look like if we were to design it from scratch?

(Note: I'm not looking for a flame war; I'm looking for genuine answers.)

21
  • 37
    eegg, welcome to TeX.SX. Note that this is a Q&A site and not for discussions or even opinions, so I'm very tended to close this post as "subjective and argumentative. Note that TeX is not "a pile of undocumented macros" but very well documented in "The TeXBook". LaTeX and its packages come with documentation and there are dozens of books about this topic. A simple Google search for "Helvetica latex" tells you how to apply it. ..... Mar 14, 2011 at 11:52
  • 14
    @eegg: While the guys at TeX-SE are quite civilized and helpful, we cannot help but notice your accusatory and generally negative tone. People would be much more willing to discuss TeX deficiencies (of which there are many) if you edit your question and try to phrase it in a neutral manner. With that being said, welcome on board, and enjoy your stay! Mar 14, 2011 at 12:06
  • 9
    After pondering this all afternoon, I've come down on the side that I don't like this type of question and don't want to see it here. If you do want "genuine answers" then please ask a genuine question. This place is for finding specific answers to specific questions, not for discussions. You could easily turn each point in your post in to a specific (and neutral) question, in which case I'd happily join in trying to convince you of the value of TeX, but I find myself not wanting to get involved in this for fear it might turn nasty. Mar 14, 2011 at 19:12
  • 16
    22 up-votes and a lot of traffic. But alas, the question has been closed. Something is wrong with this picture. Mar 15, 2011 at 13:41
  • 12
    I think this is a very good question. Yes, the tone is a bit offensive, especially for diehard TeX fans. I'm a big fan of TeX, and of Knuth. Yet the question in bold is a very good one. Yes, it could be posed more constructively, but it's definitely worth discussing.
    – LarsH
    May 31, 2013 at 15:10

8 Answers 8

79

(My apologies in advance for any "too long; didn't read" sentiments -- if you only read one thing out of this, make it this link: Best Way to Start Using LaTeX/TeX?)

My Non-Expert Background:

  • I started using LaTeX around 2002 to help engineering students at my university write their M.S. theses and PhD dissertations.
  • For several years, I made basic modifications to an older thesis style file dating to the early 1990s, and made two or three attempts at writing one from scratch.
  • I finally succeeded with a version I started in 2008, and made a lot of changes to it recently to isolate my users from the style's inner structure by default. The new style covers every feature of the old style, adds quite a bit more automation for our odd page headings in the ToC and other lists, and has around 50% fewer lines of code.

Things I've learned:

  • Writing LaTeX documents is programming as soon as you get to even a moderate level of document requirements.
  • It can be programmed badly, or programmed efficiently.
  • How badly or efficiently you program it is strongly influenced by obsessive-compulsive tendencies, curiosity, and critical thinking.

As for specific issues posed:

most error messages have a mysterious origin

I can see where you're coming from there. You certainly don't get a clean Pythonic stack trace. There's no easy debugging tools (ConTeXt may have better support, though). Debugging is generally a process of producing minimal working examples, which either lets you identify where the problem went away as you started commenting and removing code, or at least scopes the problem down enough to where you can get expert advice more easily.

Really, this reminds me of how we'd debug Fortran code back in the day with 'echo checks' when we didn't have a good debugger (or else didn't know how to use one).

I have to read a paper on font naming schemes to find how to apply Helvetica?

I'm assuming this is partially hyperbole, but it does speak to the varying levels of documentation available. Some documentation is out of date, some is incomplete. Googling 'latex font' returns a pretty unpredictable range of answers, including some that are either just wrong, or outdated, or incomplete.

Googling 'latex font selection' returned other pages, including this Wikibooks page, but it's not quite what you were looking for unless you wanted to use XeTeX (which was created, in part, to handle Unicode and arbitrary fonts more easily).

I did eventually search this site for font information, and found How do I change a font document wide? -- it didn't give a complete example, but has all the pieces there. A MWE for Helvetica by default is

\documentclass{article}
\usepackage{helvet}
\renewcommand\familydefault{\sfdefault}
\begin{document}
Hello, world.
\end{document}

where even are functions? (with a diversion into OCD territory)

I'm not sure how you define functions versus macros as such, but I can certainly define blocks of reusable code and call them. Those blocks can take varying numbers of arguments, and act accordingly. They can effectively provide default values for arguments. LaTeX's global namespace can be a hindrance, but is generally worked around by tagging local variables with a special prefix or similar.

Example: our signature sheet for the thesis committee needs one line per committee member. Some committee members have a particular title (chair or cochair), others have no particular title. Each line should also include a spot for the member to write the approval date. At one time, we had a relative ton of copied and pasted code that only differed in whether we were typesetting the name for \memberone, \membertwo, etc. and whether the person had a specific title.

Now, we have a single function (or macro, I guess) that takes two arguments, and can typeset any member's signature line.

\newcommand{\nameanddateline}[2]{% irrelevant spacing and other commands removed
  \eqparbox{name}{\hrulefill} \quad \parbox{.75in}{\hrulefill} \\ 
  \eqparbox{name}{#1\ifthenelse{\equal{#2}{}}{\relax}{, #2}}
  \quad \makebox[.75in][c]{\hfill{}Date\hfill{}} \\ 
}

Called as \nameanddateline{\committeechair}{\chairtitle} we get a signature line with the chair's title added. Called as \nameanddateline{\memberone}{}, we get a signature line with no title at all. In a version I'll be posting in the next few days, I even got rid of the \memberN parts in favor of an array of names (students would use \renewcommand{\committeemembers}{Member 1, Member 2, ...}, and I can iterate over the comma-separated values in \committeemembers.

LaTeX won't force you to write efficient or maintainable code or markup. I don't claim mine is top-tier. My Python code would possibly make you cry in anguish. But it's like any other language in regards to it being up to the programmer to decide how clean they want things to be.

no serious opportunity for actual programming (as opposed to braindead markup)

I'm not sure what you'd call braindead markup. There's enough programming capability in it for me to get my documents typeset in a relatively efficient fashion. I can define my own commands, I can loop over elements of a simple array defined as comma-separated values, and I have if/then/else structures. For my needs, it's worked out fine. I'd certainly prefer this to writing VBA or fighting master documents and other issues in MS Word.

There's a lot of stale information out there. There are a lot of bad habits ingrained in long-time users who haven't kept up with new packages, or haven't thought through how to write their documents more like code, and less like explicit markup. Case in point: I've got a guy who has 1700+ \bf commands in his dissertation, each of which throws errors in newer memoir classes by default. All of those commands are for defining vectors. If he had ever read any of the intermediate material that said you could define a \bvec{} command to do the vector formatting, he'd only have to change \newcommand\bvec[1]{\bf #1} to \newcommand\bvec[1]{\bfseries #1} once.

Similarly, a collaborator brought some odd theorem spacing to my attention. He had been working around it by including a \vspace* command to remove the extra space. I thought, "this can't possibly be the way it's designed to work" and my OCD kicked in. Within an hour or so, I had found a much cleaner solution using the \qedhere command. It's well-documented at Wikibooks, but you'd have to go look for it rather than simply implement the first fix that comes to mind.

I'm disheartened by things like this, but not shocked. Maybe my standards are too low -- just a few years ago, I had a student ask me why her professor's Fortran finite element code wouldn't compile on her system, but worked fine on his. I started writing Fortran in 1989, and I've adapted code written in the early 70's. I didn't recognize the notation on the lines throwing errors. One of the other faculty looked at and said, "Oh, that's Hollerith notation." That notation was deprecated in the Fortran 66 standard, and deleted entirely in Fortran 77. The original professor was still using it in 2004 or so. Why? Because it still compiled for him. There's a similar effect in some long-time LaTeX users.

a confusingly fragmented codebase (which binary, distribution, library should I use, and how do they interact?).

Thankfully, the fragmentation is more in theory than in practice. At least in my experience, LaTeX code generally is distribution-neutral. I've had to modify one command in my thesis style to handle two different versions of the memoir class (one from 2005 in PCTeX, and the current one on CTAN). But all I had to do was to check if a particular very simple command was defined, and if not, define it myself.

CTAN, I think, is pretty good about having clean, updated packages. Instructions or packages you get from a random Google search can obviously vary quite a lot.

31

Why it is that, within the problem domain of programmatic document formatting, TeX is the only serious player?

It's not obvious to me what you mean by this, but on the face of it, the presupposition isn't true. Based on Java, there are two exceedingly capable tools for generating PDFs: Open Office has a completely programmable back-end, whilst i-Text, a Java library for creating PDFs, allows almost complete coverage of PDF features, and is more capable in this respect than can be done with Tex engines (try putting a digital signature on your Latex'd letters). Both of these support a method of generating documents similar to HTML::Template where the document template and code are kept separate, which many regard as cleaner and more maintainable than how Latex documents are often written. FWIW, recent versions of Word allow VBA to access .NET libraries, allowing a quite high level of programmatic sophistication in document creation as well.

The strength of the Tex family doesn't lie, IMO, in its feature list, although with respect to microtypography, none of the above systems are competitive with Luatex, but rather in the ethos of document preparation and styling. Knuth's creation was extraordinarily rich. I have lost count of the number of ascii notations for mathematics whose authors believe it to be better than Knuth's: none in fact are. Knuth did not attempt to give the most general model, but just one sufficient for mathematicians writing for publication, but his model has weathered well and has been hugely influential on later designs, which generally have learned only some of the lessons on offer.

It takes time to become competent with any of the Tex systems, but once you have, it is a very natural way of preparing texts. Tex is really only hard when you want fine control over output, and then all systems are hard. The semantics of macros are often complained about and they are tricky, but it is not easy to find a computational mechanism that fits so smoothly into text.

So there are alternatives, and the Tex family has deep and distinctive strengths.

31

The alternatives to TeX exist: they are WYSIWYG word processors and desktop publishing programs. And in most commercial areas they are adopted far more widely.

TeX is still used in many academic and other fields because

  • its output is beautiful
  • it's free and open source
  • it's gained considerable inertia in the 30 years since it's been in use, being part of the academic tradition

I'm not sure what you mean by a lack of "serious programming opportunities". Functions and data structures are implemented as macros, yes, but I think that most of the power TeX users would call themselves programmers.

You might find some of these questions helpful in your quest for understanding the TeX way:

And as for putting section headings in Helvetica, it isn't that bad, but you do have to understand a few things about LaTeX and its history. TeX predates PostScript, PDF, TrueType, and most electronic typesetting standards (as well as Unicode which is a whole new set of issues). So its font mechanism is a little archaic. This has been dramatically improved with the XeTeX engine, which works natively with TrueType fonts and Unicode.

Here is a sample file, which is similar to the solution that you linked to in that it uses the sectsty package. This package is probably on your local TeX distribution anyway; no need to download or install. Unlike the example you linked to you have to run this through xelatex instead of pdflatex. Again, that executable is also probably in your local distribution too. The advantage is the the font specification is more sane.

\documentclass{article}
\title{Helvectica section headings}
% TEX.SE \url{}
\usepackage{lipsum}

\usepackage{fontspec}
\newfontfamily{\phv}{Helvetica}
\usepackage{sectsty}
\allsectionsfont{\phv}

\begin{document}
\maketitle

\section{First section}

\lipsum[1-3]

\section{Second section}

\lipsum[4-5]

\end{document}
16
  • 4
    "Functions and data structures are implemented as macros, yes" -- I guess an important part of my question is, why? Is this a design choice? Is it possible to write real code purely in macros without it being incomprehensible? Mar 14, 2011 at 12:39
  • 3
    @eegg: You simply don't get functions in a macro language, in general. There are some areas where this can be done (see for example tex.stackexchange.com/questions/11425), but this is not general. On the 'incomprehensible' question, I feel we do a good job with expl3, while others do a good job without it. However, if you're expecting function-like approaches then your bound to be confused: TeX simply does not work that way.
    – Joseph Wright
    Mar 14, 2011 at 12:49
  • 3
    @dmckee, Yes, but surely eegg is right: the fact that it's a macro language makes it impossible to write good, comprehensible code. Isn't it just poor design?
    – Neil G
    Mar 14, 2011 at 19:01
  • 3
    @Neil G: and you don't need any of this to typeset a document, no typesetting system does provide such capabilities, no markup language does. TeX has macros as a powerful way to aid in typesetting your documents, period. People do use it for more than what it was really for, but it is hardly a design fault. The same happens with m4 macro language for example. Mar 14, 2011 at 21:40
  • 5
    Re: the first sentence, "The alternatives to TeX exist: they are WYSIWYG word processors and desktop publishing programs": The question was asking for alternatives "within the problem domain of programmatic document formatting," so this isn't answering the question. WYSIWYG wp's and desktop publishing programs don't provide programmatic document formatting.
    – LarsH
    May 31, 2013 at 15:13
26

TeX is still used, simply because there is no alternative. It is far from being perfect, and there is at least a grain of truth in most, if not all of your claims. But these are, at the end of the day, unimportant.

If you need a document which requires even the slightest programmatic bit, or any level of sophistication, LaTeX, though not-ideal, is simply the best, and is likely to stay so for many more years.

I believe most users (maybe not so in this site) just turn a blind eye to the not so elegant features, and produce amazing documents in no time. The very friendly bunch here does not see these not-so-elegant portions as being so annoying. To most, these are just riddles to enjoy, just like cryptic crossword puzzle.

You do not really have to put your mind to the tough/complicated/riddle/annoying basics. You would better ignore these, at least as a starter. Read the "not so short guide", and be happy with it. If you need something extra, you just go and find a package that does that, and the quest is usually quite easy.

The difficult part is in programming something new, something that cannot be found in existing package. There are experts of this dark art: the rest of us just watch them do their wizardry with awe. Luckily, the cases where you really need to do this kind of programming are very rare: chances are that you will find it was done.

23

Presupposition failure.

First, people are trying to make TeX better for example the LaTeX3 and LuaTeX projects. (These aren't quite rebuilding LaTeX from scratch, but they're close)

Second, there are alternatives, like DocBook. In fact, many markup language projects could be seen in this light. What is HTML doing if it is not creating documents programmatically?

A little less facetiously, why is TeX still popular? There's a certain amount of "if it ain't broke...": A lot of people know how to use TeX and any new project would have a hard time convincing anyone to use it.

I think your general question could be ask of any big code project. Why is PHP the way it is? Why does HTML work like so? Why don't we just throw it all away and start from scratch? Because that is rarely a good idea.

6
  • 2
    My understanding of LuaTeX is that it's just embedding Lua into TeX. HTML (and currently DocBook IMO) have little understanding of physical documents. As for starting from scratch, I see it being done all the time (hence the hundreds (thousands?) of programming languages around); I wasn't really asking whether this is a "good idea", but why it isn't done. Mar 14, 2011 at 12:31
  • 5
    One thing I can see that fits the description of "TeX from scratch" is Lout. It seems pleasurable to work with (being more like a Real Language), and it surprises me that its userbase is so impressively small. Mar 14, 2011 at 12:36
  • 2
    The long-term LaTeX3 goal is to reimplement all of LaTeX2e. However, this is surprisingly complex (there are many unsolved problems).
    – Joseph Wright
    Mar 14, 2011 at 12:41
  • 1
    @eegg: See also ANT (tex.ac.uk/cgi-bin/texfaq2html?label=ant), which does seem to have run into the ground but is I guess somewhat in the area you are interest in.
    – Joseph Wright
    Mar 14, 2011 at 12:51
  • 1
    @eegg: LuaTeX is ... just embedding Lua into TeX - Just this is not a small thing: it has profound consequences for how one can programmatically generate documents, with much of the innards of the backend made available. Luatex also standardises on Unicode and introduces microtypography that is equal to that in Adobe's top-end products and better than everything else, among several other new features. Context Mk4 makes much use of Luatex to do what it does. Context Mk4 is well suited to typesetting Docbook manuscripts. Mar 15, 2011 at 10:15
16

From my layman's perspective, you could get somewhat far (?) by implementing the Knuth-Plass line-breaking algorithm along with the Liang's hyphenation algorithm in your general-purpose programming language of choice as an encapsulated DSL for typesetting. You'd almost certainly want to implement the math material interpretation in a similar way to TeX (compare to XML dialects, for instance, where it would be too tedious to write math by hand).

But what surprises me, as a relatively newcomer to TeX, is the number of questions posted to this site which start off with the expectation of TeX being a general-purpose programming language. I don't think that is a good approach at all.

TeX is a typesetting system, first and foremost!

14

I guess my overall answer would be 'critical mass'. Others have pointed out that most typesetting is actually done using other tools, and that TeX is mainly used in certain area. Within those areas, there is a lot of experience of how to get the results people want. Thus any new approach needs to supersede what can already be done, which is not easy. From my own experience with LaTeX3, I'd say that change even within the LaTeX world is a tough challenge.

14

You should have a look to LuaTeX :

  • Inclusion of Lua as an embedded scripting language;
  • Native support of Unicode;
  • Supporting modern font formats such as TrueType and OpenType;
  • And more.
1
  • Personally, I'm not really a TeXy, but my company has a product called DocScape which implements a data-driven publishing tool on top of Tex which I thought might be of interest to you or others reading this.
    – fforw
    Mar 15, 2011 at 8:39

Not the answer you're looking for? Browse other questions tagged .