In my opinion, l3 strings are special token lists whose where all characters has category code 12 (except space), and most interfaces in l3tl and l3str are similar. So, if I just want to process "plain" text, what exactly is their difference?

For example, should I use token list or string for option names, cross-reference labels? What is the best practice?

  • 1
    What do you mean by plain text? Verbatim material or text in the document? For option names I would use token lists (because I want my users to use macros too), same for labels. – TeXnician Aug 15 '18 at 11:53

l3 strings are for low level tex programming constructs not for natural language texts, note in particular that in pdftex you can not have accented (or any non-ascii) characters in a string as they require active characters and inputenc processing.

  • 3
    Perhaps add a simple 'Do X or Y' statement: I'd go with 'Strings are for data that will never be typeset, for example file names, identifiers, etc.: if the material may be used in typesetting, it should be a token list'. – Joseph Wright Aug 17 '18 at 7:36
  • Is there an equivalent in l3kernel to token lists in expl3? Defining one as NewDocumentCommand\tl_foo{}{token list}, that is with no argument, is cumbersome. – Erwann Oct 4 '19 at 21:31

The interface documentation itself is not entirely clear about that.

Part VII The l3str package: Strings, p. 49

TeX associates each character with a category code: as such, there is no concept of a “string” as commonly understood in many other programming languages. However, there are places where we wish to manipulate token lists while in some sense “ignoring” category codes: this is done by treating token lists as strings in a TeX sense.

A TeX string (and thus an expl3 string) is a series of characters which have category code 12 (“other”) with the exception of space characters which have category code 10 (“space”). Thus at a technical level, a TeX string is a token list with the appropriate category codes. In this documentation, these are simply referred to as strings.

String variables are simply specialised token lists, but by convention should be named with the suffix ...str. Such variables should contain characters with category code 12 (other), except spaces, which have category code 10 (blank space). All the functions in this module which accept a token list argument first convert it to a string using \tl_to_str:n for internal processing, and do not treat a token list or the corresponding string representation differently.

As string is a subset of the more general token list, it is sometimes unclear when one should be used over the other. Use a string variable for data that isn’t primarily intended for typesetting and for which a level of protection from unwanted expansion is suitable. This data type simplifies comparison of variables since there are no concerns about expansion of their contents.

screenshot of the above text from the PDF manual with "it is sometimes unclear when one should be used over the other" highlighted

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.