Skip to main content
The 2024 Developer Survey results are live! See the results

Questions tagged [token-lists]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
1 answer
39 views

Specification of Token Lists

Are Token Lists specific to the expl3 layer, or are they more general entities within latex ?
Bulldrive's user avatar
  • 161
5 votes
2 answers
146 views

Latex3: Passing empty token list to another command and detecting it as empty token list

I am sorting some tokens in a token list using the nice features of latex3 (see below command \nom). These tokens can be sometimes empty, and when they are empty they would trigger different ...
wdsgn's user avatar
  • 311
5 votes
1 answer
144 views

Understanding expansion in tblr environment

I'm trying to understand how tokens and expansion work in the context of the following problem. I'd like to create a tblr environment from the tabularray package via a command that does some of the ...
Yossarian's user avatar
2 votes
1 answer
96 views

Replacing placeholder token in token list

I'm trying to write some code that replaces a couple of placeholder tokens in a token list, but am running up against a wall since I am still very new to latex3. Currently, I get no result and no ...
meide's user avatar
  • 1,175
6 votes
2 answers
231 views

Using expl3 to load geometry parameters from a file

expl3 has defeated me again so I'm relying on the kindness of strangers again. I think the underlying problem is more about splitting input rather than the file reading. Context: I want to load the ...
Doc Octal's user avatar
  • 369
7 votes
2 answers
227 views

Expansion issues again

I seem to keep encountering the same issue. In my attempt to learn more about expansions I have written a program that starts with a sequence of 1s and then after each step increments it to generate a ...
Ted Black's user avatar
  • 495
5 votes
2 answers
510 views

Exercise 8.6 of The TeXbook

Exercise 8.6. Assume that the category codes of plain TEX are in force, except that the characters ^^A, ^^B, ^^C, ^^M belong respectively to categories 0, 7, 10, and 11. What tokens are produced from ...
Atom's user avatar
  • 665
3 votes
2 answers
261 views

LaTeX3 how to use content/value of predefined command in token list/string?

First of all, I guess that will be a very easy answer for people who are experienced in expl3. But for me its new and I'm very willing to learn that stuff to understand Latex3 better. So I hope for ...
lukeflo's user avatar
  • 1,597
0 votes
2 answers
47 views

Token list given to TikZ scope fails

I want to construct a chain list on the fly and then use it in a tikzpicture, something like this: \documentclass{article} \usepackage{tikz} \usepackage{xstring} \usetikzlibrary{chains} \newtoks\...
sgmoye's user avatar
  • 8,941
1 vote
0 answers
62 views

Compare Token List result to a string

I have a command that (beyond my control) evaluate into a token list, and I want to compare it to a given string, I tried to follow this answer and create a variant on \tl_if_eq: \ExplSyntaxOn \...
Yuval Shmuel's user avatar
3 votes
1 answer
86 views

Token read/write turns # into ##

When I try to write an input given by a user in a token list to a file, it turns # into ##, so if a users wants to write \def\hello#1{Hello #1!}, instead of writing: \def\hello#1{Hello #1!} as ...
tobiasBora's user avatar
  • 9,004
0 votes
1 answer
44 views

Manipulating a list and then passing it as an optional argument

I want to remove one key given by #1 and then pass the modified list to \lstinline[<modified list>]$#2$. I am able to remove the element but don't know the syntax of passing the rest to \...
M. Al Jumaily's user avatar
3 votes
1 answer
189 views

Automatic listing of authors in a document

I have a command \by{ } that lets me list authors of an article like this: \by{ {author 1}{tagline} \\ {author 2}{tagline} \\ {author 3}{tagline} \\ <etc.> } I would like to store author 1, ...
Vebjorn's user avatar
  • 1,884
0 votes
1 answer
65 views

How to stop converting ^^M into space?

Is it possible to make ^^M visible in the list of tokens? Now it just gets converted to space, but I need it to be visible in tokens. This is the code: \documentclass{article} \usepackage{iexec} \...
yegor256's user avatar
  • 12.2k
7 votes
2 answers
271 views

How to pass the the content from Lua function to a LaTeX3 token list (tl)?

I have a Lua function that generates LaTeX content (a string containing LaTeX commands like "\\textbf{foo}") and it can be successfully outputted with \lua_now:n and tex.print(). Now I'm ...
Zeping Lee's user avatar
2 votes
2 answers
50 views

Replacing top-level (i.e. not enclosed in braces) occurrences of a token

I'm trying to replace all top-level occurrences of a token by another in a token list. For example, replacing 0 by 1 in 0{0{0}}0 should yield 1{0{0}}1. I have something that kind of works, but forgets ...
xavierm02's user avatar
  • 787
3 votes
1 answer
50 views

Styling character depending on its position

I'm trying to write a funny style writing targeting a young audience. At some point, I would like some words to have "wave" effect on each character according to their position. I've already ...
Miraino Hikari's user avatar
1 vote
2 answers
307 views

LaTeX3 convert token list to string

I want to create a LaTeX class with optional parameters using LaTeX3. But I'm having problems converting the token list holding the parameter value to a string. I would like to know how to do this ...
Doc Octal's user avatar
  • 369
2 votes
1 answer
241 views

Why is LaTeX3 method not working with token list argument, contrary to documentation?

I have the following code in which I try to check whether a letter exists in a string: \documentclass{article} \ExplSyntaxOn \begin{document} \str_gset:Nn \ABCstring {ABC} % Setting the test string ...
AplusKminus's user avatar
0 votes
0 answers
70 views

tex.runtoks: uniqueness of the <token register>

tex.runtoks can be used to execute instructions immediately. \directlua{ tex.settoks(421,'ABCDE'); tex.runtoks(421) } Here 421 should be replaced by a unique private number, in LaTeX we could ...
Jérôme LAURENS's user avatar
2 votes
0 answers
77 views

Can you map arbitrary token sequences expandably and unambiguously to numbers or to strings of explicit character tokens of category 12?

How to map arbitrary brace-balanced sequences of non-outer tokens expandably and unambiguously to numbers or to strings consisting exclusively of explicit character tokens of category 12, if possible ...
Ulrich Diez's user avatar
  • 29.8k
2 votes
1 answer
158 views

Passing a token list variable as an argument

The code below parses out a comma-delimited bit of an argument, then passes that bit to a function, as a token list variable. It works, by using \exp_args:Neee, but I suspect that there's a better way....
Randall Fairman's user avatar
3 votes
3 answers
123 views

No space no caps tokenlist

This took me longer than I'd like to accomplish what should have been a very simple task. I needed a way to turn a tokenlist lowercase and remove the spaces. But str_lowercase:N doesn't exist! And ...
Bob's user avatar
  • 1,322
0 votes
1 answer
56 views

Expansion while processing math in lists with xparse

I'm trying to "print" a comma-separated list of strings. The actual use case is more complex, the list being stored as a key-value, but the MWE below shows the problem: when I add math to ...
steve's user avatar
  • 2,256
3 votes
4 answers
185 views

Highlighting token groups inside an expression

What I want to do is replace {...} (token group) by \{...\} (curly braces) at any level of nesting inside an expression. I'm only able to do it for level = 0. There should be an extra curly brace ...
Erwann's user avatar
  • 2,130
9 votes
1 answer
1k views

LaTeX3 - Check if an argument is empty

What is the best way to translate the following code into expl3? \documentclass[12pt]{article} \newcommand\test[1]{% \if\relax\detokenize{#1}\relax EMPTY\else FILLED\fi% } \begin{document} \test{...
projetmbc's user avatar
  • 13.4k
0 votes
0 answers
56 views

Apply \cs_if_exist to the head of a tl variable

Consider the following command definition: \NewDocumentCommand{\reportexpansion}{ m }{ \bgroup \tl_set:Nn \l_tmpa_tl { #1 } { \ttfamily \tl_to_str:V \l_tmpa_tl } \space is \space \exp_args:No \...
zwol's user avatar
  • 2,981
4 votes
3 answers
844 views

The puzzling thing about \newtoks and the token list

The code is: \documentclass{article} \newtoks \test \begin{document} \test={123} \test=\test{666} \test=\test{hhh} \the\test \end{document} and the result is: 666 hhh 123, as ...
xxllxx666's user avatar
  • 143
1 vote
2 answers
386 views

How to find unicode characters represented via \string command

I'm using PDFLaTex compiler and want to produce PDF/A-1b format file. I'm using the package \usepackage[utf8]{inputenc}. I'm having problem with the unicode encoding for PDF meta-data. I want to use ...
user2340939's user avatar
3 votes
1 answer
155 views

How do I add the value of a token list with braces in LaTeX?

In my LaTeX3 code, I frequently find myself wanting to add the value of one token list (or other variable) to another but surrounded by braces (a common use case is where I'm constructing a command to ...
Andrew Stacey's user avatar
2 votes
1 answer
76 views

Including references in a tokenlist using expl3

I am trying to create a document with math exercises, for each math exercise one or more hints are given that should be show in a later section. From the inserted hints it should be possible to jump ...
midtiby's user avatar
  • 1,863
4 votes
2 answers
687 views

expl3 - how can you pass the content of a token-list-variable as argument to another "function"?

I looked at interface3.pdf for learning about expl31. Assume one uses the scratch-token-list-variable \l_tmpa_tl. How can you pass the content of this variable as argument to another "function&...
Marsupial Wallaby's user avatar
1 vote
1 answer
33 views

I want to have access inside the body not to the \arg[n] list but with \A[n] if I call \MAT{A, 1, 2, 3, 4}?

I would like to set matrices using the code: \newcommand{\MAT}[1]{ \setsepchar{,} \readlist\arg{#1} \arg[1]= \begin{pmatrix} \arg[2]& \arg[3] \\ \arg[4]& \arg[5] \end{pmatrix} } and inside ...
Lindomar Carvalho's user avatar
0 votes
1 answer
140 views

LaTeX3: Get a token list var via its name

Assume I have a function that takes a tl var as an argument: \module_myfunc:N \l_module_tlvar_tl But what I have at hand is not the tl var but the name of the tl var: \l_module_tlvarname_tl How do I ...
Cyker's user avatar
  • 667
4 votes
1 answer
268 views

LaTeX3: Braces and spaces

I don't quite understand how braces and spaces are treated in LaTeX3. For example: The documentation uses the following convention for N parameters: \my_func:N \my_tl And for n parameters: \my_func:n ...
Cyker's user avatar
  • 667
2 votes
1 answer
61 views

Property reusability

I'm trying to create a command that will allow me to create multiple item for a game. My main problem comes from re-usability of that command. The fact that I have more than 9 arguments for some of ...
Miraino Hikari's user avatar
11 votes
1 answer
485 views

What happened to expl3's case changing functions? (e.g. \tl_upper_case:n and \tl_lower_case:n)

I had some expl3 code which relied on \tl_upper_case:n to do some expandable case changing. However today I updated my TeX distribution and the code now throws an error: ! LaTeX3 Error: '\...
Phelype Oleinik's user avatar
4 votes
2 answers
374 views

In LaTeX3, what is the best data type for storing, parsing and outputting free user input?

I'm working on a mini-parser that takes free user input and interprets certain inputs as commands. For example, the parser interprets + as \oplus or [ as "start a pre-configured array with the bracket ...
Felix Emanuel's user avatar
4 votes
5 answers
430 views

Prevent hash doubling with \message

Assume a token sequence: Token 1: Explicit hash character of category code 6. Token 2: Explicit 1 of category code 12. Token 3: Explicit hash character of category code 6. Token 4: Control word token ...
Jewdokija's user avatar
4 votes
3 answers
567 views

How to iterate through a token list to make characters uppercase, while preserving spaces?

The following (classical, I assume) "iterator" definition would be examined here: \def\zEnd{\zEnd} \def\zzIterator#1{% \ifx#1\zEnd \else #1% \expandafter\zzIterator \fi } \...
bp2017's user avatar
  • 3,796
27 votes
3 answers
1k views

Unbalanced dollar signs

I'm curious: why do the following two examples behave differently? Example 1: (Works, displays xyz in displaymath mode.) \newcommand\weird{$$xyz$} \weird$ Example 2: (Error, missing $ inserted.) \...
fmg's user avatar
  • 477
20 votes
2 answers
788 views

Expandable test for an empty token list—methods, performance, and robustness

With ε-TeX, the go-to method for testing if a <token-list> is empty is the following test: \if\relax\detokenize{<token-list>}\relax % empty \else % not empty \fi The method ...
Phelype Oleinik's user avatar
6 votes
2 answers
290 views

How to use \cs_generate_variant:Nn to compare token lists?

I want to deal with this question by my method. The example below is some codes from interface3.pdf; But throws the error:  ! LaTeX3 Error: Variant form 'VV' deprecated for base form '\...
Brooks's user avatar
  • 573
10 votes
2 answers
537 views

When to use \tl_to_str:V and when \tl_to_str:N?

According to the interface3 documentation \tl_to_str:N expects a “tl var”, which I believe might be e.g. \g_temp_tl. \tl_to_str:V should, if I got it right, expect an argument representing a control ...
Robert Siemer's user avatar
4 votes
2 answers
369 views

What is the difference between `put` and `gput` in `\tl_new:N` token lists?

Why is there one function to define (globally) e.g. a token list \tl_new:N, but there are two functions to put an element into it: \tl_put_left:Nn and \tl_gput_left:nN What does this g at the ...
Robert Siemer's user avatar
2 votes
3 answers
1k views

How to check if the argument to a macro is a specific string?

I need to be able to pass [ ] (brackets with space in between) to a \def and check whether the \def's parameter contains [ ] (rather than something else). \begin{document} \def\bptoks{[ ]} \def\bpdef#...
bp2017's user avatar
  • 3,796
7 votes
1 answer
2k views

what \relax command means?

I'm trying to compile this .tex file from ArXiVe.org in WinEdt 9.0 but some tokens are underlined with a solid line and preceeded by a strange command \relax command in WVP.aux: \relax \citation{...
user2925716's user avatar
  • 1,970
4 votes
3 answers
191 views

fully expand toks and save for later (appendix with exact copies of tables from the main text)

I have a document with some complex tables, defined by a command called \Rule with named parameters (using the keyval package). The individual rows of the table are first stored into a token list ...
Ansa211's user avatar
  • 343
10 votes
2 answers
232 views

Understanding Implicit Delimiters/Terminators

[I encourage you to check out Phelype's impressive approach] I hear about macros that do things like "expand until they reach an unexpandable token" and this question may (or may not) be related to ...
Steven B. Segletes's user avatar
2 votes
1 answer
144 views

How can I strip the percent char of a number read by csvsimple?

I have a CSV table containing some numbers in percent with the percent char explictily written in the CSV data (that can’t be changes). Is there a (LaTeX3) way to strip the percent char and preceding ...
Tobi's user avatar
  • 56.9k