Questions tagged [token-lists]

The tag has no usage guidance.

8
votes
2answers
117 views

Understanding Implicit Delimiters/Terminators

[I encourage you to check out Phelype's impressive approach] I hear about macros that do things like "expand until they reach an unexpandable token" and this question may (or may not) be related to ...
2
votes
1answer
32 views

How can I strip the percent char of a number read by csvsimple?

I have a CSV table containing some numbers in percent with the percent char explictily written in the CSV data (that can’t be changes). Is there a (LaTeX3) way to strip the percent char and preceding ...
3
votes
3answers
113 views

Expl3' tl variable storing the result of a \seq_map_inline: OK in article's title but not in beamer's title

As pointed out by the following MCE, an expl3' token list variable storing the result of a \seq_map_inline: used in \title's argument: works like a charm with the article class, doesn't work with the ...
2
votes
2answers
154 views

How to properly store the current value of int variable into a token list?

I have an volatile integer variable (counter) and at some point I wish to save its value to a token list (saved_counter). If I save it with just \tl_set, as soon as counter changes the value, this ...
3
votes
1answer
29 views

Lines (rules) and \multicolumn in tabular content built with token lists

Building on the MWE in my first ever question here (Extra space in cells when using token lists to build tabular content), I would like to add multicolumn content and \toprule (\midrule, \bottomrule, ...
9
votes
1answer
109 views

Extra space in cells when using token lists to build tabular content

What part of my code is causing the misalignment of the first column? The extra space seems to increase with the number of rows. (Part of my code is from How do I use the ampersand (&) inside a ...
0
votes
0answers
24 views

Customize chapters in table of contents from “1”, “2”, … to “Chapter 1:”, “Chapter 2:”, … using \@starttoc

I need to use this template for my thesis and having troubles modifying the labeling of chapters/sections in the table of contents. The way it is implemented right now is: "1 blah 1.1 blah-blah 1.2 ...
2
votes
1answer
30 views

How to use the token list variable from \prop_get

Here is a MWE. I use a token list, and I can get 10 and October in the final pdf file. \documentclass{article} \usepackage{xparse} \ExplSyntaxOn \tl_new:N \l_my_tl \prop_new:N \l_my_prop \...
4
votes
2answers
118 views

Convert token to string in LaTeX3

While trying to answer another question on this site, I was experimenting with \peek_after:Nw and \l_peek_token. I can set \l_peek_token OK, but then I want to test if this token is one of several ...
3
votes
1answer
51 views

How to pop (right) a token list with expl3?

The expl3 package provides \tl_put_right which enables us to append tokens to the right side of the current content of a token list variable: \documentclass{article} \usepackage{xparse} \begin{...
1
vote
0answers
35 views

Store tokens with unbalanced braces in token register

First of all, this question is not part of a real-world problem, it's just to see whether the following is possible with TeX or not. Consider this short code: \begingroup \toks0={\def\foo<} \...
2
votes
5answers
180 views

How can I condition on the last token?

I would like to write a command \foo which, for the sake of simplicity, does just one thing, which is make sure its argument ends with a dot. Thus both foo{hello world} and foo{hello world.} expand to ...
2
votes
1answer
94 views

expl3 read a comma list from file

I am using expl3 now, I want to read data from file, and store them in a comma list, then use the comma list one by one. I don't have any code, I just know exp13 have these function, so need someone ...
3
votes
1answer
63 views

Latex3: multicolumn at the beginning of a token list produces an error

I'm not sure if using token list is a good approach to generating table data. Anyway, here is a weird result. It works well in all other cases except when the first item in token list is multicolumn. ...
1
vote
1answer
32 views

Generate a list of paragraph and page number for specific labels

I was wondering if anyone could help me with this: I've got this report for which I show the paragraph numbers. The paragraph number is then used as part of the label so I can extract it later on, ...
6
votes
1answer
110 views

How can I implement a \futurelet that ignores spaces?

This is a question about token processing in TeX. What am I doing wrong when implementing \FutureLetNoSpace? The book TEX in Practice: Volume III: Tokens, Macros, by Stephan v. Bechtolsheim, really ...
5
votes
1answer
98 views

expl3 — sort a token list

L A TEX3 comes with a facility to sort list variables (sequences, token lists, or comma-lists) according to some user-defined comparison. I try to adapt the interface3 manuel example to a token list, ...
2
votes
1answer
88 views

Building up and joining “internal” lists

How can I append to a pgfkeys-style comma-separated list? Or, alternatively, how can I define a macro to join an etoolbox list with a delimiter? For an environment I'm designing, I want to accumulate ...
1
vote
1answer
63 views

LaTeX output local lists on root document

This is a followup question for this question, where Christian Hupfer provided me a great solution for a certain problem: I wanted to list the occurrences of special commands within a section at the ...
2
votes
2answers
154 views

Why “Missing Control Sequence Inserted” using Pop Function on Token Register?

When using the \Pop macro from from TeX by Topic chapter "Operations on token lists: stack macros", I get Missing control sequence inserted. Why? I copied it exactly as is! Was there a typographical ...
5
votes
2answers
153 views

Latex3 Token-List commands do not work as expected

Let's take a trivial case: getting the head and tail of a token list and reversing the order: \documentclass{article} \usepackage{xparse} \usepackage{expl3} \ExplSyntaxOn \cs_set:Npn \cs_split:n #...
6
votes
1answer
124 views

How can I define a macro with the value of a fully expanded macro?

Related Expansion of macros and box contents Why isn't a command defined by \newcommand with an optional argument expandable? I made some simple macros for lists, but I do not know how ...
3
votes
2answers
63 views

Applying Macro to xth Symbol of String

I would like to define a command that puts a sequence of symbols above the xth symbol of another sequence of symbols. It seems nicer without a separator, but for higher usability a separator may be ...
1
vote
1answer
192 views

How to create a local token list variable in LaTeX3?

The \tl_new:(Nc) functions create a new token list variable globally. Is there a way to create a new token list variable locally?
2
votes
4answers
253 views

How to handle a LaTeX3 token list as a list of tokens rather than as a list of items?

Is there a way to query and manipulate a LaTeX3 token list as a list of tokens rather than as a list of items? For instance, given the token list {a}b is it possible to find out how many tokens it ...
1
vote
2answers
129 views

Q: Loop over all tokens in the body of a macro

I am looking for TeX-code that loops over all tokens in a macro definition. \meaning transforms everything to "other" (catcode 12) characters and is thus not directly suited (with or without \@strip@...
6
votes
1answer
301 views

What are the constituents of a LaTeX3 token list?

I'm trying to understand what a LaTeX3 token list is. Unfortunately, the LaTeX3 documentation is anything but helpful. According to the LaTeX3 Interfaces documentation token lists can be viewed in two ...
5
votes
1answer
215 views

TeX's expansion rules: a case study with a token-list register

I'm trying to figure out TeX's rules of expansion. Consider the following plain TeX manuscript: \toks1{hello}% \toks10{world}% \def\mac{0}% \the\toks1\noexpand\mac% \bye When this manuscript is ...
1
vote
2answers
111 views

Is there a better method for deferring the execution of commands?

The problem I am interested in getting the following result, by not writing it directly in code. \begin{align} T(x) &= ae^{-\frac{(x-b)^2}{2c^2}} \\ \frac{1}{2} < a &\leq 1 \label{...
0
votes
0answers
46 views

Are using lists to store commands prone to stack exhaustion?

Background Using a handful of answers in this site, I have come up with a solution for building a list of equations (with tocloft), such that two or more of them can be grouped inside an align ...
1
vote
1answer
204 views

'Runaway definition' error when a token list variable is defined inside a macro

The following plain TeX manuscript \def\mymacro{\newtoks\mytoks}\bye fails to compile and generates the following error: Runaway definition? -> ! Forbidden control sequence found while scanning ...
1
vote
1answer
45 views

Assigning to a token list register without a `=` sign

I've noticed that it is possible to assign a value to a token list register without using = (the equality sign) as follows: \newtoks\mytoks% \mytoks{Hello, world!}% as opposed to \mytoks={Hello, ...
4
votes
1answer
96 views

How to create list and assign it to variable?

I need to create a comma (or any other) separated list of words for further usage and assign it to variable (macro). The words in list should be placed in separate lines. When I use for this purposes \...
3
votes
2answers
55 views

Macro for replacing matching tokens in a parameter?

I would like to have a macro, call it \magicreplace which gets some arbitrary parameter (= sequence of tokens) and replaces every occurrence of some token (say, a or x or ,) with some fixed sequence ...
2
votes
1answer
190 views

How are the operations of the TeX tokenizer and engine interleaved?

On p. 38 of Knuth's The TeXbook (1996), a TeX system is broken down into two components: the "mouth" that "chews" the .tex manuscript and converts it to tokens, and the "stomach" that does the actual ...
4
votes
1answer
121 views

TeX's reading rules (text to tokens)

Are TeX's reading rules, which define the conversion from text to tokens, and which are described on pp. 46-48 of Knuth's The TeXbook (1996), describe correctly and completely the reading rules of the ...
7
votes
3answers
210 views

Populating tabular with etoolbox list

I am trying to populate a tabular environment with an etoolbox list: \documentclass{article} \usepackage{etoolbox} \listadd\mylist{Ana}% \listadd\mylist{Bob}% \listadd\mylist{Cole}% \newcommand*{\...
6
votes
2answers
375 views

Can I test line break types? e.g. line endings of type LF or LFCR?

Is it possible to test whether a "line feed" or "line feed + carriage return" were used to end lines in a tex file? I'd like to create a log message with this information. The reasoning for this is ...
2
votes
1answer
69 views

Why does \tl_if_in (and \clist_if_in) insert extra tokens with \mathrm

I encountered a strange behaviour I can’t explain. In my code for a highlighting macro I tried to use \tl_if_in to do a basic math class test. But when \mathrm is part of the tokens to be tested some ...
2
votes
2answers
65 views

Inverted token list in a table

I read a lot of questions and answers about this matter, each doing a little of what I was looking for, but when I tried to merge them together, the whole world collapsed... My goal is to build a ...
1
vote
1answer
75 views

Trying to create dynamic table and commands alltogether

I'm trying to do a command that allow the user to create a specific table for a document but that the person can decide on the fly on the name of the tables (so he can dynamically create multiple ...
2
votes
2answers
118 views

expl3, incrementing a value in a prop-list

I read a clist and create a prop-list which should behave as a hash in which the value for a given key is the number of occurences of this value in the clist. Below my code which, so far, does work as ...
8
votes
2answers
99 views

How to store to-be-credited contributors for acknowledgment later in the appendix

In my thesis, I create numerous drawings using TikZ. In these drawings, I include pictures (e.g. icons of The Noun Project) using Creative Commons licenses. I thus have to acknowledge their authors. ...
15
votes
3answers
735 views

Character bytes and character tokens: If newlines are converted to spaces, then where does catcode 5 come into the picture?

My goal is to understand the relationship between character bytes and character tokens with respect to new line bytes. I likely do not have my facts straight. When TeX reads a file of bytes, encoding ...
14
votes
3answers
688 views

Why does the TeX scanner process tokens for register numbers and macro names differently?

This is a "How does TeX know when to stop" question. Case 1 Note: I have since found out that my understanding here is wrong. See SPOILER and answers. When TeX is constructing a macro name, it ...
2
votes
1answer
125 views

How to define \pgfkeyssetexpandedvalue?

Note: this question follows my previous question about \pgfkeyssetvalue. The \pgfkeyssetvalue is defined in pgfkeys.code.tex file by: \newtoks\pgfkeys@temptoks % [...] \long\def\pgfkeyssetvalue#1#2{...
3
votes
1answer
104 views

Why \pgfkeyssetvalue uses a token list?

The \pgfkeyssetvalue is defined by (in pgfkeys.code.tex file): \long\def\pgfkeyssetvalue#1#2{% \pgfkeys@temptoks{#2}\expandafter\edef\csname pgfk@#1\endcsname{\the\pgfkeys@temptoks}% } where \...
0
votes
1answer
247 views

Beamer boxes with variable sizes and position in frame

I want to to build flexible beamer boxes so that I can change the width and alignment within the frame. Passing the options width= and alignment=, whereas width is a valid value and alignment can be ...
4
votes
1answer
302 views

Obtaining the name of an undefined control sequence in LuaTeX

I want to implement something similar to \detokenize as a Lua function. But there is one case where I am prevented from accomplishing my goal. \directlua{tex.sprint("[" .. token.get_next().csname .. "...
12
votes
2answers
577 views

How are parameter tokens (#1,#2,…,#9) processed?

I wonder how the TeX token muncher processes parameter arguments. It seems that parameter arguments are sent token by token. Is this true? Consider the following code based on wipet's answer used to ...