LaTeX3: Elegant way to forward-reference a counter with a "future" value

Question

I'd like to do, pseudo-code, the following:

\begin{pointtracker}{Advanced Geology}
  \begin{itemize}
    \item Count the fingers on your left hand. Are there more than 9? \addpoints{1}
    \item Draw a rectangle with five edges. \addtopoints{3}
  \end{itemize}
\end{pointtracker}

To produce

---

Advanced Geology: 4 points

• Count the fingers on your left hand. Are there more than 9? (1 point)
• Draw a rectangle with five edges (3 points)

---

The xcntperchap package (which sadly is broken on current Ubuntu) achieves that by writing counters into an external file (using \iow_write) and retrieving them from there. It uses LaTeX structure levels as logical layers in which to keep and appropriately reset counts. It requires two LaTeX runs whenever a point changes.

I could definitely adapt that to the use case, but as I then still have to implement a workaround for xcntperchap to work on my colleagues' computers running Ubuntu 20.10, I might as well implement something less versatile myself (I thought), in an effort to teach myself a bit of LaTeX3. (Which, so far, is going well.)

So, is it really necessary to write things to an external file to later "forward reference" then in a second run?

Could I, using expl3 or other modern tools, for example, expand the content of a function, thereby executeíng all \addpoints functions ("function" in the LaTeX3 sense), and ending up with the total sum of points before I actually put the expansion in the code. Thereby, avoiding having to run things twice?

Answer 1

You can absorb the contents of the environment and process it twice, once in a box that will be discarded, giving \addtopoints a different meaning, namely, to add points.

Next you have the total available and you can print the whole thing.

\documentclass{article}
%\usepackage{xparse} % not needed for LaTeX 2020-10-01 or later

\ExplSyntaxOn

\NewDocumentEnvironment{pointtracker}{m+b}
 {
  % typeset the body in a box that's then discarded
  \marcus_points_count:n { #2 }
  \subsubsection*{#1: ~ \int_eval:n { \g__marcus_points_int } ~ points }
  #2
 }{}
\NewDocumentCommand{\addtopoints}{m}
 {
  \__marcus_points_print:n { #1 }
 }

\box_new:N \l__marcus_points_box
\int_new:N \g__marcus_points_int

\cs_new_protected:Nn \__marcus_points_print:n { (#1\nobreakspace points) }
\cs_new_protected:Nn \__marcus_points_add:n { \int_gadd:Nn \g__marcus_points_int { #1 } }

\cs_new_protected:Nn \marcus_points_count:n
 {
  \int_gzero:N \g__marcus_points_int
  \vbox_set:Nn \l__marcus_points_box
   {
    % the box that will be discarded, here \addtopoints just adds
    \cs_set_eq:NN \addtopoints \__marcus_points_add:n
    #1
   }
 }

\ExplSyntaxOff

\begin{document}

\begin{pointtracker}{Advanced Geology}
  \begin{itemize}
    \item Count the fingers on your left hand. Are there more than 9? \addtopoints{1}
    \item Draw a rectangle with five edges. \addtopoints{3}
  \end{itemize}
\end{pointtracker}

\end{document}

Alternatively, you can exploit the \label-\ref mechanism

\documentclass{article}
%\usepackage{xparse} % not needed for LaTeX 2020-10-01 or later

\ExplSyntaxOn

\NewDocumentEnvironment{pointtracker}{m}
 {
  \int_gincr:N \g__marcus_points_env_int
  \int_gzero:N \g__marcus_points_int
  \subsubsection*{ #1: ~ \ref{ points@\int_eval:n { \g__marcus_points_env_int } } ~ points }
 }
 {
  \tl_set:cx { @currentlabel } { \int_eval:n { \g__marcus_points_int } }
  \label { points@\int_eval:n { \g__marcus_points_env_int } }
 }

\NewDocumentCommand{\addtopoints}{m}
 {
  (#1\nobreakspace points)
  \int_gadd:Nn \g__marcus_points_int { #1 }
 }

\int_new:N \g__marcus_points_env_int
\int_new:N \g__marcus_points_int

\ExplSyntaxOff

\begin{document}

\begin{pointtracker}{Advanced Geology}
  \begin{itemize}
    \item Count the fingers on your left hand. Are there more than 9? \addtopoints{1}
    \item Draw a rectangle with five edges. \addtopoints{3}
  \end{itemize}
\end{pointtracker}

\end{document}

Each pointtracker environment steps an internal counter that's used for defining a unique label. The total points are stored in \@currentlabel and this will be what \ref will use at the next LaTeX run. Of course this might require several runs to stabilize, if you add environments between existing ones.

Answer 2

EDITED to be able to calculate global points, as well, by nesting the pointtracker environments. This question has inspired future improvements to the tokcycle package. See SUPPLEMENT.

The tokcycle package provides an elegant way to accomplish this. It passes through all the tokens of its input and accepts directives on how to treat them. By default, it echoes them to the \cytoks token list, which is then typeset at the end of the environment.

It breaks tokens into 4 categories (Characters, Groups, Macros, and Spaces) and permits a different directive for each category. Since here, we need to do something when an \addtopoints is encountered, we use the \Macrodirective to accomplish it.

In the pointracker environment, tokens are being echoed to the \cytoks token list as they are encountered. However, if it comes across an \addtopoints token, it additionally calls on the magic macro \z (hat tip to David Carlisle). From having written the package, I know that the next tokens in the input stream will be \@tokcycle <continuation of input stream>, which is telling the package to continue processing the input stream through the token cycle (we know the next thing in the input stream will be the argument to \addtopoints, that is, {<number>}).

What \z does is rearrange the input stream so that \@tokcycle{2} becomes \addtocounter{pointcount}{2}\@tokcycle{2}, accomplishing our desired goal of counting the points during the token cycle, prior to the tokens being typeset. When it finally gets around to typesetting \the\cytoks, they will be the exact tokens of the input stream, but with foreknowledge of \thepointcount.

I have now added an option to the environment for specifying the label applied to the number (default points). This will allow, in nested environments, the use of total points as the optional argument in the outer environment. One may also apply a size modifier to the mandatory argument, perhaps making the outer environment larger than the default \Large.

I again emphasize that only one LaTeX compilation has occurred here. However, each pointtracker environment cycles through the tokens of its environment looking for point allocations before typesetting, which one may think of as a form of double pass.

\documentclass{article}
\usepackage{tokcycle,environ}
\newcounter{pointcount}
\newcommand\addtopoints[1]{(\textit{#1 point}%
  \ifnum#1=1\relax\else\textit{s}\fi)}
\def\z#1#2{\addtocounter{pointcount}{#2}#1{#2}}
\NewEnviron{pointtracker}[2][points]{%
  \par\bigskip\resettokcycle
  \setcounter{pointcount}{0}%
  \Macrodirective{\addcytoks{##1}\tctestifx{\addtopoints##1}{\z}{}}%
  \def\tmp{\tokencyclexpress{\Large\bfseries #2: \thepointcount{} #1}}%
  \expandafter\tmp\BODY\endtokencyclexpress
}
\begin{document}
\begin{pointtracker}[total points]{\LARGE Science Test}
\begin{pointtracker}{Advanced Geology}
  \begin{itemize}
    \item Count the fingers on your left hand. Are there more than 9?
          \addtopoints{1}
    \item Draw a rectangle with five edges. \addtopoints{3}
  \end{itemize}
\end{pointtracker}
\bigskip
In this next phase of the test, feel free to use your calculator.
\begin{pointtracker}{Meteorology}
  \begin{itemize}
    \item How many borgs does it take to brew a Danish beer?
          \addtopoints{2}
    \item What is the meaning of life? \addtopoints{4}
  \end{itemize}
\end{pointtracker}
\end{pointtracker}
\end{document}

---

SUPPLEMENT

The answer above got me thinking on the utility of building some sort of look-ahead feature into the directives employed by tokcycle. For example, in this problem, without look-ahead, I would be forced to define and set a flag when when encountering \addtopoints in the Macro-directive and then keep an eye out in the Group-directive for the flag, so as to read the argument which follows, containing the numbers I wish to tally. And then reset the flag. The method works, but can be cumbersome.

The \z trick I used above was based on my personal knowledge of the guts of tokcycle, which clearly, typical users do not have. Wouldn't it be preferable and useful to be able to define a programming layer to accomplish this sort of task? Maybe with a syntax like this:

\def\z{\tcpop\Q\addtocounter{pointcount}{\Q}\tcpushgroup{\Q}}

In essence, pop an argument from the input stream, add the number to my tally, and then push the argument back onto the input stream (with braces). Here, \Q is just a local macro name that the user can choose.

With the introduction of tokcycle[2021-05-27] V1.4, the above syntax became a reality. Best of all, the new \z can be invoked anywhere in the directive, not just "at the end", as was required in my hard-wired approach above.

Here is a summary of what is offered new in v1.4:

While the normal processing of tokens in a token cycle gives very detailed information about the implicit/explicit/active nature of the tokens, the look-ahead features described below are not nearly as exhaustive in their discernment. They are intended to be used when one already has an idea of what kind of tokens are in the immediate future of the input stream.

In the following descriptions, \zz is a representative macro token whose actual name is selected by the user.

1. \tcpeek\zz - \futurelets the next token of the future input stream* into \zz, future input stream remains undisturbed.

2. \tcpop\zz - absorbs one argument** from the future input stream placing that argument as the replacement text of \zz.

3. \tcpopliteral\zz - like \tcpop, this absorbs an argument from the input stream. However, in this case, leading whitespace and any brace grouping is preserved in \zz, so that \zz contains the literal tokens that came off the input stream.

4. \tcpopto<tok>\zz - removes tokens from the future input stream up to and including the occurrence of <tok>, in the fashion of \def\zz#1<tok><input stream>. All removed tokens***, including the terminating <tok>, are \defed into \zz.

5. \tcpush\zz - places the replacement text of \zz as the next element of the input stream.

6. \tcpushgroup\zz - acts like \tcpush, except the replacement text of \zz is encased in an explicit brace group.

7. \tcpopwhitespace\zz - to peek beyond the white space at the running head of the input stream, without absorbing what follows, one may use this macro to absorb the white space (explicit continuous spaces signifying one explicit space token). At that point, what follows can be probed with a \tcpeek. \zz will contain a space if white space was absorbed or empty otherwise. Implicit spaces will not be absorbed by this macro.

In addition, \tcappto#1from#2 allows the replacement text of #2 to be appended to the replacement text of #1. This macro has also been combined with both popping forms as \tcpopappto and \tcpopliteralappto in which the from#2 is taken as the input stream, and the popped tokens are appended to the replacement text of the provided macro.

The above macros are to be used within the Character, Macro, and Space directives (not the Group directive). They can assist in previewing an argument when a particular macro is encountered, determining whether a space is the next token in the input stream, performing in the context of one directive an operation that spans over several tokens of input. I'm sure users will think of many more uses.

In general, use peeked tokens for making decisions, but do not output peeked tokens to \cytoks, as the token used in the directive will be reassigned each time that directive is called upon. When \cytoks is eventually typeset, only the final assignment remains.

Example of \tcpopto, to absorb, now, a future optional argument into \B, including the brackets: \tcpeek\Q\ifx[\Q\tcpopto]\B\fi

Note: if popped look-ahead token(s) need to be saved to the \cytoks output stream, it will require one expansion, since you require the replacement text of the popped macro. Thus, \addcytoks[1]{\zz}. If the \tcpop'ed look-ahead tokens were part of a group (i.e., if the immediately prior \tcpeek reveals an \ifx equivalence with \bgroup), then there are two ways to retain the grouping within \cytoks: \groupedcytoks{\addcytoks[1]{\zz}} OR \addcytoks[1]{\expandafter{\zz}}. Another alternative is to \tcpush or \tcpushgroup them back into the input stream to be handled afresh by the appropriate directive. Better still, use \tcpopliteral to extract them from the input stream with their grouping (and leading space) intact.

*When the input stream would otherwise exhibit, as the next token, the explicit cat-2 brace associated with the end of the current \tcdepth grouping, a \tcpeek\zz will instead reveal \ifx\zz\empty as true (this is a tokcycle group-protection mechanism).

**when \tcpoping, \ifspacepopped will reflect the occurrence of a leading cat-10 token in the input stream (explicit or implicit); however, blank (explicit) leading spaces (aka white space) will be lost as the argument is absorbed, in the normal fashion of TeX. In contrast, implicit spaces will be absorbed as arguments, also in the manner of TeX. The \tcpop macro will not penetrate end-of-group } or tokcycle "escape character" |, instead returning an empty result.

***As in TeX, \tcpopto will break if group absorption is unbalanced.