l3: splitting and randomizing a token list

Question

I am trying to grasp the ins and out of LaTeX3. I want to do the following: I have defined a environment randomizechoices with NewDoucmentEnvironment that grabs the contents. This contents is then parsed. When parsing, I want to split a token list on a token (to a sequence), then randomize the sequence and then build back the token list. This token list is then typeset. Now randomizing a token list is not directly available, but randomizing a sequence is. So I need to convert the token list to a (split) sequence, randomize it and build back the token list (or any way to do it). A MWE is shown below:

\documentclass{exam}

\usepackage{xparse}

\ExplSyntaxOn

% for now
\NewDocumentCommand{\inaccessible}{}{X}

\NewDocumentEnvironment{randomizechoices}{ +!b }
  {
    \tl_set:Nn \l_my_tl {#1}
   
    \tl_replace_all:Nnn \l_my_tl {\CorrectChoice} {\choice \inaccessible}
    \tl_replace_all:Nnn \l_my_tl {\correctchoice} {\choice \inaccessible}

    % shuffle on \choice in token list
    
    % - or -
    
    % Convert to sequence on \choice, shuffle, rebuild
    
    % - or -
    
    % Convert to string, split on \choice to sequence, shuffle, rebuild
    
    %
  \begin{choices}
  \tl_use:N \l_my_tl
  \end{choices}
  }
  {}

\ExplSyntaxOff

\begin{document}

\begin{questions}

\question
Blah
\begin{randomizechoices}
\choice Hello
\choice World!
\CorrectChoice and
\choice everyone!
\end{randomizechoices}

\question
Blah
\begin{randomizechoices}
\choice Hello
\choice World!
\correctchoice and
\choice everyone!
\end{randomizechoices}

\end{questions}

\end{document}

Answer 1

The following does this by using a sequence.

\documentclass{exam}

\usepackage{xparse}

\ExplSyntaxOn

% for now
\NewDocumentCommand{\inaccessible}{}{X}

% initialize the used variables
\tl_new:N \l_my_body_tl
\seq_new:N \l_my_body_seq
\NewDocumentEnvironment{randomizechoices}{ +!b }
  {
    \tl_set:Nn \l_my_body_tl {#1}
    \tl_replace_all:Nnn \l_my_body_tl {\CorrectChoice} {\choice \inaccessible}
    \tl_replace_all:Nnn \l_my_body_tl {\correctchoice} {\choice \inaccessible}
    % set sequence split at \choice
    \seq_set_split:NnV \l_my_body_seq { \choice } \l_my_body_tl
    % remove empty elements
    \seq_remove_all:Nn \l_my_body_seq {}
    \seq_shuffle:N \l_my_body_seq
    % put back randomized sequence with \choice between the items
    \tl_set:Nx \l_my_body_tl
      {
        \exp_not:N \choice
        \seq_use:Nn \l_my_body_seq \choice
      }
  \begin{choices}
    \tl_use:N \l_my_body_tl
  \end{choices}
  }
  {}

\ExplSyntaxOff

\begin{document}

\begin{questions}

\question
Blah
\begin{randomizechoices}
\choice Hello
\choice World!
\CorrectChoice and
\choice everyone!
\end{randomizechoices}

\question
Blah
\begin{randomizechoices}
\choice Hello
\choice World!
\correctchoice and
\choice everyone!
\end{randomizechoices}

\end{questions}

\end{document}

---

This builds on @egreg's fantastic answer but changes some simple aspects of it:

• I changed the marker to be a quark
• I changed the test for said marker to be fully expandable, this way the sequence used as output does only contain the contents once instead of thrice (imho, cleaner code)
• add the possibility to place a label (only makes sense if you set the seed of expl3 to a fixed value)

\documentclass{exam}

\ExplSyntaxOn

\sys_gset_rand_seed:n { \fp_eval:n { round(1e6 * pi) } }

\NewDocumentEnvironment{randomizechoices}{ o +b }
  { \jodb_choices_randomize:nn {#1} {#2} }
  {}

% initialize the used variables
\tl_new:N \l_jodb_choices_body_tl
\seq_new:N \l_jodb_choices_body_in_seq
\seq_new:N \l_jodb_choices_body_out_seq

\quark_new:N \q__jodb_choices_marker

\cs_set_protected:Npn \jodb_choices_randomize:nn #1#2
  {
    \tl_set:Nn \l_jodb_choices_body_tl {#2}
    \tl_replace_all:Nnn \l_jodb_choices_body_tl
      {\CorrectChoice} {\choice \q__jodb_choices_marker}
    \tl_replace_all:Nnn \l_jodb_choices_body_tl
      {\correctchoice} {\choice \q__jodb_choices_marker}
    \seq_set_split:NnV \l_jodb_choices_body_in_seq
      { \choice } \l_jodb_choices_body_tl
    % first item is empty
    \seq_pop_left:NN \l_jodb_choices_body_in_seq \l_tmpa_tl
    % randomize
    \seq_shuffle:N \l_jodb_choices_body_in_seq
    % build another sequence
    \seq_set_map_x:NNn \l_jodb_choices_body_out_seq \l_jodb_choices_body_in_seq
      {
        \tl_if_head_eq_meaning:nNTF { ##1 } \q__jodb_choices_marker
          {
            \exp_not:n { \CorrectChoice }
            \IfValueT {#1} { \exp_not:n { \label{#1} } }
            \exp_not:o { \use_none:n ##1 }
          }
          { \exp_not:n { \choice ##1 } }
      }
    \begin{choices}
      \seq_use:Nn \l_jodb_choices_body_out_seq {  }
    \end{choices}
  }

\ExplSyntaxOff

\begin{document}

\printanswers

\begin{questions}

\question
Blah
\begin{randomizechoices}[ans:blah]
\choice Hello
\choice World!
\CorrectChoice and
\choice everyone!
\end{randomizechoices}

\question
Blah (but see also \ref{ans:blah})
\begin{randomizechoices}
\choice Hello
\choice World!
\correctchoice and
\choice everyone!
\end{randomizechoices}

\end{questions}

\end{document}

Answer 2

The key is \seq_set_split:NnV, so the sequence is populated with the items separated by \choice and it can be randomized.

Then you can rebuild the sequence by reinserting \choice or \CorrectChoice (in case the marker is present in the item).

\documentclass{exam}

\ExplSyntaxOn

\NewDocumentEnvironment{randomizechoices}{ +b }
  {
    \jodb_choices_randomize:n { #1 }
  }
  {}

% initialize the used variables
\tl_new:N \l_jodb_choices_body_tl
\seq_new:N \l_jodb_choices_body_in_seq
\seq_new:N \l_jodb_choices_body_out_seq

\cs_new_protected:Nn \__jodb_choices_marker: { }

\cs_set_protected:Nn \jodb_choices_randomize:n
  {
    \tl_set:Nn \l_jodb_choices_body_tl {#1}
    \tl_replace_all:Nnn \l_jodb_choices_body_tl {\CorrectChoice} {\choice \__jodb_choices_marker:}
    \tl_replace_all:Nnn \l_jodb_choices_body_tl {\correctchoice} {\choice \__jodb_choices_marker:}
    \seq_set_split:NnV \l_jodb_choices_body_in_seq { \choice } \l_jodb_choices_body_tl
    % first item is empty
    \seq_pop_left:NN \l_jodb_choices_body_in_seq \l_tmpa_tl
    % randomize
    \seq_shuffle:N \l_jodb_choices_body_in_seq
    % build another sequence
    \seq_set_map_x:NNn \l_jodb_choices_body_out_seq \l_jodb_choices_body_in_seq
      {
        \tl_if_in:nnTF { ##1 } { \__jodb_choices_marker: }
          { \exp_not:n { \CorrectChoice ##1 } }
          { \exp_not:n { \choice ##1 } }
      }
    \begin{choices}
    \seq_use:Nn \l_jodb_choices_body_out_seq {  }
    \end{choices}
  }

\ExplSyntaxOff

\begin{document}

\printanswers

\begin{questions}

\question
Blah
\begin{randomizechoices}
\choice Hello
\choice World!
\CorrectChoice and
\choice everyone!
\end{randomizechoices}

\question
Blah
\begin{randomizechoices}
\choice Hello
\choice World!
\correctchoice and
\choice everyone!
\end{randomizechoices}

\end{questions}

\end{document}

Sorry, but adding a label to the correct choice is not going to work, because at each LaTeX run references will typically change.