Implementing a keyword / token / tag system

Question

I want to have a keyword / tag / label / token system for my documents. There will be a hard coded list of allowed keywords and all theorems/examples/exercises will (optionally) be tagged with one of those keywords. Compiling the document should check whether a valid keyword is used as tag and (if required) print it.

I am trying to implement using the arrayjob package. All my tags will be stored in a array called tagtoken and a command \tagit will do the job of actual tagging. I tried to hack the code with bits and pieces cargo-cult-lifted from various questions of tex.stackexchange. However I have no real knowledge of expl3.

This is my MWE

\documentclass{article}
\usepackage{arrayjob,expl3,xparse,pgffor}

\newtheorem{theorem}{Theorem}

\newarray\tagtoken
\readarray{tagtoken}{RAG&TAG&BAG} % should be case insensitive
\newarray\tokenlist

\ExplSyntaxOn
\NewDocumentCommand{\tagit}{m}
{
    \tl_set:Nn \l_tmpa_tl {\tl_lower_case:n{#1}}
    \regex_replace_all:nnN {,} {&} \l_tmpa_tl
    \regex_replace_all:nnN {\s+} {} \l_tmpa_tl
    \readarray{tokenlist}{\l_tmpa_tl} % does not work
    \regex_replace_all:nnN {&} {, } \l_tmpa_tl % just for displaying
    \newline Keywords = \tl_use:N \tl_upper_case:n{\l_tmpa_tl}
}
\ExplSyntaxOff


\begin{document}

    \begin{theorem}
        Dummy theorem \tagit{g,h , k, rag, TAG} % should throw an error and stop compilation till invalid tokens are purged.
    \end{theorem}

    %\foreach\x in {1,...,3} {\noindent tagtoken(\x) = \tagtoken(\x)\hfil\\}
    one = \tokenlist(1) % no output

    \begin{theorem}
        two \tagit{rag,TaG} % no error
    \end{theorem}

    \begin{theorem}
        three \tagit{rag, RAg} % should produce a warning for repeated tags
    \end{theorem}

\end{document}

How do I make it work? Sorry for asking a cargo-cult question.

Answer 1

There's no point in using arrayjob, as expl3 provides better functions.

\documentclass{article}
\usepackage{xparse}

\newtheorem{theorem}{Theorem}

\ExplSyntaxOn
\NewDocumentCommand{\definetags}{m}
 {
  \clist_gset:Nn \g_magguu_tags_clist { #1 }
 }

\NewDocumentCommand{\tagit}{m}
 {
  \magguu_tags_tagit:n { #1 }
 }

%% variables
\clist_new:N \g_magguu_tags_clist
\seq_new:N \l__magguu_tags_seq

%% variants
\prg_generate_conditional_variant:Nnn \clist_if_in:Nn { Nx } { T,F,TF }
\prg_generate_conditional_variant:Nnn \seq_if_in:Nn { Nx } { T,F,TF }

%% messages
\msg_new:nnnn { magguu/tags } { invalid }
 {
  Invalid~tag~'#1'~on~line~\msg_line_number:
 }
 {
  The~tag~'#1'~is~not~among~the~valid~tags. \\ Fix~it.
 }
\msg_new:nnn { magguu/tags } { duplicate }
 {
  The~tag~'#1'~is~duplicate~on~line~\msg_line_number:
 }

%% functions
\cs_new_protected:Nn \magguu_tags_tagit:n
 {
  \seq_clear:N \l__magguu_tags_seq
  \clist_map_inline:nn { #1 }
   {
    \clist_if_in:NxTF \g_magguu_tags_clist { \tl_upper_case:n { ##1 } }
     { % the tag is valid
      \seq_if_in:NxTF \l__magguu_tags_seq { \tl_upper_case:n { ##1 } }
       { % the tag is duplicate
        \msg_warning:nnn { magguu/tags } { duplicate } { ##1 }
       }
       { % add the tag
        \seq_put_right:Nx \l__magguu_tags_seq { \tl_upper_case:n { ##1 } }
       }
     }
     { % invalid tag
      \msg_error:nnn { magguu/tags }{ invalid } { ##1 }
     }
   }
   \newline Keywords:\nobreakspace\seq_use:Nn \l__magguu_tags_seq { ,~ }
 }


\ExplSyntaxOff

\definetags{RAG,TAG,BAG}

\begin{document}

\begin{theorem}
Dummy theorem \tagit{g,h , k, rag, TAG}
\end{theorem}

\begin{theorem}
two \tagit{rag,TaG} % no error
\end{theorem}

\begin{theorem}
three \tagit{rag, RAg} % should produce a warning for repeated tags
\end{theorem}

\end{document}

Output

Console output (errors and warnings)

! Package magguu/tags Error: Invalid tag 'g' on line 69

For immediate help type H <return>.
 ...                                              

l.69 Dummy theorem \tagit{g,h , k, rag, TAG}

? h

The tag 'g' is not among the valid tags.
Fix it.

? 

! Package magguu/tags Error: Invalid tag 'h' on line 69

For immediate help type H <return>.
 ...                                              

l.69 Dummy theorem \tagit{g,h , k, rag, TAG}

? 

! Package magguu/tags Error: Invalid tag 'k' on line 69

For immediate help type H <return>.
 ...                                              

l.69 Dummy theorem \tagit{g,h , k, rag, TAG}

? 

Package magguu/tags Warning: The tag 'RAg' is duplicate on line 77

Answer 2

This can be done with list processing from the etoolbox package. The idea is to create a list with allowed tags, and then check each item from the input of \tagit against this list.

A separate list with already used tags (which is reset for each new tag list) can be used to check for duplicates.

The errors and warnings can be created with the built-in \GenericError and \GenericWarning commands.

Case-insensitive checking can be achieved by creating the allowed tag list in lower case and converting the tags to lower case before checking against the list. This requires a bit of strange syntax, see https://tex.stackexchange.com/a/286386/.

In the code below the duplicate check is performed first, and the error checking and actual tag printing are done only if the tag is not a duplicate.

MWE:

\documentclass{article}
\usepackage{etoolbox}
\newtheorem{theorem}{Theorem}
\begin{document}
% create list of allowed tags
\forcsvlist{\listadd\allowedtags}{rag,tag,bag}
% macro for processing a single tag
\renewcommand*{\do}[1]{%
% put argument in lower case
\lowercase{\def\lcarg{#1}}%
% check if item is already used
\xifinlist{\lcarg}{\useditems}{\GenericWarning{xxx}{Warning: duplicated tag '#1'}}{%
% check if item is in list of allowed tags
\xifinlist{\lcarg}{\allowedtags}{\fbox{\normalfont #1} }{\GenericError{xxx}{tag not allowed: #1}{please remove tag}{further help}}}%
% add item to list of used items
\listeadd{\useditems}{\lcarg}%
}
% command for processing tag list
\newcommand{\tagit}[1]{%
% reset list of used items
\gdef\useditems{}
% execute \do for each item in the list
\docsvlist{#1}%
}

\begin{theorem}
Dummy theorem \tagit{g,h , k, rag, TAG, rag} % should throw an error and stop compilation till invalid tokens are purged.
%Dummy theorem \tagit{rag, TAG, RaG}
\end{theorem}

\end{document}

This triggers the error:

! tag not allowed: g.

please remove tag
Type  H <return>  for immediate help.
 ...                                              

l.18 Dummy theorem \tagit{g,h , k, rag, TAG, rag}
                                                  % should throw an error an...

Commenting the offending line and uncommenting the next triggers the warning:

Warning: duplicated tag 'RaG' on input line 19.

Resulting pdf:

Of course if you want to do something different with the tags (conditional printing, making a list/index somewhere else in the document, etc) then you should modify the first clause of the allowed tags check (the \fbox part).