# Automatic text highlighting based on a dictionary

I would like to automatically highlight parts of a running text, based on whether the word is in a predefined dictionary, or not. Something along the lines of...

\documentclass{article}
\mydictionary{apple,banana}%A comma-separated list of unicode characters. For simplicity the dictionary contains words, and not compound expressions, such as "yellow lemon". In other words, no spaces are allowed.
\def\autohighlightStyle1{ textcolor=red }{ dictionary=\mydictionary }%I need the infrastructure to define my own formatting. For simplicity, in this example it is enough to change the textcolor. I will add my own formatting options later by myself.
\begin{document}
\autohighlightStyle1{Monkeys like banana, but they don't like apple.}
\end{document}


And the result would be a text, where the words from dictionary \mydictionary (banana and apple) are formatted according to some instructions, here for simplicity they are colored to red.

Any chance that something like this is doable? Essentially I will need to tokenize the text by the spaces, and then compare word by word it with the contents of the dictionary, and then apply the style, and then print them out.

I am using xelatex, and unicode text (but again, in this simple example spaces function as a delimiter).

Ideally, I would like to apply this wrapper to the entire document, right after \begin{document} and close right before \end{document}, so... it should be... robust... But... for simplicity, perhaps we may assume that it receives ordinary text, as an argument. Another idea is that maybe I will have to use a style file to do something like this?

• Thank you for the bonus! Feb 20 at 23:03

Define dictionary words separated by || using the \setsepchar macro. Then, after \begin{document}, issue \def\currentword{}\tokencyclexpress and before \end{document}, issue \endtokencyclexpress. This sets up an environment where all the tokens of the document are screened in search of dictionary words first. If found, highlighting is added to the input token stream, prior to execution of the token list.

Note that word gets one style of highlighting if it is an exact match, and a different style if it merely contains the dictionary word, for example, plurals.

Highlight format for exact matches is set with \newcommand\autohighlightStyleA{\textcolor{red}}.

Highlight format for includes'' matches is set with \newcommand\autohighlightStyleB{\textcolor{blue}}.

If a dictionary word is broken across a group boundary, as in ap\textbf{ple}, it will not be recognized for highlight.

I should add that the approach can find dictionary words embedded in groups such as \textit{banana}, which is actually a difficult thing to accomplish, if it weren't for tokcycle.

\documentclass{article}
\usepackage{tokcycle,listofitems,xcolor}
\setsepchar{apple||banana}
\newcommand\testdict{%
\if\relax\detokenize\expandafter{\currentword}\relax\else
{\ignoreemptyitems
\ifnum\listlen\dictcompA[]=0\relax
\else
\ifnum\listlen\dictcompB[]>1\relax
\else
\fi
\fi
\fi
\gdef\currentword{}%
}
\makeatletter
\stripgroupingtrue
\Groupdirective{\testdict\groupedcytoks{\processtoks{#1}\testdict}}
\makeatother
\newcommand\autohighlightStyleA{\textcolor{red}}
\newcommand\autohighlightStyleB{\textcolor{blue}}
\begin{document}
\def\currentword{}\tokencyclexpress
\tableofcontents

\listoffigures

\section{My Section about a banana and two apples}

Monkeys like banana, but they don't like apple.

Monkeys like \textit{banana}, but \textit{they} don't like ap\textbf{ple}.

Monkeys \textit{like bananananas,} but \textit{they} don't like \textbf{apple}.
$$\textrm{banana} = 2^\mathrm{apples}$$
\begin{figure}[ht]
\centering
\fbox{apple and banana}
\caption{A boxed version of apple and banana''}
\end{figure}
\endtokencyclexpress
\end{document}


p.s. This works in pdflatex, xelatex, and lualatex.

• +1: So this is not based on Lua, correct? Impressive. Feb 12 at 22:16
• @Dr.ManuelKuehner Correct, it is straight LaTeX...TeX actually, as both the tokcycle and the listofitems packages can run in plain TeX. Feb 12 at 22:17
• @Steven B. Segletes, this is just wonderful. I am teaching a bilingual class, and I would like to highlight the very few words my students are expected to know. These then would stand out very clearly from what is otherwise a large wall of [foreign] text. Very much appreciated. Feb 15 at 9:27
• @Matsmath My instructions search the whole document. I'm sure you realize you can start and end the environment at will, if there are only particular sections where you want the highlighting to occur. Feb 16 at 2:51
• @Matsmath You can simplify the syntax a little by defining, in the preamble, \def\findindict{\def\currentword{}\tokencyclexpress} \let\endfindindict\endtokcycraw. Then, the invocation goes as \findindict...\endfindindict, without the explicit need to reset \currentword. Mar 1 at 17:20