Indexing all initial and non-initial words

Question

Please save your time and energy on this question, a possible solution is provided.

I've found a book with two interesting indices. It lists Tamil proverbs and the book indexes all the initial words of the proverbs (the first index) and all the other words (the second index). How could we achieve that in TeX? I enclose an initial TeX file.

In this example we should get a, Hello and How terms in the first index and all the other proverb terms in the second index. Please notice that symbols like commas, semi-colons etc. should not be a part of the indices.

% run: *latex mal-indexall-question.tex
\documentclass[a4paper]{article}
\usepackage[noautomatic]{imakeidx}
\indexsetup{firstpagestyle=empty}
\makeindex[name=initial, title=Initial words, columns=3]
\makeindex[name=noninitial, title=Non-initial words, columns=3]
\usepackage[colorlinks]{hyperref}
\begin{document}
\let\indexall=\index % To be changed...
Text before.\indexall{a," [b;] c: r: s: t; {d.}/ (e!) f? g| h¡ i¿}
Text in the middle.\indexall{Hello World!} 
Text\indexall{How are you?} after.
\printindex[initial]
\printindex[noninitial]
\end{document}

Answer 1

I'm using Lua on-the-fly to solve this task, so my solution is limited to LuaTeX. It's probably not the best approach. I am using a new command \indexall which passes its argument to a Lua function indexall.

That function deletes special characters in the text block and splits its content up by spaces. The first term is saved to the first index, all the other words are saved to the second index. We run the following lines:

lualatex mal-indexall.tex
xindy -M texindy -L general -C utf8 initial.idx
xindy -M texindy -L general -C utf8 noninitial.idx
lualatex mal-indexall.tex

I enclose an example and a preview of the PDF file.

% run: lualatex mal-indexall.tex
\batchmode % Less information to the terminal, please...
\documentclass[a4paper]{article}
\usepackage[noautomatic]{imakeidx}
\indexsetup{firstpagestyle=empty}
\makeindex[name=initial, title=Initial words, columns=3]
\makeindex[name=noninitial, title=Non-initial words, columns=3]
\usepackage[colorlinks]{hyperref}
\usepackage{luacode}
\begin{document}
\begin{luacode*}
-- List of characters to be deleted...
local deletethese={ ",", ";", ":", "%.", "!", "?", "\"", "%(", "%)", "%[", "%]", "{", "}", "/", "|", "¡", "¿" }
-- The core of this TeX file...
function indexall(text)
local c=0 -- a word counter
local textnew=text -- backup of original text
print("Processing "..text.." ...")
-- Deleting unnecessary signs...
for _, letter in ipairs(deletethese) do
  -- print(letter)
  textnew=string.gsub(textnew,letter,"")
end -- of for
--print(textnew)
-- Use space and separate index terms...
for indexword in string.gmatch(textnew, "([^ ]+)") do
  c=c+1
  if c==1 then
    tex.sprint("\\index[initial]{"..indexword.."}")
      else
    tex.sprint("\\index[noninitial]{"..indexword.."}")
  end
end
end -- of function indexall
\end{luacode*}
\def\indexall#1{\directlua{indexall([[#1]])}}
\scrollmode
Text before.\indexall{a," [b;] c: r: s: t; {d.}/ (e!) f? g| h¡ i¿}
Text in the middle.\indexall{Hello World!} 
Text\indexall{How are you?} after.
\printindex[initial]
\printindex[noninitial]
\batchmode
\end{document}