# Duplicate Index Entries

I'm just starting to experiment with indexes, not very successfully yet. I am getting lots of duplicate entries, I suppose because of hyphenation. See my example below, which I've made from a grep of my real file, in which, for "Artha-śāstra", I particularly made sure that I took the first occurence of it and copied it over the later ones, the file having been written by two authors with the theoretical possibility of different unicode characters being used for the diacritical combinations. Looking at the idx file it seems that the two instances in which the discretionary hyphen is actually used it gets expanded to \indexentry{Artha\discretionary {-}{}{}śāstra@\textit {Artha\discretionary {-}{}{}śāstra}}{1} whereas in the other ones it is just \indexentry{Artha\-śāstra@\textit {Artha\-śāstra}}{1}. But maybe it's something else.

\documentclass{article}
\usepackage{fontspec}
\usepackage{makeidx}
\makeindex
\newcommand{\ttl}[1]{{\textit{#1}\index{#1@\textit{#1}}}}
\newcommand{\skt}[1]{{\textit{#1}}}
\begin{document}
Beginning at the top of the first passage, the first thing we shall comment on is the prevalence of magical pills in early tantric literature.  These appear to be absent in the \ttl{Atharva\-veda\-pariśiṣṭa}, but we find the preparation of pills for making others fall unconscious in the mantra and \skt{prayoga} given in \ttl{Artha\-śāstra} 14.3.19ff.  To call them pills, however, is potentially confusing, since in our tantric sources they are clearly to be popped into the \skt{sādhaka}’s mouth and held there, not swallowed, and they enable the \skt{sādhaka} to obtain powers.
\footnote{This is of course not something that develops for the first time in tantric sources: the \ttl{Artha\-śāstra} uses the magical power of the cremation ground in 14.3.29, 14.3.32 and 14.3.49.  We may note however that it refers to it with the expression \skt{asaṃkīrṇa ādahane}, ‘in an unfrequented burning ground’.  Tantric sources never, as far as we are aware, use this expression: they refer instead to the \skt{śmaśāna}.}
in \ttl{Atharva\-veda\-pari\-śiṣṭa} 36.21.1 and the fourteenth day of the dark fortnight (never the eighth) is specified for several of the magical \skt{prayoga}\/s given in the \ttl{Artha\-śāstra}.%
\footnote{\ttl{Artha\-śāstra} 14.3.28, 14.3.41, 14.3.49, 14.3.58, 14.3. 69, 14.3.70, 14.3.85.  The fourteenth day is recommended for summoning the ancestors (\skt{pitṛ}) in order to satisfy them in \ttl{Atharva\-veda\-pari\-śiṣṭa} 43.6.1.}
in the \ttl{Artha\-śāstra} 4.5.7, and in \ttl{Aṣṭādhyāyī} 4.3.72.%
Probably widespread throughout the world is the use of effigies for damaging, destroying or gaining power over other people by a form of sympathetic magic in which whatever is done to the doll is held to be effected on the person represented.  It is therefore no surprise to find them in, for example, the \ttl{Artha\-śāstra} (14.3.69–72):
\ttl{Rāmāyaṇa} 5.46.38, 6.45.22; \ttl{Artha\-śāstra} 1.19.11ab, 14.3.31, 14.3.45ab, 14.3.58; \ttl{Mahābhārata} 3.274.25, 7.58.9, 7.121.30, etc.
\printindex
\end{document}


Update:

The issue reported above occured when I generated the ind file with makeindex duplicates.idx. In the meantime I found out that the same file builds fine, without duplicates, when I generate it with the following command: xindy -M texindy duplicates.idx, thus the problem is solved for me. Curiously though,texindy duplicates.idx, which I would have expected to do the same thing, while taking care of different discretionary hyphens, does not manage to avoid duplicates for words containing discretionary hyphens in footnotes.

the \- in your entries is definitely the cause of multiple listings, as you conjecture.

as you observe, in the .idx file, some of the entries contain the string \discretionary {-}{}{}. these are the entries that are input in footnotes, and this is caused by the expansion that takes place while processing a footnote.

if you can avoid placing index entries in footnotes, instead entering them just before the footnote, you should get better results.

the same thing would happen with such an index entry in a section heading or a caption, any place that is in a "moving argument" (i.e. one saved for later reuse in a different location). while the argument of \index can be considered a moving argument, it gets special treatment -- but only after the context in which it appears has been otherwise processed.

• "avoid placing index entries in footnotes, instead entering them just before the footnote": That would mean to remove the \index (or equivalent) command from the entry in the footnote and make an invisible entry just before the footnote? How is that done? I cannot change the text, if that's what you meant. – muk.li Sep 20 '15 at 12:38
• a direct \index{...} entry (taking care to avoid unwanted spaces around it) will leave no evidence in the main output. you've gotten in trouble here by embedding the indexed name in a command that's doing "double duty". you already have a command \skt to enter the name only, so inputting \index directly, while being a nuisance, only adds to the bulk of the input. (you did well to copy the "authoritative" names rather than trying to rekey them. that's where many people get in trouble.) – barbara beeton Sep 20 '15 at 12:46
• See my answer below: tex.stackexchange.com/a/269372/59023 . – muk.li Sep 24 '15 at 21:41

The same file builds without duplicates when I generate the ind file with the following command: xindy -M texindy duplicates.idx. Curiously,texindy duplicates.idx, while taking care of different discretionary hyphens, does not manage to avoid duplicates for words containing discretionary hyphens in footnotes.

• you should have mentioned in your question that you are using xindy (although it's possible to infer that, since makeindex by itself doesn't handle accented or non-latin-alphabet text well). actually, the information here would all better be added to the question itself, as it would provide a more precise testbed for someone trying to answer. – barbara beeton Sep 25 '15 at 12:22
• @barbarabeeton my first experiments few days ago were with makeindex, only now I am using xindy. – muk.li Sep 25 '15 at 12:24
• the information would still be particularly useful edited into the question itself, where it will get more notice. – barbara beeton Sep 25 '15 at 13:59
• @barbarabeeton Thanks for the suggestion. I edited the question accordingly. – muk.li Sep 25 '15 at 15:35