# Is there a place for one more index processor?

(This is not a real question, it's rather a list of thoughts and ideas.)

I'm not a serious TeXist, but this topic of index creation interests me a lot. As I like programming in Lua, I started to think if I could try an experiment or two to cover some topics in the indexing world. Let me think about it for a while. In this post/question, I'm trying to collect ideas which we are missing, could be improved or simply it would be nice to have them among out-of-the-box features.

This list might help programmers to actually program something new or me to get on the right track before I actually do some programming.

# What's ready for use? (TeX Live 2014, possibly next years too)

There are some tools and packages we use frequently for sorting and/or index creation:

# What we may like to have at our disposal? (to be updated)

Support for more languages

For xindy, I tried to prepare definition files for Sanskrt (transliteration scheme) and Pāḷi. But it's quite time consuming to set it up properly. It would be better to define just some letters, let's say in a Lua table. It would be also helpful to have an option to set up a language dialect.

Support for CJKV languages

The support for CJKV and other exotic languages is weak in xindy. They are using radicals, number of strokes and even frequency of kanjis in Japanese (JLPT levels). It means that we must handle additional data sets, e.g. Unicode Character Database; UCD or Unihan Database. I haven't checked out the new engines and tools in detail (xepersian, support for Japanese, Chinese, Korean etc.), I cannot say if the index creation is ready for TeX users.

Setting up a new set of sorting rules and letter groups

One of the examples comes from the chemistry field and its Periodic Table of Elements. I believe we could find some examples in Astrophysics. It would be interesting to sort items by its colour (name, wave), but that's not a common task.

It would be great to have two-way hyperlinks coming to/from the index. I tried that once in this question. Sometimes we turn two-way hyperlinks on elsewhere, e.g. in the footnotes.

Mixing writing

It might be possible to switch between transliteration schemes (like in XeLaTeX via map files) at the input/output side, or we could even mix them, e.g. sayōnara + さようなら which is the same index entry (romanized version, hiragana).

Index with different piece of information

I've shown one of the approaches in this question, but it is far from user-friendly. Instead of page numbers, we might have other blocks, such as sections, numbers of line in the source code, numbers of the problem/solution, numbers of some examples, verse numbers, or own counter. There is an interesting question dealing with adding a beginning of the verse to the index.

List of occurences

Also a list of occurences of the index entries (and number of index entries in the group letter) should be easily available (see this question).

Support for indexing more languages in one index

There is an open issue of finderlist, it might be useful not only for Pohnpeian + English. We should be able to change the order of languages of such a finderlist. There is one more interesting open issue: Ancient Egyptian, where could be even three different languages/sorting systems (pronunciation, transliteration, Unicode notation and translation).

Modifying rules

I tried to correct Icelandic and Vietnamese in Xindy, but it took a lot of time. In Czech, for instance, some people like to have letter groups as A, B, C, Č, D, ..., S, Š, ... U, V, W, X, Y, Z, but some people (due to lack of index entries) like to have A, B, CČ, D, ..., SŠ, ..., U, V, W, XYZ.

Testing files

In addition to the previous item, it would be great to have some testing files per language. Or some examples of / links to already created documents with index. It's possible to find some books, but it requires a lot of researching for non-native speakers in that language. It remains me test cases in data analysis to measure efficiency of a new algorithm.

Sorting engine on, indexing engine off

It would be nice to be able to switch between indexing and sorting engines. I tried to modify a xindy style once. Indexing processor might behave like a sorting engine upon request.

Sorting engine off, indexing engine on

I used a trick with adding numbers before an index term. It works, but I wouldn't recommend it for general use. It might be interesting to get an index within some group without sorting index entries within the letter/page group (it equals to an appearance in text, or sorting by page numbers without additional sorting key).

Handling numbers

Xindy is handling arabic and roman (I'm not 100% sure here) numbers well, see e.g. this question. There should be an addition to hexadecimal numbers. It could be solved with regular expressions in Lua. We may like to switch the output of the numbers (Western or Eastern Arabic, Roman, Urdu variant, Bengali, Indian numbers, ...).

Easy setting of the tags at the output side

We could be able to easily set up all tags used in the index. We could be able to limit number of elements, e.g. to see only the first element (first occurence of the index entry / definition / abbreviation). It's been discussed at TeX.SX.

Automatic repetion of the group letter

Once we have an index with more columns/pages, it would be nice to have group letters at the beginning of each column/page automatically repeated.

Adding text before and after index

We can do a lot of formatting at a TeX level, see this question, but there should be an easier way how to add information to the index. Moreover, we should be able to hyperlink whole index entry (see this question).

Creating more indices

We can do that easily with the help of the imakeidx package, see this example. We may like to have minindices (per chapter, section, page number), something similar to the minitoc package for Mini Table of Contents handling.

Index creation without TeXing

There are certain situations where we might not even need TeX at all. Lua could parse input TeX files. Michal.h21 showed one of the situations, indexing against keyword file, but that's a rare task.

# Where the TeX world stands now?

Xindy

A new language can be set via xindy-make-rules for Xindy. The structure of data looks like this. We use or we leave empty so-called slots (arrays in arrays in an array).

$language = "Czech";$prefix = "cs";
$script = "latin";$alphabet = [
['A',    ['a','A'], ['á','Á'],['ä','Ä']],
[], # a with breve (romanian)
[], # a with circumflex (romanian)
[], # a with ogonek (polish)
['B',    ['b','B']],
[], # b with hook (hausa)

[... omitted...]

['Z',    ['z','Z']],
[], # zh (albanian)
[], # zs (hungarian)
['Ž',    ['ž','Ž']],
[], # z with acute (lower sorbian, polish)
[], # z with dot above (polish)
];

$ligatures = [ [['ß'], 'after', [['s','s']]], ]; @special = ('?', '!', '.', 'letters', '-', '\'');$sortcase = "Aa";
#$sortcase = "aA"; do 'make-rules.pl';  TECkit TECkit offers fast and flexible way for transliteration schemes, but it can be used only in XeTeX on-the-fly. LuaTeX and ConTeXt Hans Hagen and comp. are developing tools very fast (ConTeXt, LuaTeX). For instance, in the /texmf-dist/tex/context/base/sort-lan.lua file, we can find sorting rules for the Czech language: [... omitted ...] definitions["cz"] = { replacements = { { "ch", ch }, { "Ch", ch }, { "CH", ch } }, entries = { ["a"] = "a", ["á"] = "a", ["b"] = "b", ["c"] = "c", ["č"] = "č", ["d"] = "d", ["ď"] = "d", ["e"] = "e", ["é"] = "e", ["ě"] = "e", ["f"] = "f", ["g"] = "g", ["h"] = "h", [ch] = "ch", ["i"] = "i", ["í"] = "i", ["j"] = "j", ["k"] = "k", ["l"] = "l", ["m"] = "m", ["n"] = "n", ["ň"] = "n", ["o"] = "o", ["ó"] = "o", ["p"] = "p", ["q"] = "q", ["r"] = "r", ["ř"] = "ř", ["s"] = "s", ["š"] = "š", ["t"] = "t", ["ť"] = "t", ["u"] = "u", ["ú"] = "u", ["ů"] = "u", ["v"] = "v", ["w"] = "w", ["x"] = "x", ["y"] = "y", ["ý"] = "y", ["z"] = "z", ["ž"] = "ž", }, orders = { "a", "á", "b", "c", "č", "d", "ď", "e", "é", "ě", "f", "g", "h", ch, "i", "í", "j", "k", "l", "m", "n", "ň", "o", "ó", "p", "q", "r", "ř", "s", "š", "t", "ť", "u", "ú", "ů", "v", "w", "x", "y", "ý", "z", "ž", }, upper = { ch = CH, }, lower = { CH = ch, } } definitions["cs"] = { parent = "cz" } [... omitted ...]  It's a nice way of handling Lua tables, but I feel it could be simplified for a regular user for setting up own sorting/indexing rules. # CLDR: A way to go? First look at the CLDR data I like an approach by the Unicode Consortium (it's similar, maybe even related, to the biber tool for sorting bibliography entries), they are providing so called CLDR data (Unicode Common Locale Data Repository; I've found this project just recently), which includes data for index creation. Let me show you an example for the Czech language (after downloading and unzipping the core.zip file, http://unicode.org/Public/cldr/27/). The rules are written in the /code/common/main/ folder in the cs.xml file.  [... omitted ...] <characters> <exemplarCharacters>[a á b c č d ď e é ě f g h {ch} i í j k l m n ň o ó p q r ř s š t ť u ú ů v w x y ý z ž]</exemplarCharacters> <exemplarCharacters type="auxiliary">[à ă â å ä ã ā æ ç è ĕ ê ë ē ì ĭ î ï ī ľ ł ñ ò ŏ ô ö ø ō œ ŕ ù ŭ û ü ū ÿ]</exemplarCharacters> <exemplarCharacters type="index" draft="contributed">[A B C Č D E F G H {CH} I J K L M N O P Q R Ř S Š T U V W X Y Z Ž]</exemplarCharacters> <exemplarCharacters type="punctuation">[\- ‐ – , ; \: ! ? . … ‘ ‚ “ „ ( )  § @ * / \&amp;]</exemplarCharacters> [... omitted ...]  If we could parse this data (see this webpage about LDML; the Unicode Locale Data Markup Language), rules for a new language (Czech one in this example) can be set up. It's covering many languages and dialects, but e.g. Pohnpeian language is not covered, yet. Therefore I think, there still should be an easy way how to define a new language even with this data available. There is probably a lot of tools to parse and preprocess these files, let me mention cldr from the JavaScript world. It's hard to say if biber can be used for sorting in index creation, even if it could, it isn't handling typical issues of the index creation. Notes about Sinhala and Tamil Some years ago, Zdeněk Wagner from Prague, showed me his experiment in Devanāgarī, but it isn't included among official Xindy files, yet. Zdeněk told me his files are not completely ready to be published. So support for Brāhmī scripts is likely still weak. One day, I would like to see sorted lists / indices e.g. in Pāḷi, preferably in Sinhala and Tamil not mentioning Japanese, Chinese, Korean and many more. It looks it might be possible as CLDR contains files si.xml (ISO 639-1; Sinhala) and ta.xml (ISO 639-1; Tamil). Allkeys from Unicode might provide some help in certain languages/situations. It looks that difficult part in programming is handling complex ligatures in those Eastern languages (next to handling radicals and number of strokes; that's another big issue). This is a portion of the table scheme a colleague of mine, Jan Kučera from Prague, sent me for Tamil. After we type a character followed by one of the MarkPlaceholders we are getting a ligature. That's a general idea, he has probably implemented it somewhere. "", "^"); "அ", "a"); "ஆ", "ā", "A", "aa"); "இ", "i"); "ஈ", "ī", "I", "ee"); "உ", "u"); "ஊ", "ū", "U", "oo"); "எ", "e"); "ஏ", "ē", "E"); "ஐ", "ai"); "ஒ", "o"); "ஓ", "ō", "O"); "ஔ", "au", "ou", "ow"); "ஃ", "ḵ", "H", "q"); "க" + MarkPlaceholder, "k", "K", "kh", "G", "gh"); "ங" + MarkPlaceholder, "ṅ", "ng", "~g", "g"); // g is க in Baraha "ச" + MarkPlaceholder, "c", "s", "ch", "C", "Ch"); "ஞ" + MarkPlaceholder, "ñ", "n^", "ny", "NY", "jn", "~j", "nJ"); // ந can be n^ in Madurai "ட" + MarkPlaceholder, "ṭ", "T", "Th", "d", "D", "Dh"); // d is த in Baraha "ண" + MarkPlaceholder, "ṇ", "N"); "த" + MarkPlaceholder, "t", "th", "dh"); "ந" + MarkPlaceholder, "n", "~n", "nN"); // n is ன in Baraha "ப" + MarkPlaceholder, "p", "P", "ph", "b", "B", "bh"); "ம" + MarkPlaceholder, "m", "M"); "ய" + MarkPlaceholder, "y", "Y"); "ர" + MarkPlaceholder, "r"); "ல" + MarkPlaceholder, "l"); "வ" + MarkPlaceholder, "v", "w"); "ழ" + MarkPlaceholder, "z", "zh", "Lx"); "ள" + MarkPlaceholder, "ḷ", "L"); "ற" + MarkPlaceholder, "ṟ", "R", "rx"); "ன" + MarkPlaceholder, "ṉ", "_n", "n2"); "ஜ" + MarkPlaceholder, "j", "jh", "J"); "ஶ" + MarkPlaceholder, "ś"); "ஷ" + MarkPlaceholder, "ṣ", "sh"); // sh is ஶ in Baraha "ஸ" + MarkPlaceholder, "S"); "ஹ" + MarkPlaceholder, "h", "~h"); "", "a", "^"); "ா", "ā", "A", "aa"); "ி", "i"); "ீ", "ī", "I", "ee"); "ு", "u"); "ூ", "ū", "U", "oo"); "ெ", "e"); "ே", "ē", "E"); "ை", "ai"); "ொ", "o"); "ோ", "ō", "O"); "ௌ", "au", "ou", "ow");  A closing note about Kana Once, I tried to prepare a style for Xindy for Kana (Hiragana and Katakana), it worked, but the core which is to be processed are the kanjis. I would like to finish my notes with a preview of this Xindy experiment from 2010. I believe that this is not the easiest way how to set up and maintain definition files for index generation. These files are not ready for production, it's highly unlikely to have an index of just Kana. In case we have a couple of terms in Kana, we would probably use \index{sayounara@さようなら} in our document. Thus, I was thinking in this post, if I(we) could do something about that to improve the current state in the world of indexing. • entirely missing from this list is how to create an index of notation in a reasonably sane way. the only really logical ones i have ever seen were hand-tuned, either by assigning sort keys individually, or created entirely by hand using \label and \ref to apply page numbers (but that means multiple \labels for any term that is to be "indexed" more than once). since ordering is, to a great extent, subjective, i don't see an easy solution to this. – barbara beeton Mar 30 '15 at 14:29 • one of my "projects for retirement" is to index tugboat. for that, the sorting of items isn't a problem (just drudgery compiling the data), but the representation of where they appear is a nightmare: volume + issue (not always needed) + page(s). already mentioned here under "Index with different piece of information", just with slightly different content. – barbara beeton Mar 30 '15 at 14:35 • @Malipivo I don't understand why do you miss the indexing plus sorting directly by TeX macros (in OPmac) in your list. You are from Czech republics and you do know OPmac. Why the \makeindex from this package is missing here? – wipet Mar 30 '15 at 15:03 • @barbarabeeton: For a customer I created some years ago a xindy style and some commands for a yearly index of a journal which appeared every month. He wanted a system like "I/2013_{715}:1,5" (month/year_{issueNr}:page,page). I don't know if someone actually can find an item with such a reference ;-). – Ulrike Fischer Mar 30 '15 at 15:35 • @UlrikeFischer -- xindy is indeed the most promising option we've got now; i'm resigned to the possibility that i may be the only person to actually use the resulting index. at least you've confirmed for me that it can be done. – barbara beeton Mar 30 '15 at 15:41 ## 1 Answer Testing files The testidx package is designed for testing indexing. It covers the Basic Latin set and some extended characters common in some European languages, but it's designed to be extendible. A simple test document: \documentclass{book} \usepackage{imakeidx} \usepackage{testidx} \makeindex \begin{document} \testidx \printindex \end{document}  This creates a test document with a seven page index that contains a symbols group, a number group, and the 26 Basic Latin letter groups A, …, Z. Some extra terms are available if the T1 font encoding is used (which provides \dh etc): \documentclass{book} \usepackage[T1]{fontenc} \usepackage{imakeidx} \usepackage{testidx} \makeindex \begin{document} \testidx \printindex \end{document}  There are two UTF-8 modes available with inputenc: • Sanitize: the sort value is first sanitized before being passed to \index to prevent the active characters from being expanded as they are written to the indexing file. This mode is designed to test the indexing application's ability to sort UTF-8 characters. • No sanitize: the sort value isn't sanitized before being passed to \index. How the UTF-8 characters are written to the indexing file depends on the definition of \index. The standard definition results in expansion. Example (the default sanitize mode): \documentclass{book} \usepackage[utf8]{inputenc} \usepackage[T1]{fontenc} \usepackage{imakeidx} \usepackage{testidx} \makeindex \begin{document} \testidx \printindex \end{document}  For example, the .idx file contains: \indexentry{élite}{4}  This will cause a problem for makeindex, which doesn't support UTF-8, but it can be used to test xindy. Example (nosanitize mode): \documentclass{book} \usepackage[utf8]{inputenc} \usepackage[T1]{fontenc} \usepackage{imakeidx} \usepackage[nosanitize]{testidx} \makeindex \begin{document} \testidx \printindex \end{document}  The .idx file now contains: \indexentry{\IeC {\'e}lite}{4}  With XeLaTeX and LuaLaTeX the result is the same in both modes as the UTF-8 characters aren't active and so don't expand. By default, the terms are variously indexed without an encap or with one of three test encaps. The dummy text is intentionally designed to trigger an encap clash. If you don't need to test for this you can switch it off with notestencaps. \documentclass{book} \usepackage{fontspec} \usepackage{imakeidx} \usepackage[notestencaps]{testidx} \makeindex[program=texindy,options={-C utf8 -L icelandic}] \begin{document} \testidx \printindex \end{document}  In this case, the index contains a default group (which includes symbols, numbers, and the letters that don't occur in the Icelandic alphabetic rule: Ć, Ł, Ś and Ż), and the letter groups A, …, D, ð, E, …, Z, Þ, Æ, Ö and Å. The index additionally includes some terms that start with a digraph (such as dd, dz, ff, ij, ll, ly, ng, th) as well as a trigraph (dzs) to allow for testing the indexing application's ability to deal with these. By default, symbols are indexed in the form \index{>alpha@$\alpha$}  or \index{<tstidxmarker@\csname tstidxmarker\endcsname \space (\tstidxcsfmt {tstidxmarker})}  or \index{E@$E\$}


to represent different ways of indexing symbols. The < and > prefixes allow different types of symbols to be grouped together but you can omit this prefix with the noprefix option.

There's also a supplementary package testidx-glossaries that uses the glossaries package interface instead of \index.

Additional test paragraphs can be added using \tstidxnewblock. The starred form requires a control sequence for the first argument that can be used to save the block number if it needs to be referenced in another block (\label/\ref aren't used as the reference is typically used as an instruction to the user and it can't be resolved if required block is omitted.) For example:

\documentclass{book}

\usepackage{fontspec}

\usepackage{imakeidx}
\usepackage[notestencaps]{testidx}

\makeindex[program=texindy,options={-C utf8 -L general}]

\tstidxnewblock*{\mystartblock}{The \tstidxopenword{cat} sat
on the \tstidxword{mat}. (This block requires block~\myendblock\
to close a range.)}

perseguiu o papagaio bobo.}

with the parrot. (This block requires block~\mystartblock.)}

\begin{document}
\testidx

\printindex
\end{document}


These new block definitions can go in a package that automatically loads testidx.

CLDR: A way to go?

There's now a new indexing alternative bib2gls but it requires glossaries-extra. Instead of using \index, terms are defined in .bib files. Since bib2gls is a Java application, if you have at least Java 8 then there you can use the CLDR as the locale provider. With Java 8, this requires -Djava.locale.providers=CLDR,JRE,SP (which is included in the bash script distributed with bib2gls). You may prefer a different order, for example -Djava.locale.providers=CLDR,SP,JRE. By placing CLDR first in the list, it's given the highest priority. If the locale isn't supported by the CLDR, then the next provider is checked. (I think Java 9 supports the CLDR by default.)

Here's a simple .bib file (called, say, entries.bib):

% Encoding: UTF-8

@index{animal}
@index{duck,parent={animal}}
@index{goose,plural={geese},parent={animal}}

@entry{sample,
name={sample},
description={an example with a description}
}

@abbreviation{html,
short="html",
long={hypertext markup language}
}

@symbol{pi,
name={\ensuremath{\pi}}
}


Here's an example document that only uses one term from the entries.bib file:

\documentclass{article}

\usepackage[record]{glossaries-extra}

src={entries}% data in entries.bib
]

\begin{document}
\gls{sample}.

\printunsrtglossaries
\end{document}


The testidx-glossaries package provides support for bib2gls:

\documentclass{book}

\usepackage{fontspec}

\usepackage[bib2gls]{testidx-glossaries}

\tstidxmakegloss

\begin{document}
\testidx

\printunsrtglossary[title={Index}]
\end{document}


\tstidxmakegloss is a convenient shortcut that does \makenoidxglossaries or \makeglossaries or \GlsXtrLoadResources depending on the package options. In this case it actually performs several \GlsXtrLoadResource calls. These can be found in the .log file if you add the verbose package option. Here is an example with explicit calls:

\documentclass{book}

\usepackage{fontspec}

\usepackage[bib2gls,manual]{testidx-glossaries}

group={Maths},
sort={letter-case},
selection={recorded and deps and see},
ignore-fields={description}
]

group={Markers},
sort={letter-case},
selection={recorded and deps and see},
ignore-fields={description}
]

sort={integer},
selection={recorded and deps and see},
ignore-fields={description}]

src={\tstidxbasebibfiles,\tstidxutfbibfiles,\tstidxglyphfile-utf8},
selection={recorded and deps and see},
ignore-fields={description}]

\renewcommand{\glsnamefont}[1]{\textmd{#1}}

\begin{document}
\testidx

\printunsrtglossary[title={Index}]
\end{document}


Each instance of \GlsXtrLoadResources creates a new block in the index/glossary (which may be identified by the type option, but in this example there's only the default main type). A block doesn't necessarily correspond to a letter group or provide any visual distinction. It just allows different sorting within one list. The group option assigns the letter group if required, but bib2gls must be called with the --group (or -g) switch to work with this option. The value of the group option is a label. If the corresponding title is different, it can be set with \glsxtrsetgrouptitle.

Handling numbers

Numbers can be sorted using one of the number sort methods. In the above, the third block is sorted according to sort={integer}. Java recognises various number systems, so as long as the sort values can be parsed they should be numerically sorted. Any values that can't be parsed will be treated as 0.

Setting up a new set of sorting rules and letter groups

Custom groups (such as numbers and symbols) can be created as shown above, but you can also supply your own rule. The rule syntax can be quite long, but the glossaries-extra-bib2gls package (which is automatically loaded by glossaries-extra when the record option is used) provides some convenient shortcuts. For example:

\documentclass{book}

\usepackage{fontspec}

\usepackage[bib2gls,manual]{testidx-glossaries}

group={Maths},
sort={letter-case},
selection={recorded and deps and see},
ignore-fields={description}
]

group={Markers},
sort={letter-case},
selection={recorded and deps and see},
ignore-fields={description}
]

sort={integer},
selection={recorded and deps and see},
ignore-fields={description}]

src={\tstidxbasebibfiles,\tstidxutfbibfiles,\tstidxglyphfile-utf8},
selection={recorded and deps and see},
ignore-fields={description},
sort={custom},
sort-rule={\glsxtrcontrolrules
;\glsxtrspacerules
;\glsxtrnonprintablerules
;\glsxtrcombiningdiacriticrules
,\glsxtrhyphenrules
<\glsxtrgeneralpuncrules
<\glsxtrdigitrules
<\glsxtrfractionrules
<a,A <b,B <c,C <ch,CH,Ch <d,D <dd,DD,Dd <e,E <f,F <ff,FF <g,G <ng,NG,Ng
<h,H <i,I <l,L <ll,LL <m,M <n,N <o,O <p,P <ph,PH,ph <r,R <s,S <t,T
<th,TH,Th <u,U <w,W <y,Y
}
]

\renewcommand{\glsnamefont}[1]{\textmd{#1}}

\begin{document}
\testidx

\printunsrtglossary[title={Index}]
\end{document}


Modifying rules

This can also be done through a custom rule. If you run bib2gls with the --debug switch, the rule will be written to the transcript .glg file. (Search for Collator rules:) You can copy and paste this, but be careful of the control codes at the start. It's best to replace them with \glsxtrcontrolrules etc as shown above.

Support for indexing more languages in one index

You can either use a custom rule that supports both languages or use a separate \GlsXtrLoadResources for each language. For example:

\GlsXtrLoadResources[src={german-terms},sort={de-CH-1996}]


The terms are referenced in the document using commands like \gls provided by the glossaries package. These become hyperlinks to the relevant entry in the list if the hyperref package is loaded before glossaries (and therefore before glossaries-extra and testidx-glossaries, which implicitly load it).

\documentclass{book}

\usepackage{fontspec}

\usepackage[bib2gls]{testidx-glossaries}

\tstidxmakegloss

\renewcommand{\glsnamefont}[1]{\textmd{#1}}

\begin{document}
\testidx

\printunsrtglossary[title={Index}]
\end{document}


The page numbers in the location lists link back. If you want the term to link back (for example, to the primary reference), then it's more complicated but can be done, although it's still experimental. (Requires at least glossaries-extra v1.29 and bib2gls v1.4.)

\documentclass{article}

\usepackage{amsmath}
\usepackage{lipsum}
\usepackage[record,indexcounter]{glossaries-extra}

\newcommand{\primary}[1]{\hyperbf{#1}}

save-index-counter=primary
]

\begin{document}

A \gls{sample}. \lipsum*[1] A \gls{duck}.

An equation:
$$\gls[counter=equation]{pi}$$
And some more equations:
\begin{align}
A &= b\\
f(x) &= \gls[counter=equation]{pi}x
\end{align}

\lipsum[2]

Another \gls[format=primary]{sample}. \lipsum*[3] Another \gls{duck}.

\gls{pi}. \lipsum[4]

A \gls{sample}. \lipsum*[5] A \gls{duck} and \gls[format=primary]{pi}.

\lipsum*[6] A \gls[format=primary]{duck}.

\printunsrtglossaries
\end{document}


Index with different piece of information

You can use a different counter for the location using the counter option (for example, \gls[counter=equation]{pi} in the above example) or you can use a custom location for example:

\glsadd[thevalue={Supplementary Material}]{sample}


By default the locations are in the same order that they were indexed, but you can separate the locations according to the counter used when indexing. For example:

\GlsXtrLoadResources[
loc-counters={equation,page},% group locations by counter
src={entries}% data in entries.bib
]


You can provide additional information in other fields. (See the "Examples" chapter in the bib2gls manual.)

Easy setting of the tags at the output side

This can be done through the glossary style or using the hooks provided by glossaries-extra. Again, see the "Examples" chapter in the bib2gls user manual.

Mixing writing

Similarly you can provide one field for one script and another field for another script. You can use \glsaddkey provided by the base glossaries package to define the field and provide some user commands to reference the field in the document. The @dualentry entry .bib format can provide a way of indexing both forms (as two separate terms, optionally in different index/glossary lists).

Sorting engine off, indexing engine on

You can switch off sorting with sort=none, but this can cause a problem for hierarchical entries so I've used flatten in the example below to remove the hierarchy:

\documentclass{book}

\usepackage{fontspec}

\usepackage[bib2gls,manual]{testidx-glossaries}

src={testidx-glossaries-mathsym,
testidx-glossaries-markers,
testidx-glossaries-numbers,
\tstidxbasebibfiles,\tstidxutfbibfiles,\tstidxglyphfile-utf8},
selection={recorded and deps and see},
ignore-fields={description},
flatten,
sort={none}
]

\renewcommand{\glsnamefont}[1]{\textmd{#1}}

\begin{document}
\testidx

\printunsrtglossary[title={Index}]
\end{document}


This will list the entries in order of definition (in the .bib files). You can also use sort=use to list in order of first use in the document or sort=random for a random list.

Sorting engine on, indexing engine off

You can select all defined entries (regardless of whether they've been referenced in the document) using selection=all. (If any have been indexed in the document, you can use save-locations=false to omit the location lists.)