Gotcha!
So in the preamble of our .tex source, we say
\usepackage{index}
\newindex{general}{general-idx}{general-ind}{General Index}
In the text, we define the index entries in this fashion:
\index[general]{\=an\=ap\=anasati}
\index[general]{sa\d{m}s\=ara}
\index[general]{\~n\=a\d{n}avimutti}
When we compile the .tex document, we will get our book.main.general-idx file, which we have to process with xindy.
The main problem is that texindy loads many .xdy modules before it gets to our custom style1.xdy.
We have to invoke xindy directly, with an extended style1.xdy, that only defines what we need to be defined.
So I opened the modules (/usr/share/xindy/...) which texindy walks through as reported in its output,
Opening logfile "/dev/null" (done)
Reading indexstyle...
Loading module "/tmp/1DxWUsIHQG"...
Loading module "lang/english/latin9-lang.xdy"...
Loading module "lang/english/latin9.xdy"...
Finished loading module "lang/english/latin9.xdy".
Finished loading module "lang/english/latin9-lang.xdy".
Loading module "tex/inputenc/latin.xdy"...
Finished loading module "tex/inputenc/latin.xdy".
Loading module "texindy.xdy"...
Loading module "numeric-sort.xdy"...
Finished loading module "numeric-sort.xdy".
Loading module "latex.xdy"...
Loading module "tex.xdy"...
Finished loading module "tex.xdy".
Finished loading module "latex.xdy".
Loading module "latex-loc-fmts.xdy"...
Finished loading module "latex-loc-fmts.xdy".
Loading module "makeindex.xdy"...
Finished loading module "makeindex.xdy".
Loading module "latin-lettergroups.xdy"...
Finished loading module "latin-lettergroups.xdy".
Finished loading module "texindy.xdy".
Loading module "page-ranges.xdy"...
Finished loading module "page-ranges.xdy".
Loading module "word-order.xdy"...
Finished loading module "word-order.xdy".
Loading module "lang/english/utf8.xdy"...
WARNING: define-letter-group: prefix "�" now maps to letter group "Þ"
Finished loading module "lang/english/utf8.xdy".
Loading module "book-order.xdy"...
Finished loading module "book-order.xdy".
Finished loading module "/tmp/1DxWUsIHQG".
Finished reading indexstyle.
Finalizing indexstyle... (done)
Reading raw-index "/tmp/4o3huWTPhU"...
Finished reading raw-index.
Processing index... [10%] [20%] [30%] [40%] [50%] [60%] [70%] [80%] [90%] [100%]
Finished processing index.
Writing markup... [10%] [20%] [30%] [40%] [50%] [60%] [70%] [80%] [90%] [100%]
Markup written into file "book.main.general-ind".
And copied the necessary parts to style1.xdy, extending the definitions for the Pali letters where necessary. In the file below, the latin alpabet is extended with the letters ā, ī, ṃ, ñ. There's more in Pali, but this will do for now.
style1.xdy:
;; xindy style file for an index with extended alphabet
(define-attributes (("default" "textbf" "textit" "hyperpage")))
;; "see" and "see also"
(define-crossref-class "see")
(markup-crossref-list :class "see" :open "\see{" :sep "; " :close "}{}")
(define-crossref-class "seealso")
(markup-crossref-list :class "seealso" :open "\seealso{" :sep "; " :close "}{}")
(markup-crossref-layer-list :sep ", ")
(require "base/numeric-sort.xdy")
(define-location-class "arabic-page-numbers" ("arabic-numbers"))
(define-location-class "roman-page-numbers" ("roman-numbers-lowercase"))
(define-location-class "Roman-page-numbers" ("roman-numbers-uppercase"))
(define-location-class "alpha-page-numbers" ("alpha"))
(define-location-class "Alpha-page-numbers" ("ALPHA"))
(define-location-class-order ("roman-page-numbers"
"Roman-page-numbers"
"arabic-page-numbers"
"alpha-page-numbers"
"Alpha-page-numbers"
"see"
"seealso"))
(require "lang/english/utf8.xdy")
(define-alphabet "latin-pali"
("a" "ā" "b" "c" "d" "e" "f" "g" "h" "i" "ī" "j" "k" "l" "m" "ṃ" "n" "ñ" "o" "p" "q" "r" "s" "t" "u" "v" "w" "x" "y" "z"))
(define-letter-groups
("a" "ā" "b" "c" "d" "e" "f" "g" "h" "i" "ī" "j" "k" "l" "m" "ṃ" "n" "ñ" "o" "p" "q" "r" "s" "t" "u" "v" "w" "x" "y" "z"))
;; =======================
;; merge rules
;; =======================
;; for the pali words
;; TODO: some variations of the TeX accent markups could be combined using an :eregexp instead of :string replacements
(merge-rule "\=a" "ā" :string)
(merge-rule "\={a}" "ā" :string)
(merge-rule "\=A" "Ā" :string)
(merge-rule "\={A}" "Ā" :string)
(merge-rule "\={\i}" "ī" :string)
(merge-rule "\={\i }" "ī" :string)
(merge-rule "\d{m}" "ṃ" :string)
(merge-rule "\d {m}" "ṃ" :string)
(merge-rule "\~~n" "ñ" :string)
(merge-rule "\~~ n" "ñ" :string)
;; LaTeX and TeX conventions
(merge-rule "\\LaTeXe *" "LaTeX2e" :eregexp :again)
(merge-rule "\\BibTeX *" "BibTeX" :eregexp :again)
(merge-rule "\\AmSTeX *" "AmSTeX" :eregexp :again)
(merge-rule "\\AmSLaTeX *" "AmSLaTeX" :eregexp :again)
(merge-rule "\\XeT *" "XeT" :eregexp :again)
(require "base/tex.xdy")
(markup-locref :open "\textbf{" :close "}" :attr "textbf")
(markup-locref :open "\textit{" :close "}" :attr "textit")
(markup-locref :open "\hyperpage{" :close "}" :attr "hyperpage")
;; =======================
;; sort rules
;; =======================
;; list pali capitals under pali lowercase
(sort-rule "Ā" "ā")
(sort-rule "Ṃ" "ṃ")
(sort-rule "Ī" "ī")
(sort-rule "Ñ" "ñ")
;; list latin capitals under latin lowercase
(sort-rule "A" "a")
(sort-rule "B" "b")
(sort-rule "C" "c")
(sort-rule "D" "d")
(sort-rule "E" "e")
(sort-rule "F" "f")
(sort-rule "G" "g")
(sort-rule "H" "h")
(sort-rule "I" "i")
(sort-rule "J" "j")
(sort-rule "K" "k")
(sort-rule "L" "l")
(sort-rule "M" "m")
(sort-rule "N" "n")
(sort-rule "O" "o")
(sort-rule "P" "p")
(sort-rule "Q" "q")
(sort-rule "R" "r")
(sort-rule "S" "s")
(sort-rule "T" "t")
(sort-rule "U" "u")
(sort-rule "V" "v")
(sort-rule "W" "w")
(sort-rule "X" "x")
(sort-rule "Y" "y")
(sort-rule "Z" "z")
;; ======================
;; markup rules
;; ======================
(require "base/page-ranges.xdy")
(markup-index :open
"\begin{theindex}
\providecommand*\lettergroupDefault[1]{}
\providecommand*\lettergroup[1]{%
\par\textbf{#1}\par
\nopagebreak
}
"
:close "~n~n\end{theindex}~n"
:tree)
(markup-indexentry :open "~n \item " :depth 0)
(markup-indexentry :open "~n \subitem " :depth 1)
(markup-indexentry :open "~n \subsubitem " :depth 2)
(markup-locclass-list :open ", " :sep ", ")
(markup-locref-list :sep ", ")
;; letter group markup
(markup-letter-group-list :sep "~n~n \indexspace~n")
(markup-letter-group :open-head "~n \lettergroupDefault{" :close-head "}" :group "default")
(markup-letter-group :open-head "~n \lettergroup{" :close-head "}")
If we now invoke xindy with
xindy -I latex -o book.main.general-ind -M style1 book.main.general-idx
We will get the ind file as below, with the extra lettergroups, which we can use with \printindex[general] in the appropriate place in the .tex document.
\lettergroup{a}
\item abiding, \hyperpage{33}
\item absorption
\subitem in daily activities, \hyperpage{132}
\item accomplishments
\subitem worldly, \hyperpage{66}
\indexspace
\lettergroup{ā}
\item \=acariya, \hyperpage{96}
\item \=Al\=ara K\=al\=ama, \hyperpage{51}
\indexspace
...
\lettergroup{n}
\item names, \hyperpage{21}, \hyperpage{23}, \hyperpage{183}
\item nature
\subitem existing according to, \hyperpage{69}
\item nibb\=ana, \hyperpage{32}, \hyperpage{33}
\subitem causes of, \hyperpage{76}
\subitem meaning of, \hyperpage{54}
\item nimitta, \hyperpage{41}
\indexspace
\lettergroup{ñ}
\item \~n\=ayapa\d {t}ipanno
\subitem those who practice for realisation of the path, \hyperpage{58}
...
May this be for the happiness of many TeXnicians.