More problems with TIPA and Fontspec

Question

I do quite a lot of work with Sanskrit and linguistics. For the Sanskrit (Devanagari letters) I used fontspec and render with xelatex. This, as far as I can tell, is the only way to go. For the linguistics, I use TIPA and would be loathe to give it up if only for ease of input.

Another thing that TIPA is very good for, though, is composing characters with multiple accents. The TIPA manual itself gives, as an example \textipa{\’{\"{\u*{e}}}}. It is extremely difficult to replicate this satisfactorily with plain ol' Unicode and system fonts.

Now, when transliterating Vedic Sanskrit (which I am currently trying to do) we often have to place an acute accent over a macron over a letter, for example:

Such a combination of accents is famously difficult to come by. The use of the two Unicode combining diacritical marks ́ (U+0301) and ̄ (U+0304) rarely produces good results. Unicode only contains the pre-composed character ḗ, it has not a, i, o, or u and nor is it ever likely to, as Unicode deprecates such pre-composed characters these days, I believe.

In the old days, I was able to produce the character above quite easily, using the answer to this question.

I have, however, got a new computer and, therefore, a new LaTeX "install". Today I walked into a whole new world of problems, when I received the error message:

(/usr/share/texlive/texmf-dist/tex/latex/metalogo/metalogo.sty)
(/usr/share/texmf/tex/latex/tipa/tipa.sty
(/usr/share/texlive/texmf-dist/tex/latex/base/fontenc.sty
(/usr/share/texmf/tex/latex/tipa/t3enc.def)
(/usr/share/texlive/texmf-dist/tex/latex/euenc/eu1enc.def))

! LaTeX Error: Command \sups already defined.
               Or name \end... illegal, see p.192 of the manual.

See the LaTeX manual or LaTeX Companion for explanation.
Type  H <return>  for immediate help.
 ...

l.478   \lower.8ex\hbox{\super{#2}}}{#1}}}`

I know that TIPA and fontspec aren't the best of friends, but even if there is a way around the accent problem (which prompted this question), I would still like to write documents with both of them, as TIPA is so good for IPA input and has a Computer Modern like font. It also supports all the different styles and families. Also, many of my old documents use both and if I ever need to recompile them, I shall have to make TIPA and fontspec work.

Anyway, back to the specific problem of today, here is the MWE that's causing problems:

\documentclass[12pt]{article}
\pagestyle{plain}
\usepackage[margin=1.8cm]{geometry}
\geometry{a4paper}
\usepackage[parfill]{parskip}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{fontspec,xltxtra}
\newfontfamily{\SA}[Script=Devanagari]{Chandas}
\usepackage{tipa}
\newcommand{\acm}[1]{\begin{IPA}\'{\={#1}}\end{IPA}}

\AtBeginDocument{
\renewcommand\textipa[2][r]{{\fontfamily{cm#1}\tipaencoding #2}}
}
\renewenvironment{IPA}[1][r]{\fontfamily{cm#1}\tipaencoding}{}

\begin{document}

  {\SA अग्ने॒ यं य॒ज्ञम॑ध्व॒रं वि॒श्वत॑\XeTeXglyph49{} परि॒भूरसि॑ । \\
  स इद्दे॒वेषु॑ गच्छति ॥ ४ ॥}

  \begin{enumerate}

    \setcounter{enumi}{3}

    \item
    \'{A}gne y\'{a}\d{m} yaj\~{n}\'{a}m adhvar\'{a}\d{m}
    vi\'{s}v\'{a}ta\d{h} paribh\acm{u}r \'{a}si; \\
    S\'{a} \'{i}d dev\'{e}\d{s}u gacchati.

  \end{enumerate}

\end{document}

Here we have the Sanskrit, as it should be, without the TIPA stuff:

\documentclass[12pt]{article}
\pagestyle{plain}
\usepackage[margin=1.8cm]{geometry}
\geometry{a4paper}
\usepackage[parfill]{parskip}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{fontspec,xltxtra}
\newfontfamily{\SA}[Script=Devanagari]{Chandas}

\begin{document}

  {\SA अग्ने॒ यं य॒ज्ञम॑ध्व॒रं वि॒श्वत॑\XeTeXglyph49{} परि॒भूरसि॑ । \\
  स इद्दे॒वेषु॑ गच्छति ॥ ४ ॥}

\end{document}

And here we have the transliteration, as it should be, without the Sanskrit stuff:

\documentclass[12pt]{article}
\pagestyle{plain}
\usepackage[margin=1.8cm]{geometry}
\geometry{a4paper}
\usepackage[parfill]{parskip}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{tipa}
\newcommand{\acm}[1]{\begin{IPA}\'{\={#1}}\end{IPA}}

\AtBeginDocument{
\renewcommand\textipa[2][r]{{\fontfamily{cm#1}\tipaencoding #2}}
}
\renewenvironment{IPA}[1][r]{\fontfamily{cm#1}\tipaencoding}{}

\begin{document}

  \begin{enumerate}

    \setcounter{enumi}{3}

    \item
    \'{A}gne y\'{a}\d{m} yaj\~{n}\'{a}m adhvar\'{a}\d{m}
    vi\'{s}v\'{a}ta\d{h} paribh\acm{u}r \'{a}si; \\
    S\'{a} \'{i}d dev\'{e}\d{s}u gacchati.

  \end{enumerate}

\end{document}

Can I have my cake and eat it?

As you can tell, this question was prompted by a specific problem producing a specific character. I thought it would be illuminating to include this context; but my problem is really a lot broader than that. For reasons unknown to me, TIPA will no longer play nice with fontspec - when once it worked so well. Does anybody know what's causing the problem and - more importantly to me - what the solution is?

---

If instead, per David Carlisle's suggestion, I switch the order of TIPA and fontspec and load TIPA first, the rendering process gets this far:

(/usr/share/texlive/texmf-dist/tex/latex/realscripts/realscripts.sty
*************************************************
* LaTeX warning: "xparse/redefine-command"
* 
* Redefining document command \textsubscript with arg. spec. 's' on line 25.
*************************************************
*************************************************
* LaTeX warning: "xparse/redefine-command"
* 
* Redefining document command \textsuperscript with arg. spec. 's' on line 28.
*************************************************
) (/usr/share/texlive/texmf-dist/tex/latex/metalogo/metalogo.sty))
(./TIPA_MWE.aux) (/usr/share/texmf/tex/latex/tipa/t3cmr.fd)
*geometry* driver: auto-detecting
*geometry* detected driver: xetex

And then it stops and my laptop starts making a lot of noise. A lot of noise. Eventually I have to interrupt it:

^C! Interruption.
\@ifstar ...new@ifnextchar *{\def \reserved@a *{#1
                                                  }\reserved@a }{#2}
l.28 ...a}\d{m} vi\'{s}v\'{a}ta\d{h} paribh\acm{u}
                                                  r \'{a}si; \\

So, if we follow David Carlisle's suggestion through, and use his lighter definition of \acm:

\newcommand{\acm}[1]{\={#1}\raisebox{.3ex}{\llap{\'{\mbox{}}}}}

We end up with something that does render. (Thanks very much!) This seems to render TIPA rather impotent though. Even something as simple as \begin{IPA}\={u}\end{IPA} will fail to render. This is a problem I never used to have. I was able to exploit the full functionality of TIPA for transliteration, as well as IPA input, and use fontspec to include characters from other scripts. As you can imagine, both are frequently desired in the same document. Can this still be done?

Answer 1

I'll let others take on the specific question about the clash between tipa and fontspec. But I wanted to provide an answer that would be helpful to other users who are facing the broader question of how to do Sanskrit transliteration (or phonetic transcription more generally) in XeLaTeX. This would especially apply to those who don't already have a lot of documents that use tipa; I understand that to a point, backwards compatibility is a valid consideration.

I think that in general the best way to handle diacritic stacking is to use a Unicode font that has good diacritic rendering, entering these symbols using an IPA keyboard layout.

Fonts with diacritic stacking

It is certainly true that some fonts are better than others in terms of their rendering of multiple diacritics. Here are some fonts (serif, sans serif, and monospaced) that do quite well with multiple diacritics even above capital letters and above narrow glyphs like i. These were compiled with fontspec and XeLaTeX.

Input of Unicode characters

The original question contends that it is easier to type tipa commands than to use Unicode system fonts. This varies widely depending on how you input the Unicode symbols. As described in this answer to Accessing IPA characters when using Charis SIL, there are IPA keyboard layouts available for Windows, Mac, and Linux that allow you to enter the transliterations in the MWE with fewer keystrokes than the tipa commands would require. For example, here are the keystrokes needed for some of the non-ASCII symbols in the MWE with the MSKLC IPA keyboard layout for Window vs. the tipa commands used in the MWE:

The keyboard layouts allow you to use fewer keystrokes than the tipa commands, and the shortcuts in the keyboard layouts are semantically organized (for example, & introduces under-diacritics and @ introduces over-diacritics).

Other benefits of using Unicode input

There are additional benefits of using Unicode input, as discussed at Typesetting phonetic symbols: Unicode or tipa?. For many users, code readability, cross-application compatibility, and font consistency are very significant factors to consider.

Here is the transliteration part of the MWE, with symbols entered using an IPA Unicode keyboard and compiled using XeLaTeX:

\documentclass{article}

\usepackage{fontspec}
\setmainfont{Charis SIL}

\begin{document}
\begin{enumerate}
\setcounter{enumi}{3}
\item Ágne yáṃ yajñám adhvaráṃ viśvátaḥ paribhū́r ási; \\
    Sá íd devéṣu gacchati.
\end{enumerate}
\end{document}

Answer 2

The clash on \sups is because poth packages are defining this command to do something related to TIPA, the tipa package isn't expecting that, the error message goes if you load it before fontspec

Then really you probably want to not overload too much of fontspec as the latin modern IPA characters are in unicode rather than T3 positions, which just leaves \acm for which I defined some approximation that possibly works here, except I can't read any of this, even in transliteration so It's possible something is missed completely.

I substituted the font for one I had that claimed to support this script.

\documentclass[12pt]{article}
\pagestyle{plain}

\usepackage{tipa}
\usepackage[margin=1.8cm]{geometry}
\geometry{a4paper}
\usepackage[parfill]{parskip}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{fontspec,xltxtra}
%\newfontfamily{\SA}[Script=Devanagari]{Chandas}
\newfontfamily{\SA}[Script=Devanagari]{Aparajita}



\newcommand{\acm}[1]{\={#1}\raisebox{.3ex}{\llap{\'{\mbox{}}}}}

\AtBeginDocument{
%\renewcommand\textipa[2][r]{{\fontfamily{cm#1}\tipaencoding #2}}
}
%\renewenvironment{IPA}[1][r]{\fontfamily{cm#1}\tipaencoding}{}

\begin{document}

  {\SA अग्ने॒ यं य॒ज्ञम॑ध्व॒रं वि॒श्वत॑\XeTeXglyph49{} परि॒भूरसि॑ । \\
  स इद्दे॒वेषु॑ गच्छति ॥ ४ ॥}

  \begin{enumerate}

    \setcounter{enumi}{3}

    \item
    \'{A}gne y\'{a}\d{m} yaj\~{n}\'{a}m adhvar\'{a}\d{m}
    vi\'{s}v\'{a}ta\d{h} paribh\acm{u}r \'{a}si; \\
    S\'{a} \'{i}d dev\'{e}\d{s}u gacchati.

  \end{enumerate}

\end{document}

Here we have the Sanskrit, as it should be, without the TIPA stuff:

\documentclass[12pt]{article}
\pagestyle{plain}
\usepackage[margin=1.8cm]{geometry}
\geometry{a4paper}
\usepackage[parfill]{parskip}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{fontspec,xltxtra}
%\newfontfamily{\SA}[Script=Devanagari]{Chandas}
\newfontfamily{\SA}[Script=Devanagari]{Nirmala UI}


\begin{document}

  {\SA अग्ने॒ यं य॒ज्ञम॑ध्व॒रं वि॒श्वत॑\XeTeXglyph49{} परि॒भूरसि॑ । \\
  स इद्दे॒वेषु॑ गच्छति ॥ ४ ॥}

\end{document}

Answer 3

Without a keyboard layout, or the time to select characters from a character map (plus also running on battery), mapped fonts become a feasible option.

First two lines = original text (with slightly different spelling to examples cited) pasted from https://www.sacred-texts.com/hin/rvsan/rv01001.htm

Red text: IAST-based transliteration (mapped font) of above.

Next two lines in black: plain keyboard adaption of IAST scheme (e.g., - stands for halant, long vowels and retroflexes are doubled, etc), used as input for:

Blue text: Roman to Devanagari mapped font

The map files are too big to post here (exceeding 30k char limit).

The fontspec mapping is done with:

\newfontfamily\translite[Mapping=plainkey-to-devanagari,Scale=1.1,Colour=blue]{Noto Serif Devanagari}
\newfontfamily\translitd[Mapping=devanagari-to-iastb,Scale=1.1,Colour=red]{Noto Sans}
\newfontfamily\englishfont{Noto Serif}

and usage is:

अग्ने यं यज्ञमध्वरं विश्वतः परिभूरसि |

स इद्देवेषु गछति ||

{\translitd
अग्ने यं यज्ञमध्वरं विश्वतः परिभूरसि |

स इद्देवेषु गछति ||
}

\textenglish{
Ag-ne yamh yaj-nyamadh-varamh vish-vatahh paribhuurasi | 

sa Id-devessu gachati ||
}

{\translite
Ag-ne yamh yaj-nyamadh-varamh vish-vatahh paribhuurasi | 

sa Id-devessu gachati ||
}

Devanagari is well designed. The mapping just does one-to-one, i.e., it is lossless. (Also, the plain-keyboard Roman text still retains some readability.)