# \DeclareLanguageMappingSuffix, inheritance, and polyglossia in biblatex

I'm having trouble getting language mapping inheritance working properly in biblatex with \DeclareLanguageMappingSuffix and language variants in polyglossia.

Consider the following MWE:

\documentclass{article}

\usepackage{filecontents}
\begin{filecontents}{english-test.lbx}
\InheritBibliographyExtras{english}
\DeclareBibliographyStrings{%
inherit = {english},
in = {in \textcolor{red}{(english)}}{in \textcolor{red}{(english)}},
}
\end{filecontents}
\begin{filecontents}{american-test.lbx}
\InheritBibliographyExtras{american}
\DeclareBibliographyStrings{%
inherit = {american},
in = {in \textcolor{blue}{(american)}}{in \textcolor{blue}{(american)}},
}
\end{filecontents}

\usepackage{xcolor}
\usepackage{csquotes}
\usepackage{polyglossia}
\setmainlanguage[variant=american]{english}
%\usepackage[american]{babel}% <- works with babel
\usepackage[style=verbose]{biblatex}
\DeclareLanguageMappingSuffix{-test}

\begin{document}
\cite{westfahl:space}
\end{document}


It gives the following output (expected, since biblatex can't detect language variants with polyglossia):

If we use \usepackage[american]{babel}, we also get the expected (and desired) output:

# Question

What do I have to add to the polyglossia case to get biblatex to load both american-test.lbx and english-test.lbx?

Whatever is added should also work with babel.

## Things I've tried:

• \usepackage[style=verbose,language=american]{biblatex} (Gets overridden when polyglossia is loaded)
• \DeclareLanguageMapping{english}{american-test} (Never loads english-test.lbx)
• \DeclareLanguageMapping{english}{american} (Neither loads american-test.lbx nor english-test.lbx)
• Is 'don't use polyglossia until it has an interface to get the language variant (preferable mapped to a babel identifier)' an acceptable answer? \usepackage[language=american]{biblatex} should not solve your problem since it ideally should force american everywhere (could be broken at the moment, I'm investigating). The \DeclareLanguageMappings over language variants are to be avoided: "Note that \DeclareLanguageMapping is not intended to handle language variants or babel/polyglossia language aliases" (p. 294 of the manual). May 19 '18 at 6:22
• BTW: I'd be interested to know what else the new .lbx handling broke for you. Until now the transition has been very smooth (apart from tex.stackexchange.com/q/428264/35864 where I could argue for user error). Just open an issue at the biblatex bugtracker. May 19 '18 at 6:24
• @moewe, the problem is that biblatex-sbl (which this is for) documents almost certainly require Hebrew. And babel is not quite there for Hebrew yet :(. May 19 '18 at 6:24
• @moewe, it didn't break much. Mostly my attempts to use \DeclareLanguageMapping to deal with variants... I might ask a separate question related to defaulting to american conventions when babel/polyglossia isn't loaded, but still allowing for british when requested by babel. May 19 '18 at 6:29
• Bummer. I'm not very hopeful that anything sensible can be done until we get the polyglossia support right (which is not happening anytime soon I'm afraid - at least given the current pace of development for polyglossia). May 19 '18 at 6:31

Update All multilingual features of biblatex should work as intended with biblatex v3.14 (or above) and polyglossia v1.46 (or above).

The MWE from the question produces the expected (babel-equivalent) output now.

If you are facing any problems with polyglossia and biblatex, update your system so that you have at least biblatex 3.14 (and a matching Biber version) and polyglossia 1.46. If that does not help, please open a bug report at https://github.com/plk/biblatex/issues.

Note in particular that the bug in polyglossia (https://github.com/reutenauer/polyglossia/issues/205) that I opened when formulating this question is resolved.

This is not a full answer, but it describes why things go wrong with polyglossia and how the entire language mapping business works.

# polyglossia and biblatex

The polyglossia support in biblatex is still patchy at best. My advice would be to avoid polyglossia if at all possible and use babel instead. polyglossia used to be a must-have for certain languages (especially for languages with non-Latin scripts or RTL languages), but recent babel development has shown improvements in that matter. For other languages (Western European certainly) there is no huge advantage in choosing polyglossia over babel (and it has been that way for quite some time).

### babel's languages vs. polyglossia main language+options

The most problematic issue by far is the detection of language variants. While babel simply and naively defines a new language for each variant or dialect it encounters, polyglossia has a much more sophisticated interface that sets the main language that can then be modified with certain attributes (for example variant to turn english into american or british or german into swissgerman or austrian; and spelling to select ngerman or german).

### Variant detection

Unfortunately, the variant/spelling variants of a main language have a non-uniform implementation across languages and there is no interface for packages or classes to find out which variant is currently in use. This is a major problem for packages that also want to act upon the language variant. It is actually worse than that, in certain situations not even polyglossia knows which variant is active! Take the following contrived example.

\documentclass{article}

\usepackage{xcolor}
\usepackage{polyglossia}
\setotherlanguage{english}
\setotherlanguage{dutch}
\setmainlanguage{german}

\def\month{1}% we need january where German German and Austrian German differ
\begin{document}
\tableofcontents

\begin{english}
\section{\protect\today}
\end{english}

\section{\protect\today}

\begin{german}[variant=austrian]
\section{\protect\textcolor{red}{\protect\today}}
\end{german}

\begin{dutch}
\section{\protect\today}
\end{dutch}
\end{document}


The date changes languages in the TOC in the same way that it changes in the section headers, but it does not change the language variant. I have reported this at https://github.com/reutenauer/polyglossia/issues/205

### Finding out which language to load

biblatex mainly works with the babel model of treating languages: Every dialect is a new language (there can be inheritance between languages, but that does not really matter for the overall view). So there would first have to be a proper interface to convert polyglossia main languages+options to a babel equivalent before biblatex can even know what language to load. That conversion is a non-trivial task because there need not be a connection between the way polygossia sets up its main language with options and the name babel has for the resulting language. I believe that this can only be properly tackled on the polyglossia side, because each gloss-....ldf essentially has to declare its babel-equivalent names. A very rudimentary proof of concept can be found at https://github.com/moewew/polyglossia/tree/bblnames, the changes to the current polyglossia version are at https://github.com/reutenauer/polyglossia/compare/master...moewew:bblnames.

Even after the language file is finally loaded, the problems for biblatex don't stop. With babel the bibstrings are simply added to \extras<babel-language>. That command is called whenever the language is switched and so the correct strings are activated. But with polyglossia only \extras<base-language> exists (there is no \extrasaustrian there is only \extrasgerman), biblatex would have to compute the base language from the babel language, all variant changes would have to be made within \extras<main-language>. But as mentioned above, there is no uniform implementation of the variants, so this requires intimate knowledge of the gloss-....ldfs.

# biblatex's language mapping

Since this question is also concerned with language mapping and that changed recently (in version 3.11, see CHANGES.md), let me explain how this works now.

Whenever biblatex now tries to load an .lbx file it tries to resolve the language mapping first.

There are specific language mappings with \DeclareLanguageMapping{<language>}{<new lbx file>} and general mappings for all languages with \DeclareLanguageMappingSuffix{<suffix>} that map <language>.lbx to <language><suffix>.lbx. The specific mapping takes precedence over a general mapping.

.lbx files may inherit from other .lbx files with \InheritBibliographyStrings{<lang>}, \InheritBibliographyExtras{<lang>} or using inherit = {<lang>}, as special 'bibstring' in \DeclareBibliographyStrings. .lbx files that do not define all strings or extras themselves should always inherit what they don't define from a sensible source (see for example Problems with biblatex' \DeclareLanguageMapping after update).

When biblatex resolves the inheritance all language mappings are still applied. Only infinite recursion is avoided.

Assume we have a file german.lbx with all the definitions for German.

The file ngerman.lbx for new orthography rules inherits from german and only changes what is necessary, it looks like this (simplified and shortened)

\ProvidesFile{ngerman.lbx}

\InheritBibliographyExtras{german}

\DeclareBibliographyStrings{%
inherit  = {german},
citedas  = {{im Folgenden zitiert als}{im Folgenden zit\adddotspace als}},
}


We also have two files for \DeclareLanguageMappingSuffix{-dp}

german-dp.lbx

\ProvidesFile{german-dp.lbx}

\InheritBibliographyExtras{german}

\DeclareBibliographyStrings{%
inherit  = {german},
castle   = {{Schloß}{Schloß}},
}


ngerman-dp.lbx

\ProvidesFile{ngerman-dp.lbx}

\InheritBibliographyExtras{ngerman}

\DeclareBibliographyStrings{%
inherit  = {ngerman},
castle   = {{Schloss}{Schloss}},
}


If you now ask biblatex to get you ngerman, it will recall the language mapping we set up and open ngerman-dp.lbx instead. ngerman-dp.lbx inherits from ngerman and so ngerman.lbx is opened (we don't follow the mapping because we don't want to open ngerman-dp.lbx and land in an infinite loop). ngerman.lbx in term inherits from german, this time the mapping is followed and so german-dp.lbx is opened. german-dp.lbx inherits from german and finally german.lbx is read. The following list shows the files opened in reverse chronological order, since inherit is always the first keyword definitions are overwritten by subsequent files

• german.lbx
• german-dp.lbx
• ngerman.lbx
• ngerman-dp.lbx

For ngerman you will end up with

citedas  = {{im Folgenden zitiert als}{im Folgenden zit\adddotspace als}},% ngerman.lbx
castle   = {{Schloss}{Schloss}},% ngerman-dp.lbx


For a weird example assume now that additionally to the above

• german.lbx defines lipsum = {lorem}
• ngerman.lbx defines lipsum = {ipsum}
• german-dp.lbx defines lipsum = {dolor}
• ngerman-dp.lbx has no entry for lipsum.

What do we end up with for ngerman (which by virtue of the mapping is ngerman-dp.lbx)? The lists shows that we get 'ipsum', i.e. the definition from ngerman.lbx.

I should add that I don't have a strong inclination towards claiming that this chain of inheritance is the correct or at least expected one. But I do feel that the way to set up the .lbx files is natural and I so I am prepared to accept the resulting chain of inheritance as natural corollary.

If ngerman-dp.lbx inherited from german-dp instead we would get the following chain

• german.lbx (inherited from german-dp.lbx)
• german-dp.lbx (inherited from ngerman-dp.lbx)
• german-dp-dp.lbx (ngerman-dp.lbx inherits german-dp, this is the result of applying the suffix, the file doesn't exist and is ignored after a warning, instead german-dp.lbx is loaded)
• ngerman-dp.lbx

In particular no definition of ngerman.lbx gets read and german-dp.lbx 's lipsum = {dolor} wins. We end up with

citedas  = {{im folgenden zitiert als}{im folgenden zit\adddotspace als}},% german.lbx
castle   = {{Schloss}{Schloss}},% ngerman-dp.lbx
lipsum   = {{dolor}{dolor}},% german-dp.lbx


ngerman-dp.lbx would have to repeat all definitions from ngerman.lbx that it wants to keep.

• May 20 '18 at 10:02

Imho the most important thing when setting up language files is to get the inheritance chain correct. You want to inherit in american-test from american and from english-test but it is unclear which from both should be the direct parent: The file says american but the expected output shows that you want english-test to win at least for the pages field.

I would setup the files so that english-test is the direct parent along with a mapping to overcome the deficiencies of polyglossia.

\documentclass{article}

\usepackage{filecontents}
\begin{filecontents}{english-test.lbx}
\InheritBibliographyExtras{english}
\DeclareBibliographyStrings{%
inherit = {english},
in = {in \textcolor{red}{(english)}}{in \textcolor{red}{(english)}},
}
\end{filecontents}
\begin{filecontents}{american-test.lbx}
\InheritBibliographyExtras{english-test}
\DeclareBibliographyStrings{%
inherit = {english-test},
in = {in \textcolor{blue}{(american)}}{in \textcolor{blue}{(american)}},
}
\end{filecontents}

\usepackage{xcolor}
\usepackage{csquotes}
\usepackage{polyglossia}
\setmainlanguage[variant=american]{english}

%\usepackage[american]{babel}% <- works with babel
\usepackage[style=verbose]{biblatex}
\DeclareLanguageMappingSuffix{-test}
\DeclareLanguageMapping{english}{american-test}

• This is basically what I used to have before biblatex 3.11. However, it doesn't work now. Notice that american.lbx never gets loaded and hence \uspunctuation (and anything else in \DeclareBibliographyExtras) is not used. You end up having to include everything from american.lbx into american-test.lbx, which is not ideal. May 20 '18 at 5:54