# BibLateX/Biber is incorrectly sorting entries with hyphens in their respective author's last name

UPDATE 2

I now finally understand that this behaviour is indeed a feature that is to be modified according to need. Thank you very much to @moewe for educating me on this. I have now set his/her answer as the accepted answer as it is much more complete than the previous one. In summary, Biber allows you to fine-tune the behaviour of sorting. Please look at the accepted answer to this question for more details and pointers.

UPDATE 1

As per page 22 of the Biber documentation, I can somewhat confirm the explanations of Benedikt and Werner. It does seem that the characters before the hyphen are considered as prefixes. But here's where it gets more interesting - this only happens when there are exactly two characters preceding the hyphen. I confirmed this by trying 1 character (A-Masri, D-Masri), 2 characters (Al-Masri, Ax-Masri), and 3-characters (Ala-Masri). The sorting works correctly for the 1 character and 3 character trials. It seems to me that there is no real practical fix to the engine code. Thus, my recommendation (for now) to others facing this issue is to rely on the sortname workaround.

ORIGINAL POST

I am observing incorrect behaviour in the sorting (by author name) of my bibliography entries. I have an entry whose main author's last name (the sorting key) contains a hyphen. This seems to cause the engine to discard everything up to the hyphen as part of the key.

Here is a MWE:

\documentclass{article}

\begin{filecontents}{references.bib}
@inproceedings{maleshkova2010investigating,
title={Investigating web apis on the world wide web},
author={Maleshkova, Maria and Pedrinaci, Carlos and Domingue, John},
booktitle={Web Services (ECOWS), 2010 IEEE 8th European Conference on},
pages={107--114},
year={2010},
organization={IEEE}
}

@inproceedings{al2007wsce,
title={WSCE: A crawler engine for large-scale discovery of web services},
author={Al-Masri, Eyhab and Mahmoud, Qusay H.},  % Notice the name here
booktitle={Web Services, 2007. ICWS 2007. IEEE International Conference on},
pages={1104--1111},
year={2007},
organization={IEEE}
}
\end{filecontents}

\usepackage[backend=biber,style=authoryear,sorting=nyt]{biblatex}
\nocite{*}

% ========= %

\begin{document}

\printbibliography

\end{document}


This results in the following (incorrect) output:

I discovered that removing the hyphen in Al-Masri rectifies the sorting. So it seems that with the hyphen the engine thinks the name is Masri, which is incorrect.

Is this a bug or a feature? If the former, how can this be fixed? If the latter, how can I work around it?

-
Consider this a feature, as you can use sortname = {...} to specify a specific way of sorting the entry in question. For example, try adding sortname = {AlMasri} to the entry (see p 26 of the biblatex documentation). – Werner Apr 14 '14 at 20:27
I feel you may be (only slightly) misunderstanding my question about bug vs. feature. While the sortname field is an excellent workaround (as per the suggestions of Benedikt Bauer and yourself; and thanks for the pointer to the documentation), the real problem is with the hyphen. As per ASCII, the hyphen is lexicographically lower in order than the English alphabets (am I even correct?). Hence, the Al-Masri entry should appear before the Maleshkova entry. Benedikt's explanation of Biber treating the part before the hyphen as a prefix makes sense. Do you have any comment on this, any chance? – dvijayak Apr 14 '14 at 20:49
See p 22 of the biber documentation where it mentions something about constructing hyphenated names. As such, I think biblatex desides sort based on M rather than A, and passes this information to biber. – Werner Apr 14 '14 at 21:04
Bingo, I would have never found that section by myself. Thank you very much. Up-voted. – dvijayak Apr 14 '14 at 21:09
@Werner I don't think that solution will work in general even though it will in this case. I'm not sure what the rules are but I'm pretty sure that Al-Masri ought not be sorted identically to Almasri which is what that workaround does. That page of the manual suggests that by default biber should not be ignoring Al- because it is only meant to ignore lowercase prefixes. – cfr Apr 14 '14 at 21:12

As pointed out in your edited question this is not so much a bug, but a feature (luckily though for us, it is a feature that can be controlled and customised).

Biber allows for sorting exceptions as well as for exceptions when creating initials. These exceptions can be declared with \DeclareNosort{} and \DeclareNoinit{} respectively.

If the commands are not present, Biber will default to

\DeclareNosort{
% strip prefixes like 'al-' when sorting names
\nosort{type_names}{\regexp{\A\p{L}{2}\p{Pd}}}
% strip diacritics when sorting names
\nosort{type_names}{\regexp{[\x{2bf}\x{2018}]}}
}

\DeclareNoinit{
% strip lowercase prefixes like 'al-' when generating initials from names
\noinit{\regexp{\b\p{Ll}{2}\p{Pd}}}
% strip diacritics when generating initials from names
\noinit{\regexp{[\x{2bf}\x{2018}]}}
}


As explained in the comments there, the first line will strip al-/Al- prefixes (the regex matches two letters at the begin of a string followed by a punctuation mark), so we can just get rid of those lines to obtain

\DeclareNosort{
% strip diacritics when sorting names
\nosort{type_names}{\regexp{[\x{2bf}\x{2018}]}}
}

\DeclareNoinit{
% strip diacritics when generating initials from names
\noinit{\regexp{[\x{2bf}\x{2018}]}}
}


This will sort the list as expected, as "al-"/"Al-" prefixes are no longer ignored, furthermore they will also be considered for initials, if you do not want that, just leave out the \DeclareNoinit{} command all together.

See also §4.5.6 Controlling Name Initials Generation and §4.5.7 Fine Tuning Sorting, p. 164-166 of the biblatex documentation.

If you do not want to put this line in your preamble whenever you write a document, you might consider biblatex.cfg or a Biber config file (more about that in the Biber documentation, §3.1 Options and config file, p. 9)

MWE

\documentclass{article}
\begin{filecontents}{\jobname.bib}
@inproceedings{maleshkova2010investigating,
title={Investigating web apis on the world wide web},
author={Maleshkova, Maria and Pedrinaci, Carlos and Domingue, John},
booktitle={Web Services (ECOWS), 2010 IEEE 8th European Conference on},
pages={107--114},
year={2010},
organization={IEEE}
}

@inproceedings{al2007wsce,
title={WSCE: A crawler engine for large-scale discovery of web services},
author={Al-Masri, Eyhab and Mahmoud, Qusay H.},  % Notice the name here
booktitle={Web Services, 2007. ICWS 2007. IEEE International Conference on},
pages={1104--1111},
year={2007},
organization={IEEE}
}
\end{filecontents}

\usepackage[backend=biber,style=authoryear,sorting=nyt]{biblatex}
\nocite{*}

\DeclareNosort{
% strip diacritics when sorting names
\nosort{type_names}{\regexp{[\x{2bf}\x{2018}]}}
}

\DeclareNoinit{
% strip diacritics when generating initials from names
\noinit{\regexp{[\x{2bf}\x{2018}]}}
}

\begin{document}
\printbibliography
\end{document}


-
This is excellent - it completes the puzzle. I have now accepted your answer. – dvijayak Apr 15 '14 at 5:49

As an explanation, I can only imagine, that biber or biblatex are treating the "al" part of the name as some kind of name prefix, similar to the "de" in dutch names.

I am quite sure to remember that there is a more global option as well, but I don't have the time now to work it out. As a simple workaround for a single name, you can just give the respective item a sortname field. Try the following entry:

@inproceedings{al2007wsce,
title={WSCE: A crawler engine for large-scale discovery of web services},
author={Al-Masri, Eyhab and Mahmoud, Qusay H.},
booktitle={Web Services, 2007. ICWS 2007. IEEE International Conference on},
pages={1104--1111},
year={2007},
organization={IEEE},
sortname = {Almasri}, % This field will be used for sorting
}

-