6

I want to define a command that will take a name and create a readable string from it, that can be used as a filename. Something like

John Doe -> doe-john.

However, I want to avoid special characters, i.e. replace non-ascii characters by their ascii equivalent. That is, I need a macro that will remove accents and replace some special letters, e.g.

æüßéñ -> aeuessen.

Is there a way to do this in LaTeX with reasonable effort?

  • the amount of effort really is proportional to how much input you want to support. it's easy enough to define a function that allows you to specify what to do with each character, but then do you want to map a few dozen european accented characters, or also transliterate cyrillic and greek and a few thousand asian characters and ... – David Carlisle Sep 25 '17 at 21:58
  • @DavidCarlisle, I don't need to go beyond european alphabets, but I feel like that list is quite long as it is. I was hoping for some way to just strip any accents off a given character (without needing to resort to a list of every possibility). If I need to consider replacements like umlauts separately (with a list), that's fine, i guess. – schtandard Sep 25 '17 at 22:03
  • unicode isn't organised in a way that makes stripping accents possible other than a lookup table (although of course in some languages that table may be in some standard string library, but tex you need to do it by hand) I see egreg's posted something (not how I'd have done it but either way you need to make a long list of replacements) – David Carlisle Sep 25 '17 at 22:05
  • If you're using inputenc (rather than XeLaTeX/LuaLaTeX) then you could let \IeC and all the accent commands to \@firstofone. (The glossaries package's \glsnoidxstripaccents does this to strip accents from the sort values when using TeX to sort.) – Nicola Talbot Sep 25 '17 at 22:53
  • @NicolaTalbot that's not a bad idea, you should post an answer, it may prevent egreg getting a tick, which is always a good thing. – David Carlisle Sep 26 '17 at 7:56
9

If you are using inputenc (rather than XeLaTeX/LuaLaTeX) you can take advantage of the fact that inputenc turns the extended characters into accent commands. For example, ü expands to \IeC{\"u}. So you can temporarily redefine the accent commands to strip them out.

Example:

\documentclass{article}
\usepackage[utf8]{inputenc}

\makeatletter
\newcommand{\stripaccents}[2]{%
 \begingroup
  % strip accents:
  \let\add@accent\@secondoftwo
  % provide replacement strings:
  \def\AE{AE}%
  \def\ae{ae}%
  \def\OE{OE}%
  \def\oe{oe}%
  \def\AA{AA}%
  \def\aa{aa}%
  \def\L{L}%
  \def\l{l}%
  \def\O{O}%
  \def\o{o}%
  \def\SS{SS}%
  \def\ss{ss}%
  \def\th{th}%
  \def\TH{TH}%
  \def\dh{dh}%
  \def\DH{DH}%
  \xdef#1{#2}%
 \endgroup
}
\makeatother

\begin{document}
\stripaccents\tmp{æüßéñ}
\show\tmp
\end{document}

This shows:

> \tmp=macro:
->aeussen.

If you have any other commands that are likely to occur in your input, you'll need to add them to \stripaccents so that they expand into something sensible.

For the umlauts, you could temporarily redefine \" so that it appends e to its argument:

\newcommand{\stripaccents}[2]{%
 \begingroup
  \def\"##1{##1e}% umlaut
  \let\add@accent\@secondoftwo
  \def\AE{AE}%
  \def\ae{ae}%
  \def\OE{OE}%
  \def\oe{oe}%
  \def\AA{AA}%
  \def\aa{aa}%
  \def\L{L}%
  \def\l{l}%
  \def\O{O}%
  \def\o{o}%
  \def\SS{SS}%
  \def\ss{ss}%
  \def\th{th}%
  \def\TH{TH}%
  \def\dh{dh}%
  \def\DH{DH}%
  \xdef#1{#2}%
 \endgroup
}

This now shows:

> \tmp=macro:
->aeuessen.

With T1 encoding you also need:

\let\@text@composite@x\@secondoftwo

in the definition of \stripaccents, as mentioned in your comment.

  • 1
    This is what I was hoping for! I am also using \usepackage[T1]{fontenc} which breaks this solution. However, adding \let\@text@composite@x\@secondoftwo to the definition of \stripaccents rectifies this. May be worth adding to the answer. – schtandard Sep 26 '17 at 15:34
  • @schtandard Sorry, I should have checked with T1. I'll add it in. – Nicola Talbot Sep 26 '17 at 15:39
  • One more thing I noticed: fontenc expands å to \r a, not \aa. Thus, instead of \AA and \aa we have to redefine \r. – schtandard Sep 26 '17 at 15:46
  • @schtandard Perhaps the best definition would be \def\r##1{##1##1} assuming that the ring accent indicates a long vowel. (If not, then a more complex definition would be required that expands the argument to something appropriate. For example, if \r a needs to be aa but \r y needs to be uo.) – Nicola Talbot Sep 26 '17 at 15:56
  • agreed. \def\r##1{##1##1} should be sufficient for my needs. – schtandard Sep 26 '17 at 17:07
4

You have to populate the list yourself, according to the given examples.

\documentclass{article}
\usepackage{xparse}

\ExplSyntaxOn
\seq_new:N \g_schtandard_search_replace_seq

\seq_gput_right:Nn \g_schtandard_search_replace_seq { {æ}{ae} }
\seq_gput_right:Nn \g_schtandard_search_replace_seq { {ä}{ae} }
\seq_gput_right:Nn \g_schtandard_search_replace_seq { {ö}{oe} }
\seq_gput_right:Nn \g_schtandard_search_replace_seq { {ü}{ue} }
\seq_gput_right:Nn \g_schtandard_search_replace_seq { {ß}{ss} }
\seq_gput_right:Nn \g_schtandard_search_replace_seq { {ñ}{n} }
\seq_gput_right:Nn \g_schtandard_search_replace_seq { {é}{e} }

\tl_new:N \l_schtandard_input_tl

\NewDocumentCommand{\makestring}{om}
 {
  \tl_set:Nn \l_schtandard_input_tl { #2 }
  \seq_map_inline:Nn \g_schtandard_search_replace_seq
   {
    \regex_replace_all:nnN ##1 \l_schtandard_input_tl
   }
  \IfNoValueTF{#1}
   {
    \tl_use:N \l_schtandard_input_tl
   }
   {
    \tl_set_eq:NN #1 \l_schtandard_input_tl
   }
 }
\ExplSyntaxOff

\begin{document}

\makestring{æüßéñ}

\makestring[\foo]{æüßéñ}\texttt{\meaning\foo}

\end{document}

enter image description here

  • And there I thought the days of me not knowing most of the commands in an answer where over... Guess it's time to learn expl3. – schtandard Sep 25 '17 at 22:15
3

If you only need filenames, but do not need them to be "human readable", then you could take advantage of \pdfstringdef

\documentclass{article}

\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage[unicode]{hyperref}

\makeatletter
\begingroup
\catcode`| 0 \catcode`\\ 12
|gdef|makestring@i\#1#2#3#4%
     {#1#2#3|if|relax#4|expandafter|@gobbletwo|fi|makestring@i#4}
|endgroup
\newcommand*{\makestring}[2]{%
   \pdfstringdef\makestring@{#2}%
   \edef#1{\expandafter\makestring@i\makestring@\relax}%
}
\makeatother



\begin{document}

\makestring{\foo}{æüßéñ}

\texttt{\meaning\foo}

\end{document}

enter image description here


A variation on this theme which is much more efficient, it show the utf8 bytes. One could produce in hexadecimal if desired. (in fact there are possibly macros in utf8.def which could be used here)

\documentclass{article}

\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}

\makeatletter
\newcommand*\MakeString[2]{%
   \begingroup
    \def\UTFviii@two@octets##1##2{\the\numexpr`##1\relax\the\numexpr`##2}%
    \def\UTFviii@three@octets##1##2##3{\the\numexpr`##1\relax\the\numexpr`##2\relax\the\numexpr`##3\relax}%
    \def\UTFviii@four@octets##1##2##3##4{\the\numexpr`##1\relax\the\numexpr`##2\relax\the\numexpr`##3\relax\the\numexpr`##4\relax}%
   \xdef#1{#2}%
  \endgroup
}
\makeatother

\begin{document}

\MakeString{\foo}{æüßéñ}

\texttt{\meaning\foo}

\show\foo
\end{document}

Produces:

> \foo=macro:
->195166195188195159195169195177.
l.23 \show\foo

I should improve so that each byte produce a three-digits decimal, here leading zeros are stripped!


Ok here it is with no stripping and 2-hex digits per byte.

edit removed usage of extra package. Defined \Byte@tohex macro possibly already provided by utf8-inputenc internally, not checked.

\documentclass{article}

\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}

\makeatletter
% I have not checked but maybe utf8-inputenc provides already
% similar macro (not even using e-TeX)
\def\Byte@tohex #1%
   {\expandafter
    \Byte@tohex@\the\numexpr(`#1+8)/16-1\expandafter
    .\the\numexpr`#1.}%
\def\Byte@tohex@ #1.#2.%
   {\Byte@onehex #1.%
    \expandafter\Byte@onehex\the\numexpr #2-16*#1.%
   }
\def\Byte@onehex #1.%
   {\ifcase #1
    0\or1\or2\or3\or4\or5\or6\or7\or8\or9%
     \or A\or B\or C\or D\or E\or F%
   \fi
   }%
\newcommand*\MakeString[2]{%
   \begingroup
    \def\UTFviii@two@octets##1##2{\Byte@tohex{##1}\Byte@tohex{##2}}%
    \def\UTFviii@three@octets##1##2##3{\Byte@tohex{##1}\Byte@tohex{##2}\Byte@tohex{##3}}%
    \def\UTFviii@four@octets##1##2##3##4{\Byte@tohex{##1}\Byte@tohex{##2}\Byte@tohex{##3}\Byte@tohex{##4}}%
   \xdef#1{#2}%
  \endgroup
}
\makeatother
\begin{document}

\MakeString{\foo}{æüßéñ}

\texttt{\meaning\foo}

\show\foo
\end{document}

produces in log

> \foo=macro:
->C3A6C3BCC39FC3A9C3B1.
l.27 \show\foo
  • I see now OP said create a readable string but I am not sure if that meant create a recognizable string this is why I am posting this nevertheless. – user4686 Sep 26 '17 at 8:53
  • of course this is also a recognizable string for the happy ones versed in octal sequences and UTF-16BE. – user4686 Sep 26 '17 at 8:58
  • @StevenB.Segletes I am only the lazy guy... I should rather imitate utf8-inputenc but use it to produce an octal or hex sequence straight from the utf8-encoded source. The \pdfstringdef is rather big macro and makes many things like so needed here. But it is time for Lunch! – user4686 Sep 26 '17 at 10:13
  • 1
    "Choose a Lazy Person To Do a Hard Job Because That Person Will Find an Easy Way To Do It" (quoteinvestigator.com/2014/02/26/lazy-job) – Steven B. Segletes Sep 26 '17 at 11:44
  • 1
    Why not just \pdfescapehex the detokenized version of the string? \message{\pdfescapehex\expandafter{\detokenize{éçß☺}}} – Bruno Le Floch Sep 26 '17 at 16:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.