# Auto replacement of characters

To make it very simple, I want to change the mapping of keyboard keys and unicode characters, but not by programming. I want to do it with LaTeX.

eg. As suggested by @Ulrike and @Phelype I've edited my MWE.

This was the code provided to me by Ulrike.

\documentclass{article}

\directlua
{
{
name = "shuffle",
type = "multiple",
data =
{
["अ"] = {"ə"},
["ब"] = {"b"},
["क"] = {"k"},
["ड"] = {"ɖ"},
},
}
}

\usepackage{fontspec}
\setmainfont{Doulos SIL}%
[
RawFeature=+shuffle,
]

\begin{document}
अबकड
\end{document}


Note - The script used here is Devanagari. The font specified is an SIL font. This kind of input will enable me to convert the data collected in local orthography to be processed in a phonetic script. The font which is used by fontspec is for the phonetic alphabet. I use XeLaTeX, so every character that I provide (with an appropriate font) looks exactly the same. I don't know how it happens in Lua.

PS - I know that my text editor can replace the characters, but I want to do it for full keyboard and not just for two or three characters. So it will be very tedious to do it for every character. Hence changing the mapping is possibly the best solution for my problem right now.

• Are you open to using LuaLaTeX?
– Mico
Jul 1, 2019 at 14:38
• I use XeLaTeX very often, have never tried Lua, but yes, if it solves my problem, I'll use lua Jul 1, 2019 at 14:41
• You need to narrow down the application a bit, I think. For example, will this be argument or environment delimited? Will the transformation apply to groups within the main text? If so, it breaks arguments that specify things like lengths, where 2pt would be converted into a nonsense unit. Likewise for color specification. If not applied to group content, then things like \textit{abc} would not be converted. So try to tell us more about how this would actually be used. Jul 1, 2019 at 14:44
• As @StevenB.Segletes commented: what is your use case exactly? Is it ok to use a command or environment, i.e., \myreplace{abcd} which is rendered as pqrs? Why do you want to do this wihtin LaTeX and not in your editor or with an external script? If you use Linux for example (or Mac or Cygwin in Windows) you can use the tr command in the terminal, or sed or awk or perl etc. etc. to replace everything with a single command. Jul 1, 2019 at 14:58
• @Niranjan - Then you should be using Ulrike's answer.
– Mico
Jul 1, 2019 at 15:31

OK, it's not quite ready for release yet, but I will use this opportunity to introduce the upcoming tokcycle package (UPDATE: tokcycle package V1.0 has now been released on CTAN, https://www.ctan.org/pkg/tokcycle, released 2019/08/21). It helps you to build tools to process tokens from an input stream. The idea here is that if you can build a macro to process an arbitrary single (non-macro, non-space) token, then tokcycle can provide a wrapper for processing an input stream on a token-by-token basis, using your provided macro.

UPDATE: Christian Tellechea has offered valuable insights into how this package may be improved, and so I am spending more time to implement as many of these improvements as I am able. UPDATE: I have listed Christian as a contributor on the package. He provided bits of code and inspired me to try to make the parsing phase of tokcycle as general as possible.

The package approach is to categorize what comes next in the input stream as either a Character, Group, Macro, or Space. Your job in creating the token-cycle is to specify the LaTeX directives on how to handle each of those four possibilities.

The package provides tools to help you build those directives, whose function it is to process the token stream and place the processed tokens in the output-stream, which is best constituted as a token register (provided by the package as \cytoks).

So let us take the problem at hand. I need to build a macro that can take a single Character token input and provide a mapping to a different character (in a different font). Here is the expandable code I propose for this:

\def\tcmapto#1#2{\expandafter\def\csname tcmapto#1\endcsname{#2}}
\def\tcremap#1{\ifcsname tcmapto#1\endcsname
\csname tcmapto#1\endcsname\else#1\fi}

\tcmapto अP
\tcmapto बQ
\tcmapto कR
\tcmapto डS


The mapping need not be to a single token. For example, \tcmapto ब{$\alpha$} is a valid mapping. The macro \tcremap basically says, if I find a remap, use it, otherwise just output the original token. I provide a remap of 4 tokens as shown above.

So now let's get to the tokcycle syntax. It provides a Plain-TeX supported syntax (tokcycle.tex) of macros \tokcycle and \expandedtokcycle and pseudo-environment \tokencycle...\endtokencycle.

The expanded version applies \expanded to the input stream before tokcycle processing (macros can be cordoned off with \noexpand).

Also, it supports xpress versions of these macros/environments, so that repeated invocations can use the most recently specified directives, rather than having to retype the directives each time.

Finally, there is the means provided, via \tokcycleenvironment<\environmentname>{}{}{}{} to create a more permanent environment with its directives locked in place.

For this MWE (using LuaLaTeX), we will use

\tokencycle
{<Character directive>}
{<Group directive>}
{<Macro directive>}
{<Space directive>}%
<token input stream>
\endtokencycle


Now for the code. First, the code to address the OP's problem. It makes use, in the Character directive, of the new \expanded TeX primitive, in the form of the macro \addcytoks[x]{\tcremap{#1}}, which will fully expand the remap before appending the result to the \cytoks token register. If your engine still does not support \expanded, you can for this case replace that macro with \edef\tmp{\tcremap{#1}}\addcytoks[1]{\tmp}.

The other directives are just the package defaults: Group Content is recommitted to the token cycle on a per token basis (\processtoks), macros are echoed literally into the output stream (\addcytoks), as are spaces.

\documentclass{article}
\usepackage{tokcycle}
\def\tcmapto#1#2{\expandafter\def\csname tcmapto#1\endcsname{#2}}
\def\tcremap#1{\ifcsname tcmapto#1\endcsname
\csname tcmapto#1\endcsname\else#1\fi}
\tcmapto अP
\tcmapto बQ
\tcmapto कR
\tcmapto डS
\begin{document}
%अबकड
\verb|\tcremap| handles a single token: \tcremap{अ}.

\verb|\tokencycle|  and \verb|\tokcycle| handle a stream of
such tokens, including embedded macros.

\noindent\hrulefill

PSEUDO ENVIRONMENT

\tokencycle
{\processtoks{#1}}
अबकड डड  \textit{बकअ} कड.

Other text for which no mapping is yet given as of \today.

अबक done.
\endtokencycle
\end{document}


• Excellent. I was wondering how you can give such detailed answers and how long it takes to do this. My best regards. Jul 1, 2019 at 19:37
• @Sebastiano I've been working on this about 5 weeks. Jul 1, 2019 at 19:59
• Thank you so much. This is a great idea. Jul 2, 2019 at 4:27
• @Sebastiano see ctan.org/tex-archive/macros/generic/tokcycle for package release. Aug 22, 2019 at 11:01
• @Sebastiano I guess ctan.org/pkg/tokcycle is what I meant. Aug 22, 2019 at 11:36

with lualatex:

\documentclass{article}
\directlua
{
{
name = "shuffle",
type = "multiple",
data =
{
["a"] = {"p"},
["b"] = {"r"},
["c"] = {"s"},
["d"] = {"t"},
},
}
}

\usepackage{fontspec}
\setmainfont{texgyreheros}%
[
RawFeature=+shuffle,
]

\begin{document}
abcd
\end{document}


• Alternatively you can use type=substitution as mentioned in the fontspec manual (page 57) - I'm not sure if there is any difference. Jul 1, 2019 at 15:16
• Hello. Thanks for the help, but I am getting following errors. Undefined control sequence. \directlua Missing \begin{document}. f fontspec error: "font-not-found" The font "texgyreheros" cannot be found. fontspec error: "font-not-found" The font "texgyreheros" cannot be found. fontspec error: "font-not-found" The font "texgyreheros" cannot be found. Font TU/texgyreheros(0)/m/n/10=texgyreheros at 10.0pt not loadable: Metric (TFM) file or installed font not found. ] Overfull \hbox (0.39941pt too wide) in paragraph Jul 1, 2019 at 15:24
• well if \directlua is not defined that you didn't use lualatex (but probably xelatex). Jul 1, 2019 at 15:26
• @Niranjan if your question is about transliteration: with xelatex you could also use a teckit mapping. Simply search the site, you will find a number of answers. Jul 1, 2019 at 15:39
• Please see the edit. Jul 1, 2019 at 15:47

You can make the characters you want to remap active and redefine them to print some other characters. However if you make, say, a active, then \newcommand (for example) won't work anymore because TeX will understand it as \newcomm and, where a gets replaced by a p.

I defined a command \remap which takes to characters and makes the first one print the second one, and another \remapchars command which should contain the \remap instructions and then typesets the argument with the new settings and reverts them afterwards to avoid problems. Do not use \remap “on the loose”. You've been warned. here you go:

\documentclass{article}

\newcommand\remap[2]{%
\catcode#1=\active
\begingroup
\lccode~=#1%
\lowercase{\endgroup\def~{\char#2}}\ignorespaces}
\newcommand\remapchars{%
\begingroup
\remap ap
\remap bq
\remap cr
\remap ds
\innerremap}
\newcommand\innerremap[1]{#1\endgroup}

\begin{document}

abcd'\remapchars{abcd'}abcd'

\end{document}

• It worked very nicely, this might be 'the' solution I am looking for, but it has not completely solved my problem yet. Right now I want to do this with an Indian script Devanagari. It uses inscript keyboard. I purposely omitted these details, because I found it unnecessary in the minimality principles. Unfortunately if I use your code to replace devanagari characters by roman, it does not work, what do you think is the reason? Jul 1, 2019 at 15:05
• @Niranjan Sorry, I have no idea about Devanagari. I did a test remapping देp, वq, नाr, गs, and रीt, and wrote \remapchars{देवनागरी'} (test file) but it came out as p r t ‘q s ’. As I said, I have no idea about Devanagari, but to me this looks wrong. Can you post an example with the desired output, please? Jul 1, 2019 at 15:13
• @Niranjan Hm... Apparently some of those characters have combining diacritics. The code however only accepts single characters. With \remap दp \remap वq \remap नr \remap गs \remap रt and \remapchars{दवनगर'}` it works. (I think I've probably just slaughtered the Devanagari script...) Jul 1, 2019 at 15:20
• Oh ok. Let me add the exact input and output in my question. Jul 1, 2019 at 15:22
• Please see the edit Jul 1, 2019 at 15:47