15

I want to escape URL paths to obtain the percent-escaping that is also applied e.g. by browsers. For this purpose I tried to do the following (out-commented lines are meant to show what I'm aiming here):

\documentclass[a5paper]{article}
\usepackage{xstring}

\begin{document}

\newcommand{\urlescape}[1]{{%
  \def\x{\StrSubstitute{#1}{\%}{\%25}}%
  \def\x{\StrSubstitute{\x}{/}{\%2F}}%
  % \def\x{\StrSubstitute{\x}{\&}{\%26}}%
  % \def\x{\StrSubstitute{\x}{ }{\%20}}%
  % \def\x{\StrSubstitute{\x}{\$}{\%24}}%
  % \def\x{\StrSubstitute{\x}{+}{\%2b}}%
  % \def\x{\StrSubstitute{\x}{,}{\%2c}}%
  % \def\x{\StrSubstitute{\x}{:}{\%3a}}%
  % \def\x{\StrSubstitute{\x}{;}{\%3b}}%
  % \edef\x{\StrSubstitute{\x}{=}{\%3d}}%
  % \def\x{\StrSubstitute{\x}{?}{\%3f}}%
  % \def\x{\StrSubstitute{\x}{@}{\%40}}%
  % \def\x{\StrSubstitute{\x}{"}{\%22}}%
  % \def\x{\StrSubstitute{\x}{<}{\%3c}}%
  % \def\x{\StrSubstitute{\x}{>}{\%3e}}%
  % \def\x{\StrSubstitute{\x}{\#}{\%23}}%
  % \def\x{\StrSubstitute{\x}{\{}{\%7b}}%
  % \def\x{\StrSubstitute{\x}{\}}{\%7d}}%
  % \def\x{\StrSubstitute{\x}{|}{\%7c}}%
  % \def\x{\StrSubstitute{\x}{\^}{\%5e}}%
  % \def\x{\StrSubstitute{\x}{\~}{\%7e}}%
  % \def\x{\StrSubstitute{\x}{[}{\%5b}}%
  % \def\x{\StrSubstitute{\x}{]}{\%5d}}%
  % \def\x{\StrSubstitute{\x}{\`}{\%60}}%
  \x}}

\urlescape{abcd/efgh\%foo\#bar baz}

\end{document}

Error messages

However, that wouldn't work; I get

! Use of \@xs@StrSubstitute@@ doesn't match its definition.
\kernel@ifnextchar ...rved@d =#1\def \reserved@a {
                                                  #2}\def \reserved@b {#3}\f...
l.35 \urlescape{abcd/efgh\%foo\#bar baz}

So I thought well there's that strange \edef thingie from the TeX book so let's give it a try:

\newcommand{\urlescape}[1]{{%
  \def\x{\StrSubstitute{#1}{\%}{\%25}}%
  \edef\x{\StrSubstitute{\x}{/}{\%2F}}%
...

But now I get the following which confuses me:

! Argument of \reserved@a has an extra }.
<inserted text>
                \par
l.35 \urlescape{abcd/efgh\%foo\#bar baz}

Solution and question I searched and I found several hints where intricate combinations of magic potions à la \noexpand, \unexpanded got thrown into the mix; sadly, I was unable to extract from those solutions an incantation that would work for me (I've probably made it sufficiently clear by now that using TeX is more of an art than a science to me).

Please do not hesitate to point out a package to do just this — URL escaping — but I think that the question of how to perform multiple replacements on strings in LaTeX is of a more general interest and I'm sure I'll need stuff like that quite often.

Update

Explanation: The question has been raised as the above list is so extensive; the simple answer is that I copy-and-pasted these characters from http://perishablepress.com/stop-using-unsafe-characters-in-urls/.

In a more specific use-case, it will be possible and probably desirable to cut down the enumeration, depending on semantics and environment. as for semantics, it will normally make a difference whether you retrieve http://example.com/foo/bar#baz or http://example.com/foo%2Fbar%23baz; whether or not to escape / and/or # in such URLs is, as such, unpredictable, so I started out with a fairly maximal list.

  • Well, \def\x{\StrSubstitute{\x}{...} doesn't really seem a good start. – egreg Jan 9 '14 at 18:01
  • think of \def\a{\a}... – percusse Jan 9 '14 at 18:01
  • See section 2.2.5 \StrSubstitute of the xstring documentation. Using the optional (last) argument might be useful. – Werner Jan 9 '14 at 18:01
  • Doesn't hyperref or one of Heiko Oberdiek's packages cover this anyway? – Joseph Wright Jan 9 '14 at 19:53
  • @JosephWright i checked this and could not find anything in it – flow Jan 10 '14 at 13:57
15

You are defining \x in terms of itself, which is not a good thing to do. You rather want to store the intermediate result in a temporary macro, which xstring allows to do with the trailing optional argument.

Store the argument after doing the first substitution and then act on the stored string. There's also a problem with expansion that has to be taken care of: the best is to suppress xstring expansions and do only one manually:

\documentclass[a5paper]{article}
\usepackage{xstring}

\newcommand{\urlescape}[1]{{%
  \noexpandarg % suppress expansions made by xstring
  \StrSubstitute{#1}{\%}{\%25}[\x]% first step
  \expandafter\StrSubstitute\expandafter{\x}{/}{\%2F}[\x]%
  \expandafter\StrSubstitute\expandafter{\x}{\&}{\%26}[\x]%
  \expandafter\StrSubstitute\expandafter{\x}{ }{\%20}[\x]%
  \expandafter\StrSubstitute\expandafter{\x}{\$}{\%24}[\x]%
  \expandafter\StrSubstitute\expandafter{\x}{+}{\%2b}[\x]%
  \expandafter\StrSubstitute\expandafter{\x}{,}{\%2c}[\x]%
  \expandafter\StrSubstitute\expandafter{\x}{:}{\%3a}[\x]%
  \expandafter\StrSubstitute\expandafter{\x}{;}{\%3b}[\x]%
  \expandafter\StrSubstitute\expandafter{\x}{?}{\%3f}[\x]%
  \expandafter\StrSubstitute\expandafter{\x}{@}{\%40}[\x]%
  \expandafter\StrSubstitute\expandafter{\x}{"}{\%22}[\x]%
  \expandafter\StrSubstitute\expandafter{\x}{<}{\%3c}[\x]%
  \expandafter\StrSubstitute\expandafter{\x}{>}{\%3e}[\x]%
  \expandafter\StrSubstitute\expandafter{\x}{\#}{\%23}[\x]%
  \expandafter\StrSubstitute\expandafter{\x}{\{}{\%7b}[\x]%
  \expandafter\StrSubstitute\expandafter{\x}{\}}{\%7d}[\x]%
  \expandafter\StrSubstitute\expandafter{\x}{|}{\%7c}[\x]%
  \expandafter\StrSubstitute\expandafter{\x}{\^}{\%5e}[\x]%
  \expandafter\StrSubstitute\expandafter{\x}{\~}{\%7e}[\x]%
  \expandafter\StrSubstitute\expandafter{\x}{[}{\%5b}[\x]%
  \expandafter\StrSubstitute\expandafter{\x}{]}{\%5d}[\x]%
  \expandafter\StrSubstitute\expandafter{\x}{\`}{\%60}[\x]%
  \x}}

\begin{document}

\urlescape{abcd/efgh\%foo\#bar baz}

\end{document}

enter image description here

Some syntactic sugar may be better:

\documentclass[a5paper]{article}
\usepackage{xstring}

\newcommand{\urlescapestep}[2]{%
  \expandafter\StrSubstitute\expandafter{\x}{#1}{#2}[\x]%
}
\newcommand{\urlescape}[1]{{%
  \noexpandarg % suppress expansions made by xstring
  \StrSubstitute{#1}{\%}{\%25}[\x]% first step
  \urlescapestep{/}{\%2F}%
  \urlescapestep{\&}{\%26}%
  \urlescapestep{ }{\%20}%
  \urlescapestep{\$}{\%24}%
  \urlescapestep{+}{\%2b}%
  \urlescapestep{,}{\%2c}%
  \urlescapestep{:}{\%3a}%
  \urlescapestep{;}{\%3b}%
  \urlescapestep{?}{\%3f}%
  \urlescapestep{@}{\%40}%
  \urlescapestep{"}{\%22}%
  \urlescapestep{<}{\%3c}%
  \urlescapestep{>}{\%3e}%
  \urlescapestep{\#}{\%23}%
  \urlescapestep{\{}{\%7b}%
  \urlescapestep{\}}{\%7d}%
  \urlescapestep{|}{\%7c}%
  \urlescapestep{\^}{\%5e}%
  \urlescapestep{\~}{\%7e}%
  \urlescapestep{[}{\%5b}%
  \urlescapestep{]}{\%5d}%
  \urlescapestep{\`}{\%60}%
  \x}}

\begin{document}

\urlescape{abcd/efgh\%foo\#bar baz}

\end{document}
  • looks terrific! i'll try first thing tomorrow! – flow Jan 9 '14 at 18:22
  • 2
    answers the question, but most of those don't need to be escaped in a url, and the replacements could be determined implicitly (as it's just the hex of the utf8 code) – David Carlisle Jan 9 '14 at 18:29
  • @DavidCarlisle Yes, of course. That's left as an exercise to the reader. – egreg Jan 9 '14 at 18:39
  • @egreg "You are defining \x in terms of itself, which is not a good thing to do"—to my understanding this is exactly what Don does on p215 of The TeX Book (edef\a{\double\a}) – flow Jan 10 '14 at 14:02
  • @DavidCarlisle you're in both respects; this list is a somewhat hypertrophic recommendation that i will like cut down somewhat. the reason it's being done in this simple-minded fashion is just for ease of implementation (and maybe performance); in a real programming language, i'd always use hexadecimal character codes. – flow Jan 10 '14 at 14:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.