# Auto generate List of \url usages within document

I am writing a thesis and I have made extensive use of hyperlinking through \url or \url within a \footnote. Now I would like to add a list (kind of like a simplified bibliography where each link appears only once) of all hyperlinks within the document. Is there a way to have that automatically generated for me?

• Do you have duplicate \urls in your document? – Werner Jul 1 '13 at 16:53
• Yes. There are a small handful of duplicates. – typR Jul 1 '13 at 16:54
• What if we redefine \url to include a \index entry? That way, we can have a list of URL's followed by their occurrences. :) – Paulo Cereda Jul 1 '13 at 17:01
• That sounds promising, unfortunately it also exceeds my proficiency in LaTeX. How would I redefine \url in that way? \newcommand{\url}{\url \index}? – typR Jul 1 '13 at 17:09
• @typR That will give you an infinite loop. You'd have to do something like \let\oldurl\url\renewcommand{\url}[1]{\oldurl{#1}\index{#1}} but if any of your urls contain characters such as ~ or @ extra coding is required. – Nicola Talbot Jul 1 '13 at 17:23

1. The following example uses hyperref (the question has mentioned "hyperlinking") and hooks into \hyper@linkurl to get the URLs.

2. The catched URLs are written into an index file \jobname-url.idx:

\urlentry{<hex coded URL>}{<page number>}


The URL are hex encoded to avoid trouble with special characters.

3. Package filecontents helps to create a style file \jobname-url.mst for makeindex. Makeindex automatically looks for a file with the same name as the input file, but with extension .mst as style file. Then only the .idx file needs to be given as argument for makeindex.

4. Makeindex generates the file \jobname-url.ind:

\begin{theurls}
\urlitem{<hex coded URL>}{<page list>}
...
\end{theurls}

5. Environment theurls and \urlitem are defined appropriately to print the list of urls. \listurlname contains the title of the section.

Remarks:

• Makeindex takes care of the sorting and removes duplicates.
• Hooking into \hyper@linkurl has the advantage, that the URL is normalized (e.g., % and \% are the same, a % with catcode 12/other).
• Hex encoding has the advantage, that special characters such as percent, hash or characters with special meaning for makeindex (at sign, ...) do not need a special treatment.

Example file:

\RequirePackage{filecontents}
\begin{filecontents*}{\jobname-url.mst}
% Input style specifiers
keyword "\\urlentry"
% Output style specifiers
preamble "\\begin{theurls}"
postamble "\\end{theurls}\n"
group_skip ""
item_0 "\n\\urlitem{"
delim_0 "}{"
delim_t "}"
line_max 500
\end{filecontents*}

\documentclass{article}
\usepackage{pdfescape}

\makeatletter
\newwrite\file@url
\openout\file@url=\jobname-url.idx\relax

\newcommand*{\write@url}[1]{%
\begingroup
\EdefEscapeHex\@tmp{#1}%
\protected@write\file@url{}{%
\protect\urlentry{\@tmp}{\thepage}%
}%
\endgroup
}
\write@url{#2}%
}
\newcommand*{\listurlname}{List of URLs}
\newcommand*{\printurls}{%
\InputIfFileExists{\jobname-url.ind}{}{}%
}
\newenvironment{theurls}{%
\section*{\listurlname}%
\@mkboth{\listurlname}{\listurlname}%
\let\write@url\@gobble
\ttfamily
\raggedright
}{%
\par
}
\newcommand*{\urlitem}[2]{%
\hangindent=1em
\hangafter=1
\begingroup
\EdefUnescapeHex\@tmp{#1}%
\expandafter\url\expandafter{\@tmp}%
\endgroup
\par
}
\makeatother

\usepackage[T1]{fontenc}
\usepackage[variablett]{lmodern}

\begin{document}
\href{http://tex.stackexchange.com/q/121977/16967}{question}
on \href{http://tex.stackexchange.com/}{\TeX.SE}.

Further examples for URLs:
\url{http://www.dante.de/}\\
\url{http://www.ctan.org/}\\
\url{mailto:me@example.org/}\\
\url{ftp://ftp.dante.de/pub/tex/}\\
\url{http://www.example.com/\%7efoo/index.html}\\
\url{http://www.example.com/%7efoo/index.html}

\printurls
\end{document}


The following commands generate the result (linux/bash):

$pdflatex test  Generates test-url.mst and test-url.idx. $ makeindex test-url


Generates test-url.ind.

\$ pdflatex test


## Update for page numbers

There are many formatting ways for the page numbers. The following example uses dots to separate the URL from the page numbers that appear at the end of the line (similar to the index of package doc). As requested the page numbers are prefixed with p., if only one page number follows and pp. otherwise. This is implented with the help of package xstring by testing the page number list, whether it contains a comma separator or a hyphen from a range specifier.

\RequirePackage{filecontents}
\begin{filecontents*}{\jobname-url.mst}
% Input style specifiers
keyword "\\urlentry"
% Output style specifiers
preamble "\\begin{theurls}"
postamble "\n\\end{theurls}\n"
group_skip ""
item_0 "\n\\urlitem{"
delim_0 "}{"
delim_t "}"
line_max 500
\end{filecontents*}

\documentclass{article}
\usepackage{pdfescape}
\usepackage{xstring}

\makeatletter
\newwrite\file@url
\openout\file@url=\jobname-url.idx\relax

\newcommand*{\write@url}[1]{%
\begingroup
\EdefEscapeHex\@tmp{#1}%
\protected@write\file@url{}{%
\protect\urlentry{\@tmp}{\thepage}%
}%
\endgroup
}
\write@url{#2}%
}
\newcommand*{\listurlname}{List of URLs}
\newcommand*{\printurls}{%
\InputIfFileExists{\jobname-url.ind}{}{}%
}
\newenvironment{theurls}{%
\section*{\listurlname}%
\@mkboth{\listurlname}{\listurlname}%
\let\write@url\@gobble
\ttfamily
\raggedright
\setlength{\parfillskip}{0pt}%
}{%
\par
}
\newcommand*{\urlitem}[2]{%
\hangindent=1em
\hangafter=1
\begingroup
\EdefUnescapeHex\@tmp{#1}%
\expandafter\url\expandafter{\@tmp}%
\endgroup
\urlindex@pfill
\IfSubStr{#2}{,}{pp}{%
\IfSubStr{#2}{-}{pp}{p}%
}.\@\space\ignorespaces
#2%
\par
}
\newcommand*{\urlindex@pfill}{% from \pfill of package doc'
\unskip~\urlindex@dotfill
\penalty500\strut\nobreak
\urlindex@dotfil~\ignorespaces
}
\newcommand*{\urlindex@dotfill}{% from \dotfill of package doc'
}
\newcommand*{\urlindex@dotfil}{% from \dotfil of package doc'
}
\makeatother

\usepackage[T1]{fontenc}
\usepackage[variablett]{lmodern}

\begin{document}
\href{http://tex.stackexchange.com/q/121977/16967}{question}
on \href{http://tex.stackexchange.com/}{\TeX.SE}.

Further examples for URLs:
\url{http://www.dante.de/}\\
\url{http://www.ctan.org/}\\
\url{mailto:me@example.org/}\\
\url{ftp://ftp.dante.de/pub/tex/}\\
\url{http://www.example.com/\%7efoo/index.html}\\
\url{http://www.example.com/%7efoo/index.html}

% further pages to generate more page numbers for testing the url index
\newpage
\url{http://www.ctan.org}
\newpage
\url{http://www.ctan.org}
\url{http://tex.stackexchange.com/}

\newpage
\printurls

\end{document}


• Wow! Absolutely fantastic! :) – Paulo Cereda Jul 1 '13 at 17:59
• How can you turn on printing of the page numbers? – Joe Corneli Jul 3 '13 at 14:15
• I guess adding a \space(p. #2) to \urlitem is a minimal idea. I'd like to have "pp." when there are multiple pages though... seems much more complicated though. Hm... not so bad after all: tex.stackexchange.com/questions/26870/… – Joe Corneli Jul 3 '13 at 14:34
• @JoeCorneli: I have updated the answer and added an example with page numbers that are preceded by p. or pp.. – Heiko Oberdiek Jul 3 '13 at 20:18
• This seems to be pretty good. Can this be made into a CTAN package? :) – wilx Dec 28 '15 at 23:58

Atention: the following code works only for simple URLs, that is, URLs that do not contain special characters, like %. For a complete solution, please refer to Heiko's answer.

As Nicola mentioned in the comments, redefining \url might be an interesting idea, but some characters in the URL might cause problems. Sadly my TeX-fu isn't good enough to overcome this issue, but here's a preliminary start:

\documentclass{article}
\usepackage{url}
\usepackage{imakeidx}

\let\originalurl\url

\makeindex[name=urls, title={Links found in this document}, columns=1]

\renewcommand{\url}[1]{\originalurl{#1}\index[urls]{\protect\originalurl{#1}}
}

\begin{document}

Hello, make sure to visit \url{http://www.google.com} and,
of course, our own place \url{http://tex.stackexchange.com}.

By the way, \url{http://tex.stackexchange.com} is awesome!

\printindex[urls]

\end{document}


The list is then generated:

Hope it helps. :)

• This will have problems with special characters in the URL, particularly %. But +1 anyway. – egreg Jul 1 '13 at 17:46
• @egreg: I know, but I have no idea on how to solve it. :) Is "Make sure to use URL shortening services for all URLs." a good excuse? :) – Paulo Cereda Jul 1 '13 at 18:00
• Dear downvoter, I really would like your help to improve my answer. Could you write down a few suggestions on what should I do in order to make my answer more adherent to a valid solution? Thanks. :)` – Paulo Cereda Jul 2 '13 at 12:26
• The excuse is good. ;-) Long URLs should never appear in the main text. – egreg Jul 2 '13 at 12:28
• +1 for the idea and the elegant clear simple approach (with the mentioned restriction). – Heiko Oberdiek Jul 3 '13 at 19:27