Automatically find fastest CTAN mirror

Question

Back when I used linux there was a program that would find you the fastest Debian package repository based on your location. Is there a similar one for CTAN? I've been manually picking ones that I think are physically close to me and basing it on the ping time, but that isn't super reliable. Just asking as I'm on a 100+ MBit/s download connection, but it still takes hours for me to install LaTeX and a very long time to update packages.

Answer 1

Assuming you're on Linux you can use this command to find the fastest mirror:

netselect -vv -t40 -s20 $(\
  curl -sSL http://dante.ctan.org/mirmon/ | \
  grep -oE '<TD ALIGN=RIGHT><A HREF="[^"]*' | \
  cut -d\" -f2 \
) | \
cut -c7- | \
LC_ALL=C xargs -tn1 -i curl -sSL -w "%{time_total} %{speed_download} {}\n" -o /dev/null {}FILES.byname | \
sort -n

This will give you the 20 fastest mirrors (actually measured download speed). It'll rank them by speed (fastest on top) and tell you how long it took to download the file list (/FILES.byname) and the average download speed.
It's also a fair bit verbose and will tell you what it does along the way. You can reduce the verbosiy by removing the -t flag in the xargs command and by removing 1 or both -v flags from the netselect command. Adjust the number of total servers you want to check for download speed with the parameter -s20.

Below is a version you can use in a shell script, that will only output the fastest mirror and nothing else. Very usefull for shell scripts.

netselect -t40 -s20 $(\
  curl -sSL http://dante.ctan.org/mirmon/ | \
  grep -oE '<TD ALIGN=RIGHT><A HREF="[^"]*' | \
  cut -d\" -f2 \
) | \
cut -c7- | \
LC_ALL=C xargs -n1 -i curl -sSL -w "%{time_total} {}\n" -o /dev/null {}FILES.byname | \
sort -n | \
head -n1 | \
cut -d\  -f2

Answer 2

netselect is no longer available using Ubuntu, so I've included a more general (if much slower) version of this answer that does not rely on it:

curl -sSL http://dante.ctan.org/mirmon/ | grep -oE '<TD ALIGN=RIGHT><A HREF="[^"]*' | cut -d\" -f2 | LC_ALL=C xargs -n1 -i curl -sSL -w "%{time_total} %{speed_download} {}\n" -o /dev/null {}FILES.byname | sort -n

This wasn't working for me today for some reason, so I used:

#!/bin/bash
if [ -z "$1" ]
then
    for mirror in `curl -sSL http://dante.ctan.org/mirmon/ | grep -oE '<TD ALIGN=RIGHT><A HREF="[^"]*' | cut -d\" -f2`
    do
        (
            host=`echo $mirror |sed s,.*//,,|sed s,/.*,,`
            echo -e `ping $host -c1 | grep time=|sed s,.*time=,,`:'  \t\t'$mirror
        ) &
        done
    wait
    exit 1
fi

and then piped that to sort -n