How soon do all CTAN mirrors update for TeX Live Utility to detect the changes?

Question

If a new package gets officially recognized with an entry at CTAN today, (on average) when can I typically expect all mirrors to be updated so that TeX Live Utility can detect the changes?

Answer 1

Will's got most of it; ctan sites send "mirror this" instructions to each other, when something's been installed. (Of course, tug.ctan.org isn't doing that just now, but it seems to be accepting instructions from cambridge and dante.)

In addition, a comparison between the three sites is run every night, and discrepancies are investigated.

Martin's being confused by the "datestamp" file. That's there for the mirror monitor only, and is generated locally (in a cron job); the synchronisation messages are processed much more rapidly than once an hour.

Note that the mirror monitor results are stored in a database, to which mirror.ctan.org has access: mirror.ctan.org should never provide a pointer to a mirror that seems >24 hours "stale".

Note the difference between ctan sites (dante, cambridge and [officially] tug) and ctan mirrors. while ctan sites do use snippets of mirroring mechanism to keep up-to-date with each other, ctan mirrors use mirroring (at fixed times, usually) and nothing else.

Answer 2

My understanding is that the three ‘master’ mirrors (US, Cambridge, Dante) push updates to each other, from which the other CTAN mirrors sync their content within at most 24 hours. I don't think there's a ‘minimum time’ you can count on, however.

Also note that there's a delay between uploading to CTAN and content being available in TeX Live (tlnet). Again, that usually happens within 24 hours but it's dependent on human-power.

Answer 3

Each mirror has a timestamp file which indicates the last time it got synced. The master servers (tug.ctan.org, dante.ctan.org, www.tex.ac.uk) synchronize with each other apparently every hour almost on the hour. At least tug's timestamp is always :01 min after the full hour and dante's :02. Their timestamps are updated every hour even when no files changed.

The other mirrors are supposed to sync once a day using a cronjob. I configured my mirror to sync every 3 hours first and got told that this is way to frequently.

You can see a nice statistic about mirror delays on the mirror monitor page.

My experiences with my packages are:
It takes about half a day until your upload is handled by the upload admins (due to the current situation it takes a little longer), and then about 25 hours (1+24) until all active mirrors picked it up. The TeXLive package should be available also about two days after the upload got processed. It is faster for an update then for a new package.

Answer 4

When a package is uploaded, there is a number of steps before it becomes available to TeX Live Utility (actually tlmgr).

1. a CTAN maintainer handles the upload to make it available at his site;
2. the upload is generally announced on ctan-ann;
3. it is pushed to other sites and pulled by mirrors;
4. a human from the TeX Live team notices the announce (if the upload wasn't announced, a script run weekly checks the actual contents of the packages in a local mirror against the files currently in TeX Live) and commits the packages in TeX Live's svn tree;
5. a nightly cron rebuilds the master package repository from the svn tree;
6. the master tlnet is pulled by a CTAN site, then pushed to other sites and pulled by mirrors as usual.

Steps 1 and 2 usually happen in a matter of hours (great job, CTANers!). Step 3 can take a few minutes to slightly more than 24h depending on the mirror used by the TL maintainer. Step 4 usually happens daily (thanks to Karl who is doing this work alone currently, afaik) or weekly for unannounced packages. It may take more time for particularly complex packages. Step 5 and 6 usually are a few hours to 48h, depending again on the local mirror.

So, the delay from a package appearing on CTAN to its availability via tlmgr or TLU varies from a few hours (best case) to more than one week depending on the mirrors, the TL team workload, and the nature of the package, but my guestimate for the average would be 3 days. Variance is dictated mostly by step 4, at least for extreme cases.