# Placing live Stack Exchange reputation in a document

I want to be able to place my current reputation into a document, such that, when I rebuild the document, the displayed reputation is updated.

For example, I might write

My current reputation on {\tt tex.se} is \texrep.


and get (assuming \texrep is defined)

To do this, I expect \texrep would be defined as \directlua{get_rep('tex')}, where get_rep is a Lua function defined in an external Lua script (say, fetch.lua).

Note: I have given an answer to my own question, for two reasons:

1. to see if my method can be improved; and
2. to share this 'discovery' with others.

Update: my answer is rather long, and has the unfortunate side effect of blocking everything else on the page! Make sure to check the other answers below mine, as they have improved on my method.

• I don't think this is a LaTeX question, though it is interesting. – Johannes_B Jun 25 '16 at 20:04
• @Johannes_B; Since it's about the capabilities of (Lua)TeX, and pushing the boundaries of a TeX document, I feel it is. – digitalis_ Jun 25 '16 at 20:06
• Just FYI, the old {\tt foo}, {\bf foo}, {\it foo} etc. commands are 20+ years deprecated. {\ttfamily foo}, {\bfseries foo}, {\itshape foo} or \texttt{foo}, \textbf{foo}, \textit{foo} are strictly encouraged. Self-answering is also explicitly acceptable on Stackexchange, although admittedly some people (and some stacks) don't like it, but you certainly don't have to justify yourself, I thought this was cool :) – Au101 Jun 25 '16 at 20:43
• @Au101; I used \tt because I was using Plain (Lua)TeX, to make the question as general as possible (so it would also apply to ConTeXt etc.): but you are right! – digitalis_ Jun 25 '16 at 20:49
• Oh, fair enough, I didn't realise you were using plain :) – Au101 Jun 25 '16 at 20:50

If you just want the code, and not the explanation, look here.

The trick is to use the Stack Exchange API to download the information on a user. For example, to get my own user information (regarding tex.se), I would go to http://api.stackexchange.com/2.2/users?inname=digitalis_&site=tex. For a different user, or a different site, you would change the respective fields.

Let's put this into a function, curl_user:

curl_user = function (user, site)
os.execute('mkdir -p cache')
print('\nGetting ' .. user .. ' from ' .. site .. '...\n')
os.execute('curl "http://api.stackexchange.com/2.2/users?inname='
.. user .. '&site=' .. site
.. '" > ' .. cache_name(user, site) .. '.gz')
os.execute('gunzip -f ' .. cache_name(user, site) .. '.gz')
end


where cache_name is

cache_name = function (user, site)
return ('cache/' .. user .. '_' .. site)
end


Because the API limits you to 300 requests per day, it's a bad idea to re-download the user information every time you need it. The solution is to cache the data, in two ways:

1. Per document build. E.g., the first time I write \texrep, it downloads the data and stores it in a variable, so that subsequent occurrences of \texrep use the value stored in memory.

2. On the hard drive. E.g., calling \texrep the first time downloads the data and saves it to a file on the hard drive (say, digitalis_tex), and subsequent document builds read the value from this file.

curl_user already deals with (1) by saving the data in the cache/ directory, under I file with the name <user>_<site>.

A consequence of (2) is that there will need to be a variable which determines whether Lua should re-download or not. We will set it to true for now, as well as creating the table which caches data 'per build'.

UPDATING = true
cache = {}


A function is needed to manage this cache:

get_user = function (user, userid, site)
if cache[user] == nil then
cache[user] = {}
end

if cache[user][site] == nil then
cache[user][site] = {}
cache[user][site]['_raw'] = extract_user(user, userid, site)
end

return cache[user][site]['_raw']
end


where extract_user is defined as

extract_user = function (user, userid, site)
if UPDATING then
curl_user(user, site)
end

io.input(cache_name(user, site))
me = string.match(raw, userid .. '.*')
return me
end


extract_user chooses whether to update the saved cache on the hard drive, and uses some pattern matching magic to filter out users which are not wanted. This is needed because of how the API request works: it actually gives you the data of all users whose names have a particular string in them. To differentiate from other users, you need the user's ID (which you can find by looking at the url when you visit your profile on http://stackexchange.com).

You can see how the cache starts to evolve as a kind of tree:

cache
└── digitalis
├── tex
│    └── _raw
└── stackoverflow
└── _raw


All that is now desired is a general function to acquire a value, given:

• a field,
• a user,
• a site.

Here is such a function:

get_field = function (field, user, userid, site)
if not (cache[user]
and cache[user][site]
and cache[user][site][field]) then
me = get_user(user, userid, site)
pure = string.match(me, '"' .. field .. '":[^,}]+')
cache[user][site][field] = string.match(pure, '[^:]+\$')
end
return cache[user][site][field]
end


It initialises the cache if parts of it are not defined, and returns the desired value.

To get the effect in the question, one may write

get_rep = function (site)
return get_field('reputation', USER, USERID, site)
end


where USER and USERID are set to some user's username and user ID.

USER = 'digitalis'
USERID = 5477562


The entire .tex file would then be

\directlua{dofile('fetch.lua')}
\def\texrep{\directlua{get_rep('tex')}}
My current reputation on {\tt tex.se} is \texrep.
\bye


Note that the functions defined here are general enough to be used for more than just getting the reputation of the user: e.g.,

• the number of badges,
• the change in reputation in the last year/quarter/month,
• the url to the profile image,
• etc.

are all possible.

Here is a ConTeXt implementation of the command. I use a slightly different API call to get the reputation.

ConTeXt has an inbuilt mechanism to download remote content. This content is cached in a directory inside texmf-cache. So, the idea is to simply download the content, unzip it, convert from json to lua table, and access the value of the key reputation. LuaTeX comes with the gzip library, and ConTeXt has a parser for JSON; so there is no need to use an external program at all.

\usemodule[json]
\startluacode
local get_user_reputation = function(user)
local url           = string.format("http://api.stackexchange.com//2.2/users/%s?order=desc&sort=reputation&site=tex", user)
local specification = resolvers.splitmethod(url)
local filename      = resolvers.finders['http'](specification) or ""

local gz = gzip.open(filename, "rb")
local str = ""
if gz then
gz:close()
end
local js  = utilities.json.tolua(str)
local rep = js["items"][1]["reputation"] or "0"
context(rep)
end

interfaces.definecommand {
name = "getrep",
arguments = {
{ "option", "number" },
},
macro = get_user_reputation,
}

\stopluacode

\starttext

My current reputation on \type{tex.se} is \getrep[323].

\stoptext


Since the json object returns a lot of information, so you can extract other keys as well. For example, in my case, this is the information that I get:

t={
["has_more"]=false,
["items"]={
{
["accept_rate"]=88,
["account_id"]=65687,
["bronze"]=188,
["gold"]=2,
["silver"]=82,
},
["creation_date"]=1280861336,
["is_employee"]=false,
["last_access_date"]=1466999559,
["last_modified_date"]=1464661963,
["profile_image"]="https://www.gravatar.com/avatar/9734cf7ba55eda3a1707cd61a452bdce?s=128&d=identicon&r=PG&f=1",
["reputation"]=44190,
["reputation_change_day"]=10,
["reputation_change_month"]=1060,
["reputation_change_quarter"]=1612,
["reputation_change_week"]=50,
["reputation_change_year"]=3347,
["user_id"]=323,
["user_type"]="registered",
["website_url"]="http://randomdeterminism.wordpress.com/",
},
},
["quota_max"]=300,
["quota_remaining"]=285,
}


By default, ConTeXt will fetch the data again if the cache is older than 1 day. If you want to update more quickly (note that there is rate throttling in stackexchange API), add:

\enabledirectives[schemes.threshold=1200]


which will update the cache after 1200 sec (i.e., 20 min).

• The thing that's very interesting to me about this one is how ConTeXt will re-fetch after a certain amount of time has elapsed (thus eliminating the need for an UPDATING variable which is set manually). – digitalis_ Jun 27 '16 at 14:56
• The built-in JSON parser is also much more clean than my regexp solution. – digitalis_ Jun 27 '16 at 14:57
• @digitalis_: LPEG based parsers are usually much cleaner than those based on regex. – Aditya Jun 28 '16 at 4:02