Placing live Stack Exchange reputation in a document

Question

I want to be able to place my current reputation into a document, such that, when I rebuild the document, the displayed reputation is updated.

For example, I might write

My current reputation on {\tt tex.se} is \texrep.

and get (assuming \texrep is defined)

To do this, I expect \texrep would be defined as \directlua{get_rep('tex')}, where get_rep is a Lua function defined in an external Lua script (say, fetch.lua).

---

Note: I have given an answer to my own question, for two reasons:

1. to see if my method can be improved; and
2. to share this 'discovery' with others.

---

Update: my answer is rather long, and has the unfortunate side effect of blocking everything else on the page! Make sure to check the other answers below mine, as they have improved on my method.

Answer 1

If you just want the code, and not the explanation, look here.

---

The trick is to use the Stack Exchange API to download the information on a user. For example, to get my own user information (regarding tex.se), I would go to http://api.stackexchange.com/2.2/users?inname=digitalis_&site=tex. For a different user, or a different site, you would change the respective fields.

Let's put this into a function, curl_user:

curl_user = function (user, site)
   os.execute('mkdir -p cache')
   print('\nGetting ' .. user .. ' from ' .. site .. '...\n')
   os.execute('curl "http://api.stackexchange.com/2.2/users?inname='
      .. user .. '&site=' .. site
      .. '" > ' .. cache_name(user, site) .. '.gz')
   os.execute('gunzip -f ' .. cache_name(user, site) .. '.gz')
end

where cache_name is

cache_name = function (user, site)
   return ('cache/' .. user .. '_' .. site)
end

Because the API limits you to 300 requests per day, it's a bad idea to re-download the user information every time you need it. The solution is to cache the data, in two ways:

1. Per document build. E.g., the first time I write \texrep, it downloads the data and stores it in a variable, so that subsequent occurrences of \texrep use the value stored in memory.

2. On the hard drive. E.g., calling \texrep the first time downloads the data and saves it to a file on the hard drive (say, digitalis_tex), and subsequent document builds read the value from this file.

curl_user already deals with (1) by saving the data in the cache/ directory, under I file with the name <user>_<site>.

A consequence of (2) is that there will need to be a variable which determines whether Lua should re-download or not. We will set it to true for now, as well as creating the table which caches data 'per build'.

UPDATING = true
cache = {}

A function is needed to manage this cache:

get_user = function (user, userid, site)
   if cache[user] == nil then
      cache[user] = {}
   end

   if cache[user][site] == nil then
      cache[user][site] = {}
      cache[user][site]['_raw'] = extract_user(user, userid, site)
   end

   return cache[user][site]['_raw']
end

where extract_user is defined as

extract_user = function (user, userid, site)
   if UPDATING then
      curl_user(user, site)
   end

   io.input(cache_name(user, site))
   raw = io.read('*all')
   me = string.match(raw, userid .. '.*')
   return me
end

extract_user chooses whether to update the saved cache on the hard drive, and uses some pattern matching magic to filter out users which are not wanted. This is needed because of how the API request works: it actually gives you the data of all users whose names have a particular string in them. To differentiate from other users, you need the user's ID (which you can find by looking at the url when you visit your profile on http://stackexchange.com).

You can see how the cache starts to evolve as a kind of tree:

cache
 └── digitalis
      ├── tex
      │    └── _raw
      └── stackoverflow
           └── _raw

All that is now desired is a general function to acquire a value, given:

• a field,
• a user,
• a site.

Here is such a function:

get_field = function (field, user, userid, site)
   if not (cache[user]
          and cache[user][site]
          and cache[user][site][field]) then
      me = get_user(user, userid, site)
      pure = string.match(me, '"' .. field .. '":[^,}]+')
      cache[user][site][field] = string.match(pure, '[^:]+$')
   end
   return cache[user][site][field]
end

It initialises the cache if parts of it are not defined, and returns the desired value.

---

To get the effect in the question, one may write

get_rep = function (site)
   return get_field('reputation', USER, USERID, site)
end

where USER and USERID are set to some user's username and user ID.

USER = 'digitalis'
USERID = 5477562

The entire .tex file would then be

\directlua{dofile('fetch.lua')}
\def\texrep{\directlua{get_rep('tex')}}
My current reputation on {\tt tex.se} is \texrep.
\bye

Note that the functions defined here are general enough to be used for more than just getting the reputation of the user: e.g.,

• the number of badges,
• the change in reputation in the last year/quarter/month,
• the url to the profile image,
• etc.

are all possible.

Answer 2

Here is a ConTeXt implementation of the command. I use a slightly different API call to get the reputation.

ConTeXt has an inbuilt mechanism to download remote content. This content is cached in a directory inside texmf-cache. So, the idea is to simply download the content, unzip it, convert from json to lua table, and access the value of the key reputation. LuaTeX comes with the gzip library, and ConTeXt has a parser for JSON; so there is no need to use an external program at all.

\usemodule[json]
\startluacode
  local get_user_reputation = function(user)
    local url           = string.format("http://api.stackexchange.com//2.2/users/%s?order=desc&sort=reputation&site=tex", user)
    local specification = resolvers.splitmethod(url)
    local filename      = resolvers.finders['http'](specification) or ""

    local gz = gzip.open(filename, "rb")
    local str = ""
    if gz then
        str = gz:read("*all")
        gz:close()
    end
    local js  = utilities.json.tolua(str)
    local rep = js["items"][1]["reputation"] or "0"
    context(rep)
  end

  interfaces.definecommand {
      name = "getrep",
      arguments = {
        { "option", "number" }, 
      },
      macro = get_user_reputation,
  }

\stopluacode

\starttext

My current reputation on \type{tex.se} is \getrep[323].

\stoptext

Since the json object returns a lot of information, so you can extract other keys as well. For example, in my case, this is the information that I get:

t={
 ["has_more"]=false,
 ["items"]={
  {
   ["accept_rate"]=88,
   ["account_id"]=65687,
   ["badge_counts"]={
    ["bronze"]=188,
    ["gold"]=2,
    ["silver"]=82,
   },
   ["creation_date"]=1280861336,
   ["display_name"]="Aditya",
   ["is_employee"]=false,
   ["last_access_date"]=1466999559,
   ["last_modified_date"]=1464661963,
   ["link"]="http://tex.stackexchange.com/users/323/aditya",
   ["profile_image"]="https://www.gravatar.com/avatar/9734cf7ba55eda3a1707cd61a452bdce?s=128&d=identicon&r=PG&f=1",
   ["reputation"]=44190,
   ["reputation_change_day"]=10,
   ["reputation_change_month"]=1060,
   ["reputation_change_quarter"]=1612,
   ["reputation_change_week"]=50,
   ["reputation_change_year"]=3347,
   ["user_id"]=323,
   ["user_type"]="registered",
   ["website_url"]="http://randomdeterminism.wordpress.com/",
  },
 },
 ["quota_max"]=300,
 ["quota_remaining"]=285,
}

By default, ConTeXt will fetch the data again if the cache is older than 1 day. If you want to update more quickly (note that there is rate throttling in stackexchange API), add:

\enabledirectives[schemes.threshold=1200]

which will update the cache after 1200 sec (i.e., 20 min).