Summer vacations! The wonderful time when you realize that while you have been teaching classes and grading scripts, science has been moving on and now you must scramble to catch up.
Not so long ago you would go to the library and browse through the journals to quickly see what you had missed out on. Doing the digitial equivalent of this by visiting journal websites and looking at tables of contents takes too many clicks for my tastes. Thankfully the RePec project provides bibliographic data for Economics working papers and journal articles in machine-readable format. Add to that some pre-packaged open-source machine learning software and you can do what was not possible in the stack-walking days: cluster articles according to topic.
I ran the standard Latent Dirichlet Allocation clustering algorithm on the titles and abstracts of articles to group them into twenty topics. Based on my interests I chose articles published in 2014 and 2015 in the leading general interest economics journals and the leading field journals in macroeconomics and game theory.
The clustering code is on GitHub and
here are the clusters for your enjoyment. Following the link are the most probable words for the cluster. The prefix T: indicates the occurence of the word in the title. The words have been stemmed so that for example ‘calibration’ and ‘calibrated’ have booth been reduced to calibr’.
- Cluster 0: T:auction, T:signal, T:buyer, T:communic, T:seller, inform, T:bid
- Cluster 1: T:liquid, T:loan, T:regul, T:lend, T:fund, T:financi T:crisi, T:deposit
- Cluster 2: T:busi, T:busi T:cycl, T:fluctuat, T:friction, T:general T:equilibrium, T:calibr, T:macroeconom
- Cluster 3: T:profit, T:entri, T:industri, T:consum, T:adjust, T:innov, T:sale
- Cluster 4: T:bond, T:cash, T:corpor, T:investor, T:flow, T:spread, T:news
- Cluster 5: T:contract, T:fund, T:ex, T:bargain, T:citationref, T:post, T:ex T:ant
- Cluster 6: T:network, T:learn, T:resourc, T:interact, T:converg, T:choos, T:coordin
- Cluster 7: T:intern, T:foreign, T:region, T:industri, T:global, T:extern, T:integr
- Cluster 8: T:equationsourc, T:mi, T:math, T:format, T:inlineequ, T:equationsourc T:format, T:xlink
- Cluster 9: T:consum, T:save, T:wealth, T:discount, T:insur, T:expenditur, T:inequ
- Cluster 10: T:payoff, T:belief, T:action, T:perfect, T:nash, T:space, game
- Cluster 11: T:portfolio, T:avers, T:option, T:investor, T:forecast, T:index, T:risk T:avers
- Cluster 12: T:inflat, T:fiscal, T:interest T:rate, T:central, monetari, T:long T:run, T:short
- Cluster 13: T:percent, T:unit, T:panel, T:document, T:0, T:previous, T:unit T:state
- Cluster 14: T:ambigu, T:comput, T:solut, T:linear, T:simul, T:approxim, T:continu
- Cluster 15: T:pareto, T:domin, T:satisfi, T:rank, T:assign, T:core, T:maxim
- Cluster 16: T:debt, T:default, T:hous, T:financ, T:constraint, T:borrow, T:collater
- Cluster 17: T:school, T:colleg, T:student, T:health, T:insur, T:program, T:educ
- Cluster 18: T:worker, T:tax, T:wage, T:unemploy, T:job, T:skill, T:human
- Cluster 19: T:vote, T:group, T:polit, T:voter, T:particip, T:experiment, T:major
This was my first time running any kind of machine-learning algorithm. The most important lesson for me was that as in any other statistical work, having clean data is very important. Some of the titles have HTML and LaTeX markup, or copyright notices with the publisher’s name, and the model latches on to this as important features even though they are not.
I also saw the value of the IPython notebook format over writing scripts. You get inline graphs. Being able to restart the code from somewhere in the middle is a boon when some parts of the code is expensive to run. But I got bitten a few times by variables left over from previous runs affecting results or not being able to restart computation in the middle because, say, a previous run had closed a database connection.