Tagged: t-SNE

wiki_popes

Grouping entries in Slovenian Wikipedia by contributors

Share Button

Some time ago, I helped Miha Mazzini extract some data from Slovenian Wikipedia. For that, I needed to write a comprehensive parser, extracting not only titles and text, but also number of overall and per-contributor revisions, along with contributor usernames.

So, for each entry, I got a list of contributing accounts and number of edits that were performed by that account. I wondered: how are the areas of expertise distributed among all those contributors? Are some of them specialized mainly for science, some others for politics, and so on? And, perhaps more interestingly, where do these areas overlap? Do we have experts for sport, who also happen to curate political entries?

To find out, I extracted all entries with more than 25 edits, vectorized them by contributing accounts, and ran t-SNE to cluster them spatially and prepare the visualization. When t-SNE layout was complete, k-means clustering was run on the x,y coordinates only to be able to distinguish those areas by color. It must be said that these colored groups don’t always coincide with semantic grouping, so take it with a grain of salt. It’s there mainly to make the map look better and to improve legibility. Font size is in proportion with number of revisions that an entry had so far.

Here’s the entire map as one big 9000 x 6000 image. Click the image to display, then zoom into it with mouse. You can find cropped clusters and some commentary below.

Wikipedia entries by contributing authors
Wikipedia entries by contributing authors – entire map (click for enlargement)

Turns out the areas of expertise are pretty well delineated.

Let’s look at some clusters. Here we have some geographic Wikipedia entries, mainly countries and some historical persons. That sounds logical – editing an entry about a great ruler probably causes one to contribute to an entry about his or her country.

wiki_countries

Here are some famous Slovenian people, mostly writers, intermixed with some towns. In the lower left quadrant, there are alo some Slovenian politicians. It sounds funny that the late Communist ruler Josip Broz Tito is so close to Janez Janša, who is the current leader of right-wing opposition. It appears that there is a number of people who edit both entries. I wonder why. Here’s an article by Miles Mathis about editing of Wikipedia. I don’t know what to think of it, but I surely read a lot about autocratic rule of (English) Wikipedia editors to give it some credence. I don’t know about Slovenian version, and I don’t want to speculate, but this is as good an opportunity as any to start thinking about it.

wiki_history

Here is a cluster of lists. It seems that there exists an entire group of people who curate them, regardlessly of their content.

wiki_lists

Here’s a funny cluster dealing mostly with public transit in Slovenia. It almost seems that there are some bureaucrats in the government that edit these entries on taxpayer dime. I could probably find that out, if I traced the IPs in the edit logs. If someone hires me as an Internet detective, I might do that, but I made these pictures for fun.

wiki_lpp

This is an interesting cluster. It appears that many same people edit entries about Euroviviion Contest, parliamentary elections, World cup in basketball and World cups in skiing, along with two new parties in Slovenian parliament: ZAAB, which is the remainder of majority party in the last parliamentary term, and SMC, which is a new majority party. Both parties, along with Pozitivna Slovenija (former majority party)  were founded hastily right before elections, and won them by a big landslide. I wonder how would political analysts comment on their (speculatively) members’ love for sports contests. wiki_politicians

Here we have many religious personalities, mostly many popes.wiki_popes

A grouping of entries about Slovenian popular music.

wiki_popevke

Some entries about historical scientists and natural sciences.

wiki_science

More geographical entries, along with some entries about Slovenian highways.

wiki_traffic

Here’s the center of the map. It would follow logic that entries with many non-specialized contributors are drawn towards it. It’s generally more chaotic that the outskirts, but here are great many contemporary and historical art personalities grouped together..

wiki_writers

Another snapshot from the center, mostly consisting of entries about worker’s rights and things related to work.

wiki_workers

So here it is. For the technically minded – everything was done in JavaScript with Andrej Karpathy‘s tsnejs library, clusterfck for k-means, and d3 for drawing.

I also made an inverted map, on which the contributors were shown, grouped by the entries they made revisions to. It’s not so interesting for general public, but if someone wants to see it, it’s available by request.

Similarities between representatives in Slovenian parliament

Share Button

The title should actually be “An exploration of dimensionality reduction techniques on voting dataset from Slovenian parliament”.

I’ve long been procrastinating with proper and comprehensive study of various machine learning techniques, especially those related to neural networks. I feel I made a few baby steps towards that goal with this research, which is actually a writeup of a project I made for a local newspaper in collaboration with excellent designer Aljaž Vindiš (follow him on Twitter).

The dataset comes from another project that I’m collaborating on with Transparency International Slovenia and Institut Jožef Stefan. Zakonodajni monitor is a platform for inspecting the legislative process and for following the activity of parliamentary representatives, intended mostly for journalists and researchers. Among other things, it contains records of every vote cast in parliamentary sessions by every representative, which is then used for various statistics and visualizations. It also has an API for public access to that data, although I have it in a local database too, making it somewhat easier for me to explore it.

This project is an attempt to visualize relationships between representatives and parties in two dimensional space, or on a line, to better understand the dynamics of power in Slovenian politics. It’s a part of my ongoing collaboration with Dnevnik newspaper for data analysis and visualization. Since the project was not supposed to be interactive from the start, one important constraint was that the results should be fit for a paper version of the newspaper.

Dataset

Each representative has a great many properties in the database, but among them is a vote vector, containing a record of her or his votes so far. A “yea” vote is 1, “nay” is -1, and abstinence for whatever reason is 0. At the time of this project, there were a little less than 650 votings in this parliamentary term, so input data for each representative was a vector with approximately 650 dimensions. Our objective was to construct one- or twodimensional visualization, which would hopefully confirm our existing knowledge about alliances between parties and individuals in the parliament, and, if possible, reveal new and interesting information.

To effectively communicate this information, we had to employ some dimensionality reduction techniques, of which we tried three:

  • PCA (principal components analysis),
  • autoencoder,
  • t-SNE

In the end, we decided on t-SNE because it’s fast and convenient, but other two methods, with the exception of PCA in two dimensions, gave very similar results.

What is “dimensionality reduction”, you might ask? It’s a set of techniques to make sense of complex data. A shadow is a simple natural reduction technique, because it’s a projection from three dimensional space into two. Going on with this analogy, if you want to recognize a person from its shadow, the position of the sun matters a great deal. For example, sun directly over person’s head doesn’t give us much information about the person’s shape. It’s necessary to find a proper angle.

These various techniques have much to to with proper positioning of the “sun” in relation to data, to retain maximum possible amount of information in the projection. Of course, if you project from 650 dimensions into one, a lot of information is lost. Also, in many cases it’s not immediately clear what is the exact meaning of the axes in the projection. Read on, I’ll try to elaborate below.

Autoencoder

We started with an autoencoder. An autoencoder is a form of artificial neural network that is often used for dimensionality reduction. It is a deep neural network with many layers that essentially tries to teach itself identity, that is to say, it’s trained to generalize patterns in data in by compressing the knowledge in some way, and then recreating it. We used an autoencoder with 650 inputs, two layers of 100 neurons each, then a bottleneck layer with two neurons only, followed by an inverted structure acting as a decoder. When training was complete, every representative’s vector was again propagated through the network, and activations of the two bottleneck neurons saved as a coordinate pair. These were then plotted on a 2D canvas, resulting in a image shown below.

poslanci_autoencoder_v3

Legend for clarification:

  • brown (SMC), blue (DeSUS) and red (SD) are leftist position parties with heavy majority in the parliament. Much could be said about their leftism, but let’s leave at that.
  • violet (SDS) and green (NSi) are rightmost opposition parties. They are vehemently anti-communist (SDS) and catholic-conservative (NSi)
  • rose is oppositional ZAAB, which is a party of former prime minister Alenka Bratušek. It leans to the left.
  • grey is oppositional ZL, which is Slovenia’s version of Syriza.

 

The dataset was relatively small, so the autoencoder was implemented in JavaScript with Karpathy’s excellent convnet.js library. Training took two hours on a i7 machine with 16GB RAM.
As a small branch of this project, we also tried to arrange the representatives on an ideological spectrum. For this, a similar neural network was used, but we first trained it with the most left-and right-leaning representatives to obtain extremes, then fed the others through it and plotted the regression scores in one dimension. This arrangement is somewhat different than the final one.

poslanci_by_ideology

 

Principal component analysis

Next on was an attempt to validate our results with PCA. Principal components analysis is a (quote) “technique, used to emphasize variation and bring out strong patterns in a dataset. It’s often used to make data easy to explore and visualize” . It’s essentially a method for projecting data from multidimensional space to a lowerdimensional (say 2D) one, where we try to retain as much information as possible. The first axis is chosen so that the variance along it is maximized, maximizing the information, the others follow in a similar fashion, with the constraint that they must be orthogonal.
We ran PCA for one- and twodimensional solutions, giving solutions on images below.

poslanci_pca_2d

 

Here’s the one-dimensional variant:

poslanci_by_ideology_pca

t-SNE

Finally, we used t-SNE algorithm with one- and twodimensional solutions. t-SNE (or “t-distributed stochastic neighbor embedding”) is another technique for dimensionality reduction, well-suited for visualizing complex datasets in 2D or 3D. You’ll mostly see it in articles dealing with classification of complex data, for instance images and words, where you can see nice plots of similarily-themed images or words with similar meaning clustered together. Here we used it on our voting data, and the results were quite good. First we tried a 2D visualization. It’s roughly similar to the one derived from autoencoder.

Dot sizes correspond to voting attendance. You can see that the representatives with lower attendance are drawn to the center. Also, note that the violet group (SDS party), which is the true and fervent opposition, is relatively close to those with lower attendance. This is simply because the opposition frequently employs obstruction as a parliamentary tactics, or are simply not there due to other reasons.

The neutral control point is the azure rectangle in the center. It’s simply a hypotethical rep that always abstained.

poslanci_tsne_v6

See the voting records for the opposition (yellow is absent, red is a vote against, blue a vote in favor):

controls_opposition

And here are records for some ministers:

controls_missing

Compare these with the position:

controls_position

Partly confirming validity and possible artefacts, we moved on. What we realized so far was that the absences introduced errors in position, and that these errors tend to draw those absent towards the center, possibly confusing the arrangement in a way that some people could wonder: what does this clearly positional rep do close to the opposition? Is (s)he leaning towards them in voting? No, this is simply an artefact that absences introduce into the positioning due to the way these methods work.

Then we decided that we’d maybe like a simpler visualization, one that is more suitable for a paper medium. So we ran t-SNE again in one dimension, then we used a “beeswarm” layout to sketch things out. The beeswarm is essentially a one-dimensional layout, in which some clustered elements are pushed onto a plane to avoid overcrowding on the single axis.

bees_v1
Finally, and mostly for aesthetic reasons, we converted that into hex-binned layout. Number of hexagons corresponds to number of dots above, but voting attendance is encoded in opacity, and party affiliation is represented by color. Here is a sketch:

poslanci_hex_opacity

Here’s a closer view of the opposition:poslanci_hex_close

As a final step, we removed everyone who was present at less that 200 voting sessions, and also added three control points:

  • neutral: a hypothetical representative who always abstains,
  • all yea: a hypothetical representative who always votes in favor of the proposition,
  • all nay: a hypothetical representative who always votes against the proposition

The neutral control point neatly bisects the space between the position and opposition, not counting the mostly absent representatives from the position. The other two would be relevant if all propositions came from the position – it would then be at the extreme pro-government position. In reality, many acts are proposed by the opposition, so they are just not relevant. In the image below, these are the azure hexagons. Neutral is the leftmost one.

poslanci_hex2
And here is a finished version, expertly done by a pro designer:

hex_dnevnik

Added bonus: visualization of tSNE in 3D:

poslanci_3d

Closeup on opposition:

poslanci_3d_2

Interpretation

So, what does this visualization really show? I’d like to say that since the acts subject to voting are mostly put forward by the governing coalition, it’s an arrangement of representatives on a continuum of support for government policy. But that is simply not so, as many acts are proposed by the opposition. It’s more like that the arrangement depends on an individual’s position in relation to majority’s vote, which might or might not relate to the above.

This often coincides with arrangement on ideological spectrum, but it’s not the same. You might wonder what is a cluster of weakly-colored representatives in the right-middle. These are mostly ministers that cast a few votes in the beginning of the parliamentary term, but which then left to be members of actual government. They still were members at one point in time, so we included them in our research, but we might have easily dropped them, since they don’t really figure in day-to-day parliamentary work.

Most of the errors and contra-intuitive positioning are due to gaps in representatives’ voting records. These methods compare voting records component-wise, so if, for example, we have two members of the same party, who substituted for each other (one was there when another wasn’t, as is the case with the ministers), we can only compare their available records with everyone else, but not among themselves.

t-SNE was also done in JavaScript, with another one of Karpathy’s libraries (tsnejs).

Here’s a final look at the data: a hierarchical clustering of all the representatives, including those mostly absent.

hierarchical

Original datasets and code are available by request. My mail address is on the About page.