State projects and county budget visualizations – a collaboration with Transparency International Slovenia

Share Button

These two projects are a result of recent collaboration with Transparency International Slovenia. The datasets were provided by the state, and I was asked to develop visualizations that would structure the information in an accessible way. Much help was also provided by members of Institut Jožef Štefan.

State project browser

The first project is a browser of all projects, initiated by state institutions, from 1991 on. The idea was to let users discover, where and for what purposes the money goes in their county. The dataset and visualization allow for exploration by various categories, as well as time.

The projects in the dataset also contain projects that are still in the planning phase, and won’t be completed until year 2025. With this tool, citizens can hopefully inspect the planned expenditures for roads, water sources, and other categories of infrastructure, culture and other fields of development, and compare that with their own expectations.

It allows browsing and filtering of projects by statistical regions and counties, as well as displaying the timeline of all projects, which is basically an  expandable version of a Gantt chart.

To see the interactive project website, click here, or click the image below.

State projects app
State projects app

The original data is provided on the project’s “About” page.

County budget browser

The new project is a straightforward visualization of county budgets. The budgets are displayed as dynamic, zoomable hierarchical (“sunburst”) diagrams. They react to each other, allowing a side-by-side comparison of budgets of two user-selected counties.

The visualization enables users to delve into expenses and incomes of all Slovenian counties on separate tabs.

To see the interactive project website, click here, or click the image below.

County budgets app
County budgets app

 

Technology and design

The data cleanup and preparation was done with some Python scripts. The sunburst diagram accepts hierarchical data in a tree format, so this provided an interesting exercise of converting a tabular dataset into a nested dictionary of optional depth.

The visualizations were done in d3, which is really an indispensable tool for any serious work in online visualization.

Both projects were minimalistically, yet expertly designed by Tomaž Plahuta (Bitnik, Eno).

Check out the projects and let me know your opinion in the comments!

Map of apartment building management companies in Ljubljana

Share Button

Some time ago, I did an analysis of building manager companies for dnevnik.si. It was an interesting project, because there’s much controversy surrounding this topic. Some of these companies have been in bitter disputes with the residents and competing companies over their business, generating a lot of newspaper coverage.

We did a simple analysis to deterimine market shares and relative winners and losers in comparison to market leader. Here’s an image of final page in the newspaper. Click it to open the PDF.

upravniki_dnevnik
Newspaper page in Dnevnik.si

 

We also did a lot of other stuff that didn’t find their way into the final article. I’m now publishing an interactive map, showing the distribution of these companies across town. There’s usually one per apartment building. For this map, I only processed the apartment buildings with more than one apartment, because that’s where business is.

Click on the image below to open the map in a new tab, or click here.

Click for interactive map of apartment building managers
Click for interactive map of apartment building managers

Click on a building, and a popup will display the address and relevant history. It’s also possible to choose a company, and the map will refresh, displaying only the buildings in the selected company’s management. There’s also a button to display only those buildings where managers changed in the interval from years 2005 to 2015.

There’s also a graphic showing how many buildings a company gained from other companies. Here’s a screenshot. Click will take you to the same page as image above.

Who gained boildings from whom in Ljubljana
Who gained buildings from whom in Ljubljana

 

The original dataset came from GURS, but then we were able to acquire an update from the city government. We decided to use that, because they gave us three different updates, each five years apart, from which we were able to calculate the differences. You can see them in table below.

  • total flats: how many apartments are managed by a company,
  • gained: how many flats were gained by the company from 2005 to 2015,
  • lost: how many flats were lost by the company from 2005 to 2015.

Original data is here.

Company Total flats Gained Lost
SPL D.D., FRANKOPANSKA ULICA 18A, 1519 LJUBLJANA 107130 38 429
INSTA D.O.O., KOPRSKA ULICA 94, 1000 LJUBLJANA 2338 0 183
METALKA STANOVANJSKE STORITVE D.O.O., INDUSTRIJSKA CESTA 1, 1000 LJUBLJANA 22529 669 382
SZ RUSKI CAR D.O.O., GLINŠKOVA PLOŠČAD 22, 1000 LJUBLJANA 5515 256 182
PRACTIC D.O.O., LJUBLJANA, CELOVŠKA CESTA 147, 1000 LJUBLJANA 8355 92 511
VALINA UPRAVLJANJE, D.O.O., LITIJSKA CESTA 45, 1000 LJUBLJANA 480 72 10
DONF D.O.O., MAJORJA LAVRIČA ULICA 12, 1000 LJUBLJANA 6891 167 741
STANDOM ENGINEERING OTRIN D.O.O., BRNČIČEVA ULICA 13, 1231 LJUBLJANA – ČRNUČE 5661 113 20
AKTIVA UPRAVLJANJE D.O.O., LJUBLJANSKA CESTA 12F, 1236 TRZIN 8764 335 52
ŽELVA D.O.O., SAMOVA ULICA 9, 1000 LJUBLJANA 13550 187 236
UPRADOM D.O.O., LINHARTOVA CESTA 13, 1000 LJUBLJANA 3141 187 0
MASTRA D.O.O., ZALOŠKA CESTA 153, 1120 LJUBLJANA 6566 198 260
VALOR D.O.O., LJUBLJANA, ROZMANOVA ULICA 1, 1000 LJUBLJANA 351 26 42
M.I.S. D.O.O. LJUBLJANA, PROLETARSKA CESTA 4, 1000 LJUBLJANA 3999 318 22
LORA, D.O.O., ZALOŠKA CESTA 163, 1000 LJUBLJANA 5084 0 24
TENIT, D.O.O., LJUBLJANA, GAŠPERŠIČEVA ULICA 23, 1120 LJUBLJANA 2631 154 146
TABOR UPRAVLJANJE IN VZDRŽEVANJE D.O.O., TABOR 9, 1000 LJUBLJANA 4729 71 0
UVG D.O.O., LINHARTOVA CESTA 17, 1000 LJUBLJANA 7189 304 503
MANICOM D.O.O., POVŠETOVA ULICA 65, 1000 LJUBLJANA 118 0 34
KOVAČEVIČ MARIJA, MAJDE VRHOVNIKOVE ULICA 18, 1000 LJUBLJANA 48 0 0
PATE, D.O.O., TOPNIŠKA ULICA 19, 1000 LJUBLJANA 1867 0 800
STANDOM D.O.O., ULICA ANE ZIHERLOVE 6, 1000 LJUBLJANA 5306 48 462
SKUPNOST LASTNIKOV POT K RIBNIKU 16-22, POT K RIBNIKU 16, 1000 LJUBLJANA 656 0 0
GDM D.O.O., MIVKA 19, 1000 LJUBLJANA 38 0 38
HIŠNI SERVIS MATIČ D.O.O. LJUBLJANA-ČRNUČE, CESTA 24. JUNIJA 74, 1231 LJUBLJANA – ČRNUČE 994 173 28
DISTOR D.O.O., KAVČIČEVA ULICA 58, 1000 LJUBLJANA 2792 102 109
HEKER D.O.O., LJUBLJANA, CELOVŠKA CESTA 144, 1000 LJUBLJANA 3119 33 96
VEGRIM D.O.O., LJUBLJANA, MEDVEDOVA CESTA 25, 1000 LJUBLJANA 3519 11 18
T & S D.O.O., LJUBLJANA, PARMOVA ULICA 53, 1000 LJUBLJANA 944 50 14
MAVSAR D.O.O., GORAZDOVA ULICA 19, 1000 LJUBLJANA 1505 0 47
G.V. CENTER D.O.O. LJUBLJANA, BRNČIČEVA ULICA 31, 1231 LJUBLJANA – ČRNUČE 7701 16 23
EMONA PLUS D.O.O., ŠMARTINSKA CESTA 130, 1528 LJUBLJANA 4855 1096 60
MAG DOM GORAZD MAKAROVIČ S.P., PODPEČ 21, 1312 VIDEM-DOBREPOLJE 166 14 0
STANOVANJSKA ZADRUGA TOMAČEVO JARŠE , Z.O.O., TOMAČEVO 1, 1000 LJUBLJANA 8272 32 10
KOMSTAN D.O.O., LJUBJANA, CELOVŠKA CESTA 291, 1000 LJUBLJANA 786 0 460
DOMO, D.O.O., MAJORJA LAVRIČA ULICA 12, 1000 LJUBLJANA 7649 845 42
LORA – LJUBLJANA, D.O.O., ZALOŠKA CESTA 163, 1000 LJUBLJANA 3119 20 215
AL-MA LJUBLJANA, BRODARJEV TRG 3, 1000 LJUBLJANA 233 0 233
SCH – GROUP INVEST D.O.O., DUNAJSKA CESTA 63, 1000 LJUBLJANA 10 0 0
STANOVANJSKA ZADRUGA RTV Z.O.O., LINHARTOVA CESTA 17, 1000 LJUBLJANA 1521 0 952
REKO D.O.O., PARMOVA ULICA 39, 1000 LJUBLJANA 37 0 17
REYAN D.O.O. LJUBLJANA, NEUBERGERJEVA ULICA 3, 1000 LJUBLJANA 3889 60 167
FINANCE-OPERATIVA D.O.O., KNEZA KOCLJA ULICA 37, 1000 LJUBLJANA 3105 255 52
DOM EFEKT D.O.O., OBIRSKA ULICA 1A, 1000 LJUBLJANA 5988 0 0
EMONA UPRAVLJANJE D.O.O., ŠMARTINSKA CESTA 130, 1000 LJUBLJANA 2367 0 286
S & H D.O.O., LJUBLJANA, MLINSKA POT 19, 1231 LJUBLJANA – ČRNUČE 32 0 32
UPSO D.O.O., PRUŠNIKOVA ULICA 95, 1210 LJUBLJANA – ŠENTVID 1980 328 0
GOSPODAR D.O.O., TRŽAŠKA CESTA 42, 1000 LJUBLJANA 13859 332 6
ALTARA D.O.O., ULICA HERMANA POTOČNIKA 19, 1000 LJUBLJANA 56 56 0
LESNINA LGM SI D.O.O., PROLETARSKA CESTA 4, 1000 LJUBLJANA 266 12 14
KIDREAL D.O.O., ZALOŠKA CESTA 1, 1000 LJUBLJANA 780 0 0
STUPRA D.O.O. LJUBLJANA, KRAKOVSKI NASIP 22, 1000 LJUBLJANA 72 0 16
DOMCOMMERCE D.O.O., PUHOVA ULICA 1, 1000 LJUBLJANA 186 0 60
TERCA D.O.O., ŠENTRUPERT 124, 8232 ŠENTRUPERT 677 77 0
FERDOM D.O.O., DOLENJSKA CESTA 242, 1000 LJUBLJANA 154 154 0
TABOR LJUBLJANA, D.D., TABOR 9, 1000 LJUBLJANA 288 130 93
KREATOR POSLOVNI CENTER, D.O.O., LITIJSKA CESTA 38, 1119 LJUBLJANA 318 0 0
PRIMA PLUS D.O.O., SLOVENČEVA ULICA 93, 1000 LJUBLJANA 1696 72 34
GRUDA MARTIN PILIH S.P., VIDEM 1, 1380 CERKNICA 96 96 0
PUN NEPREMIČNINE D.O.O. LJUBLJANA, KERSNIKOVA ULICA 10A, 1000 LJUBLJANA 2140 38 410
TEVIS – AGENCIJA ZA KADRE, D.O.O., KOTNIKOVA ULICA 28, 1000 LJUBLJANA 706 22 0
IMOVINA, D.O.O., LJUBLJANA, GOSPOSVETSKA CESTA 10, 1000 LJUBLJANA 214 16 0
NAŠA SOSESKA D.O.O., BRILEJEVA ULICA 8, 1000 LJUBLJANA 1898 743 0
MARBONA D.O.O. LJUBLJANA, ZALOŠKA CESTA 269, 1000 LJUBLJANA 455 199 0
DOMPLAN, D.D., BLEIWEISOVA 14, 4000 KRANJ 1160 24 0
SINTAL-EKO D.O.O., LITOSTROJSKA CESTA 38, 1000 LJUBLJANA 993 22 13
CIGALE & CO. D.O.O., GORIŠKA ULICA 63, 1000 LJUBLJANA 1996 0 54
MINISOPH, D.O.O., LJUBLJANA, GOSPOSKA ULICA 3, 1000 LJUBLJANA 80 0 0
FIDELIS D.O.O. LJUBLJANA, ZIHERLOVA ULICA 39, 1000 LJUBLJANA 138 0 101
KRIČKA ANA-IVANA S.P. UPRAVLJANJE S STAN. ZGRADBAMI, CESTA V MESTNI LOG 38, 1000 LJUBLJANA 364 0 262
SEGA IGOR, ČERNETOVA ULICA 26, 1000 LJUBLJANA 10 0 0
JUSTIN GABRIJEL, FLANDROVA ULICA 10, 1210 LJUBLJANA – ŠENTVID 8 0 0
SKUPNOST LASTNIKOV STANOVANJ LINHARTOVA 66, LINHARTOVA CESTA 66, 1112 LJUBLJANA 112 0 0
LIKAR IN SIN D.O.O., EFENKOVA CESTA 61, 3320 VELENJE 78 0 78
KATIN D.O.O., ROJČEVA ULICA 26, 1000 LJUBLJANA 25 0 25
SO – 06 SMOLIČ STANISLAV D. N. O., JAKČEVA ULICA 2, 1000 LJUBLJANA 167 0 167
ARTTEL D.O.O., STANEŽIČE 2, 1210 LJUBLJANA – ŠENTVID 821 0 345
AGING 1, D.O.O., DUNAJSKA CESTA 109, 1000 LJUBLJANA 66 0 0
DIMIC PETER, ULICA STANETA SEVERJA 2A, 1000 LJUBLJANA 12 0 0
SKUPNOST LASTNIKOV BRATOVŠEVA PLOŠČAD 35 IN 37, BRATOVŠEVA PLOŠČAD 37, 1000 LJUBLJANA 116 0 0
ZORKO D.O.O. LJUBLJANA, AGROKOMBINATSKA CESTA 12, 1000 LJUBLJANA 8 0 0
DEAN DARABOŠ S.P., NOVE FUŽINE 33, 1000 LJUBLJANA 429 405 0
SIMON ZORE S.P., KRANJSKA CESTA 4, 4240 RADOVLJICA 26 26 0
IZHOD D.O.O. LJUBLJANA, PERIČEVA ULICA 13, 1000 LJUBLJANA 155 0 25
MMC & CO. D.O.O. LJUBLJANA, TABOR 9, 1000 LJUBLJANA 499 0 159
SKUPNOST LASTNIKOV HIŠE MUCHERJEVA 3, MUCHERJEVA ULICA 3, 1000 LJUBLJANA 16 0 16
DONOR D.O.O., BRILEJEVA ULICA 14, 1000 LJUBLJANA 117 5 20
SPO D.O.O., BLAŽEVA ULICA 3, 4220 ŠKOFJA LOKA 65 11 0
DOP D.O.O. LJUBLJANA, VOJKOVA CESTA 50, 1000 LJUBLJANA 150 0 0
SKUPNOST ETAŽNIH LASTNIKOV VOGELNA 8 LJUBLJANA, VOGELNA ULICA 8, 1000 LJUBLJANA 112 0 0
SKUPNOST LASTNIKOV STANOVANJ LINHARTOVA 62, LINHARTOVA CESTA 62, 1000 LJUBLJANA 112 0 0
ZVEZDA 18-21 D.O.O., ZVEZDA 18, 1210 LJUBLJANA – ŠENTVID 92 0 92
MAHIČ JASMIR, PAPIRNIŠKA POT 2, 1261 LJUBLJANA – DOBRUNJE 40 0 0
STANISLAV TAJČMAN S.P. UPRAVLJANJE STAN. HIŠ, PRVOMAJSKA ULICA 10, 1000 LJUBLJANA 120 0 0
SKUPNOST LASTNIKOV BRATOV UČAKAR 108/110, ULICA BRATOV UČAKAR 110, 1000 LJUBLJANA 96 0 96
LAPUH MILAN, VOJKOVA CESTA 11, 1000 LJUBLJANA 16 0 0
ODA NEPROFITNA STANOVANJSKA ZADRUGA Z.O.O., DUNAJSKA CESTA 101, 1000 LJUBLJANA 112 0 0
B TECH D.O.O., CESTA ANDREJA BITENCA 68, 1000 LJUBLJANA 43 31 0
MILAVC ZDENKO, VRHOVNIKOVA ULICA 11, 1000 LJUBLJANA 18 0 0
DROFENIK D.O.O., STEGNE 7, 1000 LJUBLJANA 250 202 0
LUTOVAC PLUS LJUBLJANA, IGRIŠKA ULICA 3, 1000 LJUBLJANA 18 0 0
MARIJA ZALOŽNIK S.P., CESTA V MESTNI LOG 38, 1000 LJUBLJANA 116 106 0
PINACEA D.O.O., MORAVČE PRI GABROVKI 64, 1274 GABROVKA 82 0 0
DOMA, D.O.O., GROSUPLJE, ZALOG PRI ŠKOFLJICI 2, 1291 ŠKOFLJICA 80 80 0
HAAD LJUTIČ&CO. K.D., KAŠELJSKA CESTA 121, 1260 LJUBLJANA – POLJE 12 0 0
ŠANJUG BARBKA, MAJDE VRHOVNIKOVE ULICA 22, 1000 LJUBLJANA 48 0 0
STANOVANJSKA ZADRUGA NAŠA HIŠA Z.B.O., JAKČEVA ULICA 39, 1000 LJUBLJANA 154 0 154
STUPAR STEVO, ŠMARTINSKA CESTA 15, 1000 LJUBLJANA 10 0 0
STANOVANJSKA ZADRUGA MARIBOR Z.O.O., KREKOVA ULICA 18, 2000 MARIBOR 32 0 0
BRANDT UPRAVLJANJE D.O.O., POLJANSKI NASIP 32, 1000 LJUBLJANA 16 0 0
KOMMUNIO D.O.O., METELKOVA ULICA 7B, 1000 LJUBLJANA 39 39 0
PRIMUS PROJEKT D.O.O., SREDNJEVAŠKA ULICA 85, 1291 ŠKOFLJICA 54 6 0
PRIMOS D.O.O. LJUBLJANA, SVETČEVA ULICA 14, 1000 LJUBLJANA 8 0 0
ISIS D.O.O., SLOVENSKA CESTA 17, 1000 LJUBLJANA 20 0 0
PECHER ANDREJ, VIDEMSKA ULICA 1, 1000 LJUBLJANA 6 0 0
DOLHAR MARIJA, FLANDROVA ULICA 8, 1210 LJUBLJANA – ŠENTVID 10 0 0
SZ GLINŠKOVA PLOŠČAD 22 Z.O.O. LJUBLJANA, GLINŠKOVA PLOŠČAD 22, 1000 LJUBLJANA 50 0 50
FINISTUS – BLATNIK&BLATNIK D.N.O, BEBLERJEV TRG 3, 1000 LJUBLJANA 190 0 0
BINO D.O.O., JAKČEVA ULICA 43, 1000 LJUBLJANA 308 0 0
PETAN JANEZ, KREMŽARJEVA 10, 1000 LJUBLJANA 6 0 0
DOM D.O.O., DELPINOVA ULICA 8, 5000 NOVA GORICA 20 0 0
MEZEK JANEZ, AGROKOMBINATSKA CESTA 4, 1000 LJUBLJANA 44 0 0
TERČELJ ANDREJ, TRUBARJEVA CESTA 64, 1000 LJUBLJANA 8 8 0
ELDERA, D.O.O., MEDENSKA CESTA 81E, 1000 LJUBLJANA 77 0 17
DOM SISTEMI D.O.O., ULICA LOJZETA HROVATA 3A, 4000 KRANJ 60 60 0
NATUM D.O.O. LJUBLJANA, CESTA V PODBORŠT 10, 1231 LJUBLJANA – ČRNUČE 10 0 0
UPIS D.O.O., DUNAJSKA CESTA 437, 1291 ŠKOFLJICA 5 5 0
SKUPNOST LASTNIKOV RUSJANOV TRG 9, RUSJANOV TRG 9, 1000 LJUBLJANA 558 0 0
KITAK, SI.INVEST K.D., LJUBLJANA, BRODARJEV TRG 7, 1000 LJUBLJANA 7 7 0
SREČKO KITAK S.P., BRODARJEV TRG 7, 1000 LJUBLJANA 30 0 0
SKUPNOST LASTNIKOV TRNOVSKA ULICA 8, TRNOVSKA ULICA 8, 1000 LJUBLJANA 51 0 51
ŽAVBI VINCENC, TOPNIŠKA ULICA 14, 1000 LJUBLJANA 26 0 26
HORVAT VLADIMIR, FLANDROVA ULICA 6, 1210 LJUBLJANA – ŠENTVID 16 0 0
ATAC D.O.O. LJUBLJANA, DUNAJSKA CESTA 104, 1000 LJUBLJANA 8 0 0
SKUPNOST ETAŽNIH LASTNIKOV STANOVANJSKE STAVBE SMOLETOVA 18, LJUBLJANA, SMOLETOVA ULICA 18, 1000 LJUBLJANA 98 0 0
SKUPNOST LASTNIKOV GORNJI TRG 20, GORNJI TRG 20, 1000 LJUBLJANA 14 0 0
SKUPNOST ETAŽNIH LASTNIKOV STANOVANJSKE STAVBE VOGELNA 10 LJUBLJANA, VOGELNA ULICA 10, 1000 LJUBLJANA 54 0 54
K”ITAK D.O.O., BRODARJEV TRG 7, 1000 LJUBLJANA 34 0 0
BLAZNIK VOJKO, FLANDROVA ULICA 7, 1210 LJUBLJANA – ŠENTVID 8 0 0
SKUPNOST LASTNIKOV, PAVŠIČEVA 6 LJUBLJANA, PAVŠIČEVA ULICA 6, 1000 LJUBLJANA 152 0 0
SKOMINA MARIJA, TRŽAŠKA CESTA 65, 1000 LJUBLJANA 9 9 0
UNIH D.O.O., FAJFARJEVA ULICA 33, 1000 LJUBLJANA 64 64 0
BOJAN GLAVAN S.P., VIKRČE 22, 1211 LJUBLJANA-ŠMARTNO 84 0 0

Could computers vote instead of parliamentary representatives?

Share Button

Haha, what a funny question. Of course they can’t. How can one teach a computer all the intricacies of lawmaking process, and trust it well enough to let it vote? This must surely be a recipe for disaster.

Yet, as I realized in previous research, the parties mostly demand ruthless discipline from their parliamentary representatives at voting time, simply to be able to actually govern in Slovenian multiparty democracy, where there’s never an absolute winner. This leads to coalition governments, where every vote counts towards a majority.

That means that in a polarized parliament, one could theoretically predict a representative’s vote by examining the votes cast by all other representatives. If an opposition party proposes to vote on an act, it’s very likely that members of government block will uniformly, or at least predominantly, vote against it, and vice versa. There are few exceptions to that rule, namely some profoundly ethical decisions, in which majority parties will let their members vote by conscience. But they are few and far apart.

Fun with neural networks

I decided to test this out by modeling some representatives by neural networks, and training the networks with a few voting sessions and their outcomes in the beginning of the parliamentary term.

Model for each representative was fed votes by every other rep except him- or herself as input, and his or her vote as desired output. This was repeated and repeated again for all hundred training sessions, until the model converged (loss fell under 0.05).

It was then shown voting sessions iz hasn’t seen yet, and tasked to predict the outcomes.

The results are shown in images below. For each representative, the image contains:

  • name and party,
  • training vector (the votes he/she cast in first 100 voting sessions – red for “against”, blue for “in favor”, yellow for absence for whatever reason),
  • actual votes (400 votes the network hasn’t seen and was trying to predict),
  • predicted votes (how the neural network thought the representative would vote), and
  • difference indicator (with red rectangles for wrong prediction, green rectangles for correct prediction, and yellow rectangles for absence)

I didn’t bother too much with statistics, to see who was the most predictable, neither did I try to predict voting for every rep.

In short, those with the mainly green bottom strip were the most predictable.

Government coalition

DeSUS

predict_kopmajerpredict_hrsak_ivanpredict_jenkojanapredict_karl_erjavec

SMC

predict_blazic_srecko predict_brglez_miran predict_dekleva_erika predict_kolesa_anita predict_lay_franc predict_zorman_brankopredict_vervega_vesna

SD

predict_mursic_bojanapredict_nemec_matjaz

Opposition

NSi

predict_novak_ljudmila predict_tonin_matej

 

IMNS

 

 

predict_horvat_jozef predict_goncz_laszlo

SDS

predict_jansa_janez predict_bah_zibert_anjapredict_cus_andrejpredict_breznik_francpredict_godec_jelka predict_mahnic_zanpredict_podkrajsek_bojanpredict_pojbic_marijanpredict_sircelj_andrej

ZL

predict_vatovec_matejpredict_trcek_francpredict_hanzek_matjazpredict_kordis_miha

ZAAB

predict_bratusek_alenka

predict_mesec_luka  predict_moderndorfer_janipredict_vilfan_peter

A cursory examination of results yields several realizations:

  • even in best predictions with lowest error rate, the model doesn’t predict absences well, especially for representatives with low incidence of absence in training data. This is intuitively understandable on two levels: first, it’s hard for the network to generalize something it didn’t observe, and second, absences can happen on a human whim, which is unreachable for a mathematical model. For representatives of opposition parties, who frequently engage in obstruction as a valid tactics, the model fares a little better.
  • the model predicts best the voting behavior of majjority party (SMC) members.
  • the model utterly fails to predict anything for representatives whowere absent in training period (duh).

So, could we substitute the actual representatives with simple neural networks? Not with this methodology. The problem is that we need votes of everyone else in the same session to predict the vote of modeled rep, so at the time of prediction, we already have their vote. We don’t have a way of inferring votes from scratch, or from previous votes.

We could, in theory, try to predict each rep’s vote independently from others by training the network on proposed acts’ texts. I speculate that a deeper network could correlate vectorized keywords in training texts with voting outcomes, and then be able to predict voting for each rep independently based on previous unseen texts. Maybe I’ll do that when I get the texts and learn a bit more. It’s still ANN 101 period for me.

I used a simple perceptron with 98 inputs (there have been 99 representatives in this term, counting also current ministers and substitutes), a hidden layer of 60 neurons, and a softmax classifier on the end.

As usual, I used Karpathy’s convnetjs for modeling, and d3 for visualization. Dataset comes from Zakonodajni monitor.

Ours and theirs – an endless duel between computer generated Slovenian news commenters

Share Button

I’ve been following comments under Slovenian web news items for quite some time. The commenters there are well known for their animosity towards anyone who disagrees with their political worldview. Reading the comment section usually means immersing yourself in verbal filth, depravity and all imaginable kinds of hate speech.

Some time ago I wrote software for scraping comments off these websites and have been since then storing them in a database. They are useful for a number of things. For example,  I’ve had some success with stylometry (identifying commenters by their writing style, even when they post under different name), but this is a matter of another post. I also helped SAZU compiling a list of new slang words for the new Dictionary of Slovene Language.

So here’s a lighter project for people who don’t read these comments, neither they want to. If you want to see it all, just click the image below and behold the auto-generated stream for a minute or two.

Note that these are not real comments. The text is generated from two Markov chains, which have been initialized with texts of left-wing and right-wing commenters.  The comments used are approximately a year old, lest someone accuses me of participating in election campaign of some kind. The web page simply generates a few sentences from one, then from the other, and so it continues ad nauseam infinitum.

I think it’s a fitting commentary of Slovenian mentality. Slovenian-speaking visitors will notice that, even if the texts are probabilistically computer-generated, there’s still ample hurling of insults based on the outcome of the last World War. There’s quite a lot of that.

Also, even though both sides pack serious vitriol, the right wingers use more classic hate speech, and they write comparatively worse.

See for yourself!

Naši in vaši
Naši in vaši

Technically, it was a breeze to make. First I pulled entire corpora of selected commenters from the database in text form, then I used RiTA, a generative text tool, for initializing the two models and generating sentences. The code is very short, most of it has to do with displaying and scrolling.

But initializing models from loaded text:

model.loadText(text);

And then generating sentences with just:

var sentences = model.generateSentences(nr);

Such is the beauty of RiTA.