Tagged: open data

State projects and county budget visualizations – a collaboration with Transparency International Slovenia

Share Button

These two projects are a result of recent collaboration with Transparency International Slovenia. The datasets were provided by the state, and I was asked to develop visualizations that would structure the information in an accessible way. Much help was also provided by members of Institut Jožef Štefan.

State project browser

The first project is a browser of all projects, initiated by state institutions, from 1991 on. The idea was to let users discover, where and for what purposes the money goes in their county. The dataset and visualization allow for exploration by various categories, as well as time.

The projects in the dataset also contain projects that are still in the planning phase, and won’t be completed until year 2025. With this tool, citizens can hopefully inspect the planned expenditures for roads, water sources, and other categories of infrastructure, culture and other fields of development, and compare that with their own expectations.

It allows browsing and filtering of projects by statistical regions and counties, as well as displaying the timeline of all projects, which is basically an  expandable version of a Gantt chart.

To see the interactive project website, click here, or click the image below.

State projects app
State projects app

The original data is provided on the project’s “About” page.

County budget browser

The new project is a straightforward visualization of county budgets. The budgets are displayed as dynamic, zoomable hierarchical (“sunburst”) diagrams. They react to each other, allowing a side-by-side comparison of budgets of two user-selected counties.

The visualization enables users to delve into expenses and incomes of all Slovenian counties on separate tabs.

To see the interactive project website, click here, or click the image below.

County budgets app
County budgets app

 

Technology and design

The data cleanup and preparation was done with some Python scripts. The sunburst diagram accepts hierarchical data in a tree format, so this provided an interesting exercise of converting a tabular dataset into a nested dictionary of optional depth.

The visualizations were done in d3, which is really an indispensable tool for any serious work in online visualization.

Both projects were minimalistically, yet expertly designed by Tomaž Plahuta (Bitnik, Eno).

Check out the projects and let me know your opinion in the comments!

Mapping parking infractions in Manhattan, NYC, by car make

Share Button

This is a technical explanation of procedure to map parking infractions in Manhattan for every available car make. To see the interactive visualization, click here, or click the image below. Otherwise read on.

Heatmaps for Audi and Bentley
Heatmaps for Audi and Bentley

Last year I published an Android app to enable Slovenian drivers to better avoid areas frequently inspected by parking wardens. It works by geolocating the user and then plotting issued paring tickets in the vicinity, with a breakdown by month, time of day and temperature on another screen. It was not a huge hit, but it did reasonably well for such a small country and no marketing budget.

I was thinking of making a version for New York City, but then abandoned the project. These visualizations are all that remains of it.

I started with downloading the data from New York Open Data repository. It’s here. The data is relatively rich, but it’s not geocoded. Luck had it that Mapbox just rolled out a batch geocoder at that time, and it was free with no quotas. So I quickly sent around 100,000 adresses through it and saved the results in a database for later use. The processed result is now available on Downloads page in form of JSON files, one per car make.

The actual drawing procedure was easier than I thought. I downloaded street data from New York GIS Clearinghouse and edited out everything but Manhattan with QGis.

First I tried a promising matrix approach, but I was unable to rotate the heatmap so that it would make sense. Here’s an example for Audi:

Matrix - Audi
Matrix – Audi

 

As you can see, it is a heatmap, but doesn’t look very good.

So I wrote a Python script that went through all street segments and awarded a point if there was an infraction closer that 100 meters from the relevant segment. Then I just used matplotlib to draw all the street segments, coloring them according to the maximum segment value.

A result for Audi now looks like this:

Audi
Audi

All that remained was drawing required images for animated GIFs, each for every hour for every car make. This was done with minimal modifications to original script (I learned Python multithreading in the process). The resulting images were then converted to animated GIFs with ImageMagic.

The whole procedure took approximately 12h of calculating and rendering time on a i7-6700 with 32 GB RAM. I guess I could shave several hours from that time, but I just let it run overnight.

See interactive version here, and tell me what you think in the comment section, if you feel like it.

 

Densities of different tree species in Ljubljana on minimaps

Share Button

This is a rework of the visualization I did for the Dnevnik newspaper. The Ljubljana government was generous enough to give us a location database with information on species and location of every tree within city limits. I thought it would be nice to render every species on its own map, so that the distributions can be compared.

Instead of just drawing points where each tree is, I calculated distance from each building to all the trees, and increased the building “score” if a tree was within 150 meters distance. Then I colored the buildings according to the score – the darker green it is, the more trees in its vicinity.

See the detailed version by clicking here or the image below.

Tree densities: example minimaps
Tree densities: example minimaps

The aforementioned article depicted the areas with higher potential for causing allergenic reactions due to specific tree species that grow there, but it also has a detailed map with every building colored in proportion with its distance from trees in vicinity.

See the article by clicking here or the image.

Dnevnik article screenshot
Dnevnik article screenshot

 

Map of apartment building management companies in Ljubljana

Share Button

Some time ago, I did an analysis of building manager companies for dnevnik.si. It was an interesting project, because there’s much controversy surrounding this topic. Some of these companies have been in bitter disputes with the residents and competing companies over their business, generating a lot of newspaper coverage.

We did a simple analysis to deterimine market shares and relative winners and losers in comparison to market leader. Here’s an image of final page in the newspaper. Click it to open the PDF.

upravniki_dnevnik
Newspaper page in Dnevnik.si

 

We also did a lot of other stuff that didn’t find their way into the final article. I’m now publishing an interactive map, showing the distribution of these companies across town. There’s usually one per apartment building. For this map, I only processed the apartment buildings with more than one apartment, because that’s where business is.

Click on the image below to open the map in a new tab, or click here.

Click for interactive map of apartment building managers
Click for interactive map of apartment building managers

Click on a building, and a popup will display the address and relevant history. It’s also possible to choose a company, and the map will refresh, displaying only the buildings in the selected company’s management. There’s also a button to display only those buildings where managers changed in the interval from years 2005 to 2015.

There’s also a graphic showing how many buildings a company gained from other companies. Here’s a screenshot. Click will take you to the same page as image above.

Who gained boildings from whom in Ljubljana
Who gained buildings from whom in Ljubljana

 

The original dataset came from GURS, but then we were able to acquire an update from the city government. We decided to use that, because they gave us three different updates, each five years apart, from which we were able to calculate the differences. You can see them in table below.

  • total flats: how many apartments are managed by a company,
  • gained: how many flats were gained by the company from 2005 to 2015,
  • lost: how many flats were lost by the company from 2005 to 2015.

Original data is here.

Company Total flats Gained Lost
SPL D.D., FRANKOPANSKA ULICA 18A, 1519 LJUBLJANA 107130 38 429
INSTA D.O.O., KOPRSKA ULICA 94, 1000 LJUBLJANA 2338 0 183
METALKA STANOVANJSKE STORITVE D.O.O., INDUSTRIJSKA CESTA 1, 1000 LJUBLJANA 22529 669 382
SZ RUSKI CAR D.O.O., GLINŠKOVA PLOŠČAD 22, 1000 LJUBLJANA 5515 256 182
PRACTIC D.O.O., LJUBLJANA, CELOVŠKA CESTA 147, 1000 LJUBLJANA 8355 92 511
VALINA UPRAVLJANJE, D.O.O., LITIJSKA CESTA 45, 1000 LJUBLJANA 480 72 10
DONF D.O.O., MAJORJA LAVRIČA ULICA 12, 1000 LJUBLJANA 6891 167 741
STANDOM ENGINEERING OTRIN D.O.O., BRNČIČEVA ULICA 13, 1231 LJUBLJANA – ČRNUČE 5661 113 20
AKTIVA UPRAVLJANJE D.O.O., LJUBLJANSKA CESTA 12F, 1236 TRZIN 8764 335 52
ŽELVA D.O.O., SAMOVA ULICA 9, 1000 LJUBLJANA 13550 187 236
UPRADOM D.O.O., LINHARTOVA CESTA 13, 1000 LJUBLJANA 3141 187 0
MASTRA D.O.O., ZALOŠKA CESTA 153, 1120 LJUBLJANA 6566 198 260
VALOR D.O.O., LJUBLJANA, ROZMANOVA ULICA 1, 1000 LJUBLJANA 351 26 42
M.I.S. D.O.O. LJUBLJANA, PROLETARSKA CESTA 4, 1000 LJUBLJANA 3999 318 22
LORA, D.O.O., ZALOŠKA CESTA 163, 1000 LJUBLJANA 5084 0 24
TENIT, D.O.O., LJUBLJANA, GAŠPERŠIČEVA ULICA 23, 1120 LJUBLJANA 2631 154 146
TABOR UPRAVLJANJE IN VZDRŽEVANJE D.O.O., TABOR 9, 1000 LJUBLJANA 4729 71 0
UVG D.O.O., LINHARTOVA CESTA 17, 1000 LJUBLJANA 7189 304 503
MANICOM D.O.O., POVŠETOVA ULICA 65, 1000 LJUBLJANA 118 0 34
KOVAČEVIČ MARIJA, MAJDE VRHOVNIKOVE ULICA 18, 1000 LJUBLJANA 48 0 0
PATE, D.O.O., TOPNIŠKA ULICA 19, 1000 LJUBLJANA 1867 0 800
STANDOM D.O.O., ULICA ANE ZIHERLOVE 6, 1000 LJUBLJANA 5306 48 462
SKUPNOST LASTNIKOV POT K RIBNIKU 16-22, POT K RIBNIKU 16, 1000 LJUBLJANA 656 0 0
GDM D.O.O., MIVKA 19, 1000 LJUBLJANA 38 0 38
HIŠNI SERVIS MATIČ D.O.O. LJUBLJANA-ČRNUČE, CESTA 24. JUNIJA 74, 1231 LJUBLJANA – ČRNUČE 994 173 28
DISTOR D.O.O., KAVČIČEVA ULICA 58, 1000 LJUBLJANA 2792 102 109
HEKER D.O.O., LJUBLJANA, CELOVŠKA CESTA 144, 1000 LJUBLJANA 3119 33 96
VEGRIM D.O.O., LJUBLJANA, MEDVEDOVA CESTA 25, 1000 LJUBLJANA 3519 11 18
T & S D.O.O., LJUBLJANA, PARMOVA ULICA 53, 1000 LJUBLJANA 944 50 14
MAVSAR D.O.O., GORAZDOVA ULICA 19, 1000 LJUBLJANA 1505 0 47
G.V. CENTER D.O.O. LJUBLJANA, BRNČIČEVA ULICA 31, 1231 LJUBLJANA – ČRNUČE 7701 16 23
EMONA PLUS D.O.O., ŠMARTINSKA CESTA 130, 1528 LJUBLJANA 4855 1096 60
MAG DOM GORAZD MAKAROVIČ S.P., PODPEČ 21, 1312 VIDEM-DOBREPOLJE 166 14 0
STANOVANJSKA ZADRUGA TOMAČEVO JARŠE , Z.O.O., TOMAČEVO 1, 1000 LJUBLJANA 8272 32 10
KOMSTAN D.O.O., LJUBJANA, CELOVŠKA CESTA 291, 1000 LJUBLJANA 786 0 460
DOMO, D.O.O., MAJORJA LAVRIČA ULICA 12, 1000 LJUBLJANA 7649 845 42
LORA – LJUBLJANA, D.O.O., ZALOŠKA CESTA 163, 1000 LJUBLJANA 3119 20 215
AL-MA LJUBLJANA, BRODARJEV TRG 3, 1000 LJUBLJANA 233 0 233
SCH – GROUP INVEST D.O.O., DUNAJSKA CESTA 63, 1000 LJUBLJANA 10 0 0
STANOVANJSKA ZADRUGA RTV Z.O.O., LINHARTOVA CESTA 17, 1000 LJUBLJANA 1521 0 952
REKO D.O.O., PARMOVA ULICA 39, 1000 LJUBLJANA 37 0 17
REYAN D.O.O. LJUBLJANA, NEUBERGERJEVA ULICA 3, 1000 LJUBLJANA 3889 60 167
FINANCE-OPERATIVA D.O.O., KNEZA KOCLJA ULICA 37, 1000 LJUBLJANA 3105 255 52
DOM EFEKT D.O.O., OBIRSKA ULICA 1A, 1000 LJUBLJANA 5988 0 0
EMONA UPRAVLJANJE D.O.O., ŠMARTINSKA CESTA 130, 1000 LJUBLJANA 2367 0 286
S & H D.O.O., LJUBLJANA, MLINSKA POT 19, 1231 LJUBLJANA – ČRNUČE 32 0 32
UPSO D.O.O., PRUŠNIKOVA ULICA 95, 1210 LJUBLJANA – ŠENTVID 1980 328 0
GOSPODAR D.O.O., TRŽAŠKA CESTA 42, 1000 LJUBLJANA 13859 332 6
ALTARA D.O.O., ULICA HERMANA POTOČNIKA 19, 1000 LJUBLJANA 56 56 0
LESNINA LGM SI D.O.O., PROLETARSKA CESTA 4, 1000 LJUBLJANA 266 12 14
KIDREAL D.O.O., ZALOŠKA CESTA 1, 1000 LJUBLJANA 780 0 0
STUPRA D.O.O. LJUBLJANA, KRAKOVSKI NASIP 22, 1000 LJUBLJANA 72 0 16
DOMCOMMERCE D.O.O., PUHOVA ULICA 1, 1000 LJUBLJANA 186 0 60
TERCA D.O.O., ŠENTRUPERT 124, 8232 ŠENTRUPERT 677 77 0
FERDOM D.O.O., DOLENJSKA CESTA 242, 1000 LJUBLJANA 154 154 0
TABOR LJUBLJANA, D.D., TABOR 9, 1000 LJUBLJANA 288 130 93
KREATOR POSLOVNI CENTER, D.O.O., LITIJSKA CESTA 38, 1119 LJUBLJANA 318 0 0
PRIMA PLUS D.O.O., SLOVENČEVA ULICA 93, 1000 LJUBLJANA 1696 72 34
GRUDA MARTIN PILIH S.P., VIDEM 1, 1380 CERKNICA 96 96 0
PUN NEPREMIČNINE D.O.O. LJUBLJANA, KERSNIKOVA ULICA 10A, 1000 LJUBLJANA 2140 38 410
TEVIS – AGENCIJA ZA KADRE, D.O.O., KOTNIKOVA ULICA 28, 1000 LJUBLJANA 706 22 0
IMOVINA, D.O.O., LJUBLJANA, GOSPOSVETSKA CESTA 10, 1000 LJUBLJANA 214 16 0
NAŠA SOSESKA D.O.O., BRILEJEVA ULICA 8, 1000 LJUBLJANA 1898 743 0
MARBONA D.O.O. LJUBLJANA, ZALOŠKA CESTA 269, 1000 LJUBLJANA 455 199 0
DOMPLAN, D.D., BLEIWEISOVA 14, 4000 KRANJ 1160 24 0
SINTAL-EKO D.O.O., LITOSTROJSKA CESTA 38, 1000 LJUBLJANA 993 22 13
CIGALE & CO. D.O.O., GORIŠKA ULICA 63, 1000 LJUBLJANA 1996 0 54
MINISOPH, D.O.O., LJUBLJANA, GOSPOSKA ULICA 3, 1000 LJUBLJANA 80 0 0
FIDELIS D.O.O. LJUBLJANA, ZIHERLOVA ULICA 39, 1000 LJUBLJANA 138 0 101
KRIČKA ANA-IVANA S.P. UPRAVLJANJE S STAN. ZGRADBAMI, CESTA V MESTNI LOG 38, 1000 LJUBLJANA 364 0 262
SEGA IGOR, ČERNETOVA ULICA 26, 1000 LJUBLJANA 10 0 0
JUSTIN GABRIJEL, FLANDROVA ULICA 10, 1210 LJUBLJANA – ŠENTVID 8 0 0
SKUPNOST LASTNIKOV STANOVANJ LINHARTOVA 66, LINHARTOVA CESTA 66, 1112 LJUBLJANA 112 0 0
LIKAR IN SIN D.O.O., EFENKOVA CESTA 61, 3320 VELENJE 78 0 78
KATIN D.O.O., ROJČEVA ULICA 26, 1000 LJUBLJANA 25 0 25
SO – 06 SMOLIČ STANISLAV D. N. O., JAKČEVA ULICA 2, 1000 LJUBLJANA 167 0 167
ARTTEL D.O.O., STANEŽIČE 2, 1210 LJUBLJANA – ŠENTVID 821 0 345
AGING 1, D.O.O., DUNAJSKA CESTA 109, 1000 LJUBLJANA 66 0 0
DIMIC PETER, ULICA STANETA SEVERJA 2A, 1000 LJUBLJANA 12 0 0
SKUPNOST LASTNIKOV BRATOVŠEVA PLOŠČAD 35 IN 37, BRATOVŠEVA PLOŠČAD 37, 1000 LJUBLJANA 116 0 0
ZORKO D.O.O. LJUBLJANA, AGROKOMBINATSKA CESTA 12, 1000 LJUBLJANA 8 0 0
DEAN DARABOŠ S.P., NOVE FUŽINE 33, 1000 LJUBLJANA 429 405 0
SIMON ZORE S.P., KRANJSKA CESTA 4, 4240 RADOVLJICA 26 26 0
IZHOD D.O.O. LJUBLJANA, PERIČEVA ULICA 13, 1000 LJUBLJANA 155 0 25
MMC & CO. D.O.O. LJUBLJANA, TABOR 9, 1000 LJUBLJANA 499 0 159
SKUPNOST LASTNIKOV HIŠE MUCHERJEVA 3, MUCHERJEVA ULICA 3, 1000 LJUBLJANA 16 0 16
DONOR D.O.O., BRILEJEVA ULICA 14, 1000 LJUBLJANA 117 5 20
SPO D.O.O., BLAŽEVA ULICA 3, 4220 ŠKOFJA LOKA 65 11 0
DOP D.O.O. LJUBLJANA, VOJKOVA CESTA 50, 1000 LJUBLJANA 150 0 0
SKUPNOST ETAŽNIH LASTNIKOV VOGELNA 8 LJUBLJANA, VOGELNA ULICA 8, 1000 LJUBLJANA 112 0 0
SKUPNOST LASTNIKOV STANOVANJ LINHARTOVA 62, LINHARTOVA CESTA 62, 1000 LJUBLJANA 112 0 0
ZVEZDA 18-21 D.O.O., ZVEZDA 18, 1210 LJUBLJANA – ŠENTVID 92 0 92
MAHIČ JASMIR, PAPIRNIŠKA POT 2, 1261 LJUBLJANA – DOBRUNJE 40 0 0
STANISLAV TAJČMAN S.P. UPRAVLJANJE STAN. HIŠ, PRVOMAJSKA ULICA 10, 1000 LJUBLJANA 120 0 0
SKUPNOST LASTNIKOV BRATOV UČAKAR 108/110, ULICA BRATOV UČAKAR 110, 1000 LJUBLJANA 96 0 96
LAPUH MILAN, VOJKOVA CESTA 11, 1000 LJUBLJANA 16 0 0
ODA NEPROFITNA STANOVANJSKA ZADRUGA Z.O.O., DUNAJSKA CESTA 101, 1000 LJUBLJANA 112 0 0
B TECH D.O.O., CESTA ANDREJA BITENCA 68, 1000 LJUBLJANA 43 31 0
MILAVC ZDENKO, VRHOVNIKOVA ULICA 11, 1000 LJUBLJANA 18 0 0
DROFENIK D.O.O., STEGNE 7, 1000 LJUBLJANA 250 202 0
LUTOVAC PLUS LJUBLJANA, IGRIŠKA ULICA 3, 1000 LJUBLJANA 18 0 0
MARIJA ZALOŽNIK S.P., CESTA V MESTNI LOG 38, 1000 LJUBLJANA 116 106 0
PINACEA D.O.O., MORAVČE PRI GABROVKI 64, 1274 GABROVKA 82 0 0
DOMA, D.O.O., GROSUPLJE, ZALOG PRI ŠKOFLJICI 2, 1291 ŠKOFLJICA 80 80 0
HAAD LJUTIČ&CO. K.D., KAŠELJSKA CESTA 121, 1260 LJUBLJANA – POLJE 12 0 0
ŠANJUG BARBKA, MAJDE VRHOVNIKOVE ULICA 22, 1000 LJUBLJANA 48 0 0
STANOVANJSKA ZADRUGA NAŠA HIŠA Z.B.O., JAKČEVA ULICA 39, 1000 LJUBLJANA 154 0 154
STUPAR STEVO, ŠMARTINSKA CESTA 15, 1000 LJUBLJANA 10 0 0
STANOVANJSKA ZADRUGA MARIBOR Z.O.O., KREKOVA ULICA 18, 2000 MARIBOR 32 0 0
BRANDT UPRAVLJANJE D.O.O., POLJANSKI NASIP 32, 1000 LJUBLJANA 16 0 0
KOMMUNIO D.O.O., METELKOVA ULICA 7B, 1000 LJUBLJANA 39 39 0
PRIMUS PROJEKT D.O.O., SREDNJEVAŠKA ULICA 85, 1291 ŠKOFLJICA 54 6 0
PRIMOS D.O.O. LJUBLJANA, SVETČEVA ULICA 14, 1000 LJUBLJANA 8 0 0
ISIS D.O.O., SLOVENSKA CESTA 17, 1000 LJUBLJANA 20 0 0
PECHER ANDREJ, VIDEMSKA ULICA 1, 1000 LJUBLJANA 6 0 0
DOLHAR MARIJA, FLANDROVA ULICA 8, 1210 LJUBLJANA – ŠENTVID 10 0 0
SZ GLINŠKOVA PLOŠČAD 22 Z.O.O. LJUBLJANA, GLINŠKOVA PLOŠČAD 22, 1000 LJUBLJANA 50 0 50
FINISTUS – BLATNIK&BLATNIK D.N.O, BEBLERJEV TRG 3, 1000 LJUBLJANA 190 0 0
BINO D.O.O., JAKČEVA ULICA 43, 1000 LJUBLJANA 308 0 0
PETAN JANEZ, KREMŽARJEVA 10, 1000 LJUBLJANA 6 0 0
DOM D.O.O., DELPINOVA ULICA 8, 5000 NOVA GORICA 20 0 0
MEZEK JANEZ, AGROKOMBINATSKA CESTA 4, 1000 LJUBLJANA 44 0 0
TERČELJ ANDREJ, TRUBARJEVA CESTA 64, 1000 LJUBLJANA 8 8 0
ELDERA, D.O.O., MEDENSKA CESTA 81E, 1000 LJUBLJANA 77 0 17
DOM SISTEMI D.O.O., ULICA LOJZETA HROVATA 3A, 4000 KRANJ 60 60 0
NATUM D.O.O. LJUBLJANA, CESTA V PODBORŠT 10, 1231 LJUBLJANA – ČRNUČE 10 0 0
UPIS D.O.O., DUNAJSKA CESTA 437, 1291 ŠKOFLJICA 5 5 0
SKUPNOST LASTNIKOV RUSJANOV TRG 9, RUSJANOV TRG 9, 1000 LJUBLJANA 558 0 0
KITAK, SI.INVEST K.D., LJUBLJANA, BRODARJEV TRG 7, 1000 LJUBLJANA 7 7 0
SREČKO KITAK S.P., BRODARJEV TRG 7, 1000 LJUBLJANA 30 0 0
SKUPNOST LASTNIKOV TRNOVSKA ULICA 8, TRNOVSKA ULICA 8, 1000 LJUBLJANA 51 0 51
ŽAVBI VINCENC, TOPNIŠKA ULICA 14, 1000 LJUBLJANA 26 0 26
HORVAT VLADIMIR, FLANDROVA ULICA 6, 1210 LJUBLJANA – ŠENTVID 16 0 0
ATAC D.O.O. LJUBLJANA, DUNAJSKA CESTA 104, 1000 LJUBLJANA 8 0 0
SKUPNOST ETAŽNIH LASTNIKOV STANOVANJSKE STAVBE SMOLETOVA 18, LJUBLJANA, SMOLETOVA ULICA 18, 1000 LJUBLJANA 98 0 0
SKUPNOST LASTNIKOV GORNJI TRG 20, GORNJI TRG 20, 1000 LJUBLJANA 14 0 0
SKUPNOST ETAŽNIH LASTNIKOV STANOVANJSKE STAVBE VOGELNA 10 LJUBLJANA, VOGELNA ULICA 10, 1000 LJUBLJANA 54 0 54
K”ITAK D.O.O., BRODARJEV TRG 7, 1000 LJUBLJANA 34 0 0
BLAZNIK VOJKO, FLANDROVA ULICA 7, 1210 LJUBLJANA – ŠENTVID 8 0 0
SKUPNOST LASTNIKOV, PAVŠIČEVA 6 LJUBLJANA, PAVŠIČEVA ULICA 6, 1000 LJUBLJANA 152 0 0
SKOMINA MARIJA, TRŽAŠKA CESTA 65, 1000 LJUBLJANA 9 9 0
UNIH D.O.O., FAJFARJEVA ULICA 33, 1000 LJUBLJANA 64 64 0
BOJAN GLAVAN S.P., VIKRČE 22, 1211 LJUBLJANA-ŠMARTNO 84 0 0

Could computers vote instead of parliamentary representatives?

Share Button

Haha, what a funny question. Of course they can’t. How can one teach a computer all the intricacies of lawmaking process, and trust it well enough to let it vote? This must surely be a recipe for disaster.

Yet, as I realized in previous research, the parties mostly demand ruthless discipline from their parliamentary representatives at voting time, simply to be able to actually govern in Slovenian multiparty democracy, where there’s never an absolute winner. This leads to coalition governments, where every vote counts towards a majority.

That means that in a polarized parliament, one could theoretically predict a representative’s vote by examining the votes cast by all other representatives. If an opposition party proposes to vote on an act, it’s very likely that members of government block will uniformly, or at least predominantly, vote against it, and vice versa. There are few exceptions to that rule, namely some profoundly ethical decisions, in which majority parties will let their members vote by conscience. But they are few and far apart.

Fun with neural networks

I decided to test this out by modeling some representatives by neural networks, and training the networks with a few voting sessions and their outcomes in the beginning of the parliamentary term.

Model for each representative was fed votes by every other rep except him- or herself as input, and his or her vote as desired output. This was repeated and repeated again for all hundred training sessions, until the model converged (loss fell under 0.05).

It was then shown voting sessions iz hasn’t seen yet, and tasked to predict the outcomes.

The results are shown in images below. For each representative, the image contains:

  • name and party,
  • training vector (the votes he/she cast in first 100 voting sessions – red for “against”, blue for “in favor”, yellow for absence for whatever reason),
  • actual votes (400 votes the network hasn’t seen and was trying to predict),
  • predicted votes (how the neural network thought the representative would vote), and
  • difference indicator (with red rectangles for wrong prediction, green rectangles for correct prediction, and yellow rectangles for absence)

I didn’t bother too much with statistics, to see who was the most predictable, neither did I try to predict voting for every rep.

In short, those with the mainly green bottom strip were the most predictable.

Government coalition

DeSUS

predict_kopmajerpredict_hrsak_ivanpredict_jenkojanapredict_karl_erjavec

SMC

predict_blazic_srecko predict_brglez_miran predict_dekleva_erika predict_kolesa_anita predict_lay_franc predict_zorman_brankopredict_vervega_vesna

SD

predict_mursic_bojanapredict_nemec_matjaz

Opposition

NSi

predict_novak_ljudmila predict_tonin_matej

 

IMNS

 

 

predict_horvat_jozef predict_goncz_laszlo

SDS

predict_jansa_janez predict_bah_zibert_anjapredict_cus_andrejpredict_breznik_francpredict_godec_jelka predict_mahnic_zanpredict_podkrajsek_bojanpredict_pojbic_marijanpredict_sircelj_andrej

ZL

predict_vatovec_matejpredict_trcek_francpredict_hanzek_matjazpredict_kordis_miha

ZAAB

predict_bratusek_alenka

predict_mesec_luka  predict_moderndorfer_janipredict_vilfan_peter

A cursory examination of results yields several realizations:

  • even in best predictions with lowest error rate, the model doesn’t predict absences well, especially for representatives with low incidence of absence in training data. This is intuitively understandable on two levels: first, it’s hard for the network to generalize something it didn’t observe, and second, absences can happen on a human whim, which is unreachable for a mathematical model. For representatives of opposition parties, who frequently engage in obstruction as a valid tactics, the model fares a little better.
  • the model predicts best the voting behavior of majjority party (SMC) members.
  • the model utterly fails to predict anything for representatives whowere absent in training period (duh).

So, could we substitute the actual representatives with simple neural networks? Not with this methodology. The problem is that we need votes of everyone else in the same session to predict the vote of modeled rep, so at the time of prediction, we already have their vote. We don’t have a way of inferring votes from scratch, or from previous votes.

We could, in theory, try to predict each rep’s vote independently from others by training the network on proposed acts’ texts. I speculate that a deeper network could correlate vectorized keywords in training texts with voting outcomes, and then be able to predict voting for each rep independently based on previous unseen texts. Maybe I’ll do that when I get the texts and learn a bit more. It’s still ANN 101 period for me.

I used a simple perceptron with 98 inputs (there have been 99 representatives in this term, counting also current ministers and substitutes), a hidden layer of 60 neurons, and a softmax classifier on the end.

As usual, I used Karpathy’s convnetjs for modeling, and d3 for visualization. Dataset comes from Zakonodajni monitor.