Tagged: language

Visualizing drug talk on bluelight.ru

Share Button

In mainstream media, there’s not a lot to be found about recreational drugs except horror stories and arguments for prohibition. From time to time we also hear that Steve Jobs liked to drop acid when he was young, that countless Vietnam vets easily kicked heroin habit upon coming home, and, as US-fed-sponsored study found out, that psychedelic mushrooms can bring a lasting and positive personality change in more than half of those who take them.

Where to find good information? There exist internet communities, so-called harm-reduction forums, where one can spend a few hours to discover that the truth is not black and white. Surely junkies exist, and using meth daily is not a life strategy anyone could recommend, but not all drugs were created equal. There are many classes of recreational drugs, each acting on specific chemical pathways in body – uppers on dopamine, hallucinogens on serotonin, downers on GABA, etc.

Mapping drugs

I thought it would be nice to visualize these drug groups based on what users of harm-reduction forums say, so I analyzed around 1.2 million posts on bluelight.ru and constructed a simple diagram that tells a lot. It was constructed in such a way that drugs that are frequently mentioned together, appear together. Circle radii are proportional with frequency of appearance of the same drugs in the posts. Methodology is explained at the bottom of the post.

Here’s the diagram, pan and zoom at will:

Click here to peruse a clickable, searchable version of the same diagram (give it a second to load). To download a high-resolution image (8000 x 6000), click here (black) or here (white).

The drug groups are color coded for better readability. Starting from the top:

  • light blue group: mostly antidepressives – SSRIs such as Prozac (fluoxetine), Zoloft and such.
  • violet group:  mainly contains benzodiazepines such as Xanax, Valium, and Lorazepam, which are commonly abused, but there are a lot of other downers there.
  • orange group: opiates and opioids, soch as heroin, oxycontin and the like. There were so many mentions of “opiates” without referring to a specific chemical that I considered it would be a pity to leave the word out.
  • dark yellow group on the right: mostly dissociatives such as ketamine and DXM, but there’s also a subgroup on the right side. It forms a larger group, mixed with differently colored drugs, that could be called “shamanic corner”, as it mostly contains so-called entheogens and natural concoctions such as ayahuasca.
  • light orange group: mainly nootropics such as Piracetam. Some use them to enhance a psychedelic or MDMA experience, but they have a more general use as memory, intelligence and sensory enhancers.
  • red group: I don’t know what to call this, but these are “working man’s drugs”. The common drugs that we hear about in the media. Some of these drugs are not considered drugs at all, for example alcohol and tobacco, but the Bluelight discussions show that they are very common. Thinking about it, one must have something to drink while one insufflates synthetic powders, and a cigarette is also a good thing to have while waiting for something stronger to take hold.
  • green group: psychedelic drugs such as shrooms, LSD, DMT and mescaline, along with many newer variations and analogs, such as 2C-X family, the DMT analogs and the whole Tihkal inventory.
  • blue group: Ecstasy (MDMA) and newer stimulants and entactogens, such as methylone, mephedrone, etc. “Plant foods” and “bath salts” are in this category.

Mapping effects

Simply mapping out the drugs is nice, but additional step seemed in order: mapping coincidence of various effects the drugs have on users. Again, posts were analyzed, but in addition to drugs, some (not all!) common effects were extracted and mapped in a network. Result is in the diagram below. Darker dots are effects, lighter are drugs. Size is again proportional to number of mentions in all posts.

Click here to peruse a clickable, searchable version of the same diagram. To download a high-resolution image, click here (black) or here (white).

Note that above diagram does not indicate semantic relationships between drugs and their effects. For example, why is “marijuana” close to “death”? Maybe there was a lot of talk about fear of death that the marijuana experience helps to resolve, or maybe people like to describe how they are dying of laughter while smoking weed. I honestly don’t know. I suspect it’s because of close relationship between mentions (not necessarily use!) of marijuana and those of alcohol, cocaine and methamphetamine, which could have a more significant relation with death or dying.
What’s really notable is heavy clustering of adverse effects around opiates, and relative absence of same around psychedelics. Based on Bluelight data, I can safely conclude that psychedelic drugs do not cause users to complain a lot, except maybe mentioning hallucinations and visuals, but, well …

Drug use over the years

My whole database contains posts from 2010 until March 2013. Here’s an analytical tool to better understand what’s going on in the recreational drug market community. Time is on horizontal axis, while the proportion of posts mentioning specific drug relative to all posts in that month is on the vertical axis.

Play around with interactive chart to discover emerging trends, or simply to behold the wax and wane of specific chemicals as they compete for users’ neurological apparatuses, while their manufacturers are temporarily evading ever stricter analog laws:

Commentary: Bluelight is a harm reduction forum, historically established for the users to be able to tell a good Ecstasy pill from the bad, so MDMA is the most mentioned drug. Use of “classic” drugs doesn’t change much, but it’s interesting to note the rise of new “research chemicals” such as NBOME family, new cathinones (3-MMC), new synthetic canabinoids (STS-135) and different amphetamines, prevalently methamphetamine. You can also see how the newly banned drugs, for example mephedrone, go out of use, and their analogs, in this case 3-MMC, replace them.

Methodology and tools

First, all the Bluelight forums were crawled and contents, dates and other metadata of all posts put into a SOLR index. That took approximately two days of not too aggressive load on their server (thanks Bluelight for not banning my IP).
To make first two network diagrams, undirected graphs were constructed with JGraphT library so that all extracted entities – drugs and effects – in every post were connected as nodes. Mentions of all extracted entities were counted to make the dots size show frequencies, not network degrees. That yielded complete graphs to be visualized with Gephi. Gephi files were exported to a TileMill-friendly format to render map tiles. Tiles are displayed on the site using Leaflet.
To make the interactive chart, SOLR was used to produce time series. Data was then packed into suitable format for the Flot library to be able to display.
To extract entities, two dictionaries were used – one for drugs, one for effects. You can download them here: drugs / effects.
If anyone is interested in the SOLR core, I can put it on Dropbox. Send me a note, my email is on the About page.

What is not here, but could be

  • analysis of effects that specific drugs have over time
  • a chart of effects only
  • some different visualization that could help to establish relationships between specific drugs and effects they have. For example, it’s been known for some time that mephedrone and various dragonflies have vasoconstrictive effects. Maybe some other relationship could be inferred that way.
  • first map should be clickable to search on Wikipedia, I’ll add that as soon as I figure out the Wax lib.

I may revisit this theme in the future.

Some pics:
drug_effects_diagram

Drug talk visualizations

Exploring Hollywood values through IMDB genres and tags

Share Button

A typical Hollywood story always portraits life in a twisted way. Movies are infused with values. There are typical stories: justice always prevails in the end, even if it means the death of a good guy; the coming-of-age story, in which hero becomes a man, the revenge story, in which the hero is wronged in the beginning, and must regain his life and justice in the course of the film. In American movies, family values are all-important, and so on.

These values are interrelated in the movie world, but what is their importance relative to other values? Is war a good or a bad thing, as portrayed in the movies? Is friendship close to romance, and is marriage close to love? What is science fiction – action, adventure or fantasy?

There happens to be a treasure trove of useful information on IMDB to visualize these relations. Each movie belongs to one or more genres, and on every movie page, there are tags for themes that occur in it. One could construct a network of movies that are interrelated through genres and tags they share. If two films share a tag, they must be closer than films that don’t share it. But there are many tags and over ten genres, so how does it look?

It looks like this (click image to launch interactive page):

 
Network graph

Roughy 15,000 movies, as presented on IMDB. Full map in a bigger window. There’s also a post showing how a social network of actors evolves over time from 1960 to 2013.

If a circle is bigger, it means it has more connections (movies associated with it). For example, The “Drama” tag seems to be the biggest, because apparently a big part of movies are dramas.

Same-colored circles belong to common categories, so for example “Drama”, “Romance”, “Love”, “Friendship”, “Marriage” and surprisingly “History” and “Biography” belong to the same group. Romance and drama are actually genres, and “love”, “history” and “biography” are tags. If you zoom in, you can see the movies associated with each tag and category.

It seems that most of Hollywood romance takes place in New York City, and that there’s a lot of sex going on there at the same time. There is some friendship involved, but not much. It’s interesting that marriage is on the opposite side of romance in relation to sex. It also seems that there is a lot of romantic activity on the set, as actors and actresses are closely related to it. This must be an artifact of Hollywood self-reflection.

On the other hand, California, as represented in movies, seems much more family-oriented. There’s a lot of boys, children, girls and babies around it. There’s also a lot of dreams and female nudity.

It’s also fun to construct sentences containing words of closely positioned tags. Drugs and money lead to suicide? Death by doctor in a hospital? Murder someone, get apprehended by police and go to prison?

It’s also apparent that sci-fi is nothing but a sub-genre of adventure. I always thought there are more brains to it. And fantasy seems nothing more than adventure for family audiences.

Have fun browsing the map, and let me know if you discover more fun facts.

Recent experiments with Gephi led me to speculate that it’s possible to extract meaning from a large volume of data with network analysis. This is a first post in this blog, and also the first in a series dealing with data and visualization.

If you are interested in making your own diagrams like that, here’s a how-to.

Edit: after being mentioned in Canadian Business magazine (thanks Matthew McClearn!), I should maybe add an explanation of my interpretation above. I’m actually interpreting relationships as portrayed in the movies, as inserted in the IMDB database. So what I wrote was actually an interpretation of an aggregation of simplifications of interpretations. I still think that my methodology is sound – after all, the diagram looks OK, and actually makes sense. It’s just curious to interpret.