RCE: Visualizing the Results, Part 1

(This post will be updated as new data – late submissions – come in. Another post analysing the correlation between our regime data and data from other human rights datasets will appear later - check back on BB and the blog for more in the next week. Ask questions below!!!!).

(Last updated 2018-06-02 16:16:11)

Measures and basic correlations

We have now gathered enough data to start answering the research question you selected at the beginning of the term. You designed a classification system in tutorials, and individually gathered data on lots of countries (143 collected by 106 students so far to be exact). And so far, the quality of the data is truly excellent; a measure of democracy extracted from your answers correlates at 0.89 with the polyarchy index of the V-Dem (“Varieties of Democracy”) dataset (the gold standard in political science), which is really fabulous – the best it’s ever been in this class! (Remember correlation coefficients go from -1 to 1; higher numbers mean greater correlations; a correlation of .9 is huge, and a correlation of 0 means no relationship). Your data also correlates at 0.89 with Freedom House’s numbers, and at 0.87 with Polity IV, two other highly professional datasets used to measure democracy. Well done!

To construct this measure, I grouped together all questions in the survey that had to do with elections, the judiciary, freedom of expression, and the like: (questions 1-6, “To what extent is the government accountable to its population?”, “How much control does your regime have over its judiciary?”, “To what extent does your regime have freedom of the press?”, “Do you have any freedom of assembly/political expression?”, “To what extent does your regime allow other political parties to compete fairly in elections?”, “Is there universal adult suffrage?”). I simply averaged your answers together for each year and country and rescaled them to the 0-1 range.

In figure 1 below, you can see the correlations of the POLS209 measure of democracy with several other measures: the Freedom House index of freedom, the Polity IV measure of political institutions, and several of the V-Dem indexes of democracy (v2x_ variables):

This is not bad at all; as you can see, the correlations between professionally-collected measures of democracy are sometimes a bit higher (often a little above 0.9), but your correlations with these other measures are very high correlation (indeed, higher than the correlation between some of these large-scale indexes and some of the more obscure measures of democracy out there). Professionals do not always agree about whether a country is democratic or not. These results are great!

More specifically, the POLS209 measure of democracy correlates best with “liberal” measures of democracy (the “polyarchy” measure of democracy, Freedom House, polity), and least well with “egalitarian” measures of democracy (V-Dem’s measure of participatory [v2x_partip] and egalitarian democracy [v2x_egal]). So it’s clearly measuring something about the standard “liberal” conception of democracy.

You also answered six other questions in the survey. I thought question 7 (“To what extent is power concentrated in the leader?”) was a measure of personalism more than democracy; questions 8 and 10 (“To what extent is the economy under state control?” and “Is there a state-enforced ideology, and if so, is there punishment for opposing it?”) measured totalitarianism; question 11 (“How influential is the military in decisions?”) measured whether the regime was a military one; and questions 9 and 12 (“How prevalent is torture and degrading treatment in the justice system?” and “To what extent does a regime protect the equality of its population?”) measured respect for other human rights. These measures are all highly but not perfectly correlated with your measure of democracy:

More “militaristic”, “personalistic”, and “totalitarian” countries are less likely to be democratic; and other human rights (torture and equal treatment) are more likely to be respected in more democratic countries, at least by your measure.

Time trends

Most countries were classified by only one student, but some were rated at different time periods by two or more students. Some of you chose longer periods, others chose shorter periods of time, but most chose periods of at least 10 years (median 13 years, min 2 years, max 70), which gives us a reasonable amount of regime data (2800 country-years, including some transitions to democracy or to authoritarianism, since the periods chosen by you often overlapped for a given country).

The simplest thing to do with this data is simply to average each student rating per year and plot the results per country, showing the variation over time and space in the POLS209 “democracy score”. In the figure below, the coloured lines represent your measure of democracy, while the light grey broken lines represent the V-Dem “polyarchy” measure of liberal democracy. I’ve split your measure into three categories (democracy, hybrid, non-democracy) for ease of visualization:

As you can see, your measure is extraordinarily close to V-Dem’s judgments in most cases; mostly you’re a bit more generous, but not much. Some notable disagreements I can see are Haiti (where you claim the country is more authoritarian today than V-dem judges), Iran and Yugoslavia (where V-Dem thinks the country was less democratic than you), Lesotho (where your coding is more constant than V-Dem’s, which thinks there was a lot more change in the period of study), and Mozambique (another country where you are a lot more generous than V-Dem).

The raw data (including justifications, by Username and country) for this post is available here. The processed dataset (including the measure of democracy I calculated, as well as the measures of personalism, militarism, and totalitarianism) is available here. The first file may be useful if you want to peruse the justifications given for particular countries; the second may be useful if you want to do any further analysis (but you can probably ignore it unless you are technically adept).

I will be adding more analysis as the week goes on, but this should get you started.