Boston, Massachusetts Mar 26, 2022 (Issuewire.com) - On March 2, 2022, the Varieties of Democracy (V-Dem) Project released its latest, updated version of its award-winning dataset, which provides historical and modern-day data of 202 countries on all sorts of important aspects of politics, including governing systems, political equality, media and academic freedom, civil liberties and civil society, state sovereignty, and the ways in which political power, state services, personal freedoms, and wealth are distributed among people (both citizens and non-citizens).
Since 2014, I’ve been honored to be one of the experts invited to contribute annually to Varieties of Democracy’s award-winning dataset. There are so many reasons I like this project, but one of the reasons is a methodological one—in other words, the way the project carefully gathers its data and how it assesses the validity of these data. Anyone can write a survey (really) and ask for responses. We hear marketing and journalistic claims of “the survey says…” or “nine out of ten people say…” all the time. But not all surveys are equal in their scientific strength. A lot has to happen to make sure that a survey collects accurate information from a representative group of people and that we as scientists know how to understand and analyze this information.
Overall, the V-Dem Project invites participation from 3,700 experts who have extensively lived and/or worked in their country of expertise. This is important because what V-Dem is trying to collect is well-informed academic and practical knowledge, and these experts are well-positioned to provide it.
But—and this is crucial—even experts can be wrong. Experts can be misinformed or their knowledge could be partial or outdated. And, experts—just like every other human being on this planet—can be biased (sometimes without even knowing it), based on their life experiences, cultural knowledge, and inner philosophy. What I love about the V-Dem Project is how it has created ways of checking the accuracy and bias level of the expertise given.
For example, after every question—whether it is about the percentage of territory that the state effectively controls, the monopolization of power by certain groups (such as the wealthy or a particular ethnicity or gender), the balance between meritocracy and favoritism in state administrative appointments, etc.—V-Dem asks the expert to rate, on a scale of 1-100, how sure they are about this answer. The scale is broken up into sections like, “There is very little chance I am wrong” or “I am fairly certain about this, but experts have substantial disagreements.” This option allows experts to offer their knowledge with some humility (a rare opportunity in academia!), and it also provides valuable information for the V-Dem coders, who are able to “weight” the experts’ answers based on the strength of their surety that they are providing accurate information. Weighting means that not all answers are equally averaged together, but rather the answers that are offered with more certainty end up being counted more than the answers offered with less certainty.
Of course, sometimes the people who are the most wrong in life are the ones who think they are the most right. (Sigh!) No amount of self-assessment will help these people see that perhaps there is a chance they might be mistaken about a piece of knowledge. So there is another way that the V-Dem Project checks the accuracy of its knowledge experts—by using vignettes to assess the level of bias that an expert may bring (consciously or unconsciously) to the survey. Vignettes—short stories or examples—are a way of checking how each individual understands the concepts and questions under study. I’ve used the technique of vignettes in my own grant-funded original surveys to better understand Qatari responses to questions about economic well-being and political efficacy (see, for example, my article with colleague Justin Gengler of Qatar University in the International Journal of Public Opinion Research). In the same way, the V-Dem Project provides an opportunity for its experts to answer vignette questions about hypothetical countries to get a sense of how each expert is biased toward answering questions about political matters. This individual bias can then be taken into account, just like the level of certainty, when averaging and analyzing the collected expertise as a whole.
Another thing I love about the V-Dem Project is that it shares its data freely. If you want to try your hand at statistical analysis, you can freely download all of their datasets in various formats, and you can also use V-Dem’s online graphing tools to create your own visual images of the data. For those of us who prefer to let the stats experts do their stats thing, you can download and read the analysis of the V-Dem Project team in their Democracy Report 2022 at this link. Entitled “Autocratization Changing Nature?”, the Executive Summary points out:
- The level of democracy enjoyed by the average global citizen in 2021 is down to 1989 levels. The last 30 years of democratic advances are now eradicated.
- Dictatorships are on the rise and harbor 70% of the world population – 5.4 billion people.
- The EU may be facing its own wave of autocratization – 20% of members are autocratizing.
- The “epidemic of coups” in 2021 suggests that autocrats are becoming bolder.
- Polarization escalates towards toxic levels in 40 countries.
These conclusions are concerning, but knowledge is power! Becoming aware of the worldwide trends in governance, freedoms, and polarization is the first step toward active global citizenship. In that respect, the Varieties of Democracy Project does a great service for us all.
Media Contact
Jocelyn Sage Mitchell *****@yahoo.com
