Do political scientists think differently about politics?
Or, biases in how ordinary people think
Do political scientists have a distinctive way of looking at politics? I am not thinking here about whether they are ideologically different than the average citizen - more liberal or conservative - but about whether they systematically use different standards in assessing politics and see different cause and effect relationships in how politics works.
Insofar as political science has developed a distinctive wissenschaft not to mention real knowledge about politics, arguably political scientists should see politics differently than most untrained citizens. On the other hand, political science seems to lack the sort of defined worldview that one finds among economists or even sociologists. It is hard to imagine political scientists writing books like The Economist’s View of the World (Rhoads 1985) or The Sociological Imagination (Mills 2000), which can be easily written about these fields.
These questions are important because if we accept that political scientists are experts in their field, then the fact that the public holds different views may bode poorly for democracy. If citizens are using the wrong standards for assessing democracy or for looking at cause and effect, then they may be electing or supporting candidates and parties that harm their welfare. This, of course, depends on our accepting that political scientists actually possess expert knowledge.
To date it has been difficult to study the distinctiveness of the political science worldview because of the absence of data to systematically compare political scientists and ordinary citizens. Recently, the Bright Line Watch surveys have remedied this gap. Over the course of the Trump presidency, they have queried political scientists and the public about the nature of democracy and the current functioning of democracy in the US and elsewhere.
A preliminary glance at their main results reveals both similarities and differences between these groups. Somewhat surprisingly, the US public and political scientists are in broad agreement on their ranking of the importance of key elements of democracy, how democracy in the US is evolving (i.e., negatively), and how democratic other countries are.
Yet, we can also see some divergences. Political scientists are more likely than citizens to see formal institutions as essential to democracy and less likely to see informal norms and behavior as important. If we follow Caplan in viewing expert-public discrepancies as biases on the part of the public, we might call this an anti-institutional bias among the public. Relatedly, the public seems more concerned with corruption and morality in politics than political scientists who focus more on policy consequences. Put another way, the public is more likely to embrace Weber’s ethic of ultimate ends compared to political scientists who follow the ethic of responsibility. This might be called an anti-consequentialist bias. Finally, there is some evidence of greater pessimism in the public. Citizens (even Trump supporters) are less sanguine than political scientists in their evaluation of US democracy.
In short, political scientists may think in different ways than the public, though they may not be as different in their realm as economists are in theirs.
There has been relatively little study of the world views of political scientists. The problem has been the lack of survey research that compares their views to those of others.
A number of surveys do consider both the public and elites, particularly foreign policy elites, but these elites are often practitioners rather than academics. Page and Bouton (2006), for example, analyze a set of surveys of foreign policy elites and the public conducted by the Chicago Council on Global Affairs. They find that elites tend to be more favorable toward unilateral actions by the US and less supportive of international institutions than the public, though it is unlikely that this difference would apply to academic experts who tend to be internationally-minded.
Meanwhile Kertzer (2020) has recently conducted a meta-analysis of studies comparing elites and the public. His main result is overall similarities after controlling for relevant differences in the demographics of these groups. Again, however, these elites are not usually political scientists.
There are a handful of surveys of political scientists, but they tend not to be accompanied by surveys of the public and they tend to focus on issues only of concern to political scientists. One could include here the TRIP surveys of international relations specialists and a forthcoming survey of social scientists by Grossman that includes political scientists. Caplan et al. (2013) is one of the few explicit comparisons of political scientists and the public. They find that the public tends to overestimate the influence of most political actors, which might make it hard for them to hold politicians accountable.
The closest parallel to the kind of study that I have in mind is Caplan’s The Myth of the Rational Voter (2008). He compares the responses of citizens and economists to a set of identical questions about factors that are affecting the economy. He finds substantial differences in their replies even after controlling for education and ideology. This seems to indicate that economists do think differently, though Caplan’s main takeaway is that citizens suffer from certain biases rather than that economists are distinctive. He calls them pessimistic bias (a tendency to overestimate the severity of economic problems), make-work bias (a tendency to underestimate the benefits of conserving labor), anti-foreign bias (a tendency to underestimate the economic benefits of interactions with foreigners), and anti-market bias (tendency to underestimate the benefits of the market mechanism).
Can we find similar differences in the way that the public and political scientists think about politics? It is not obvious what sort of differences we might find. We could imagine a similar kind of pessimism among ordinary people as Caplan finds, though the public and experts may simply be pessimistic about different sorts of things. Similarly, one could imagine the public being more anti-foreign than political scientists, though Page and Bouton find considerable internationalism among the public. My prior would be that the public evaluates politics in a more moralistic or deontological way than political scientists - they focus on individual sins rather than structural issues. Arguably this is what is behind Caplan’s make-work and anti-market-bias.
Alternatively, it is possible that political scientists are not so different from citizens who share their ideological inclinations and level of education. Given the diversity of the field and the lack of a ruling paradigm like rationality in economics, the political science way of thinking may not be so distinctive or may be more influenced by ideology, especially when considering events close to home. It is even possible that political scientists may be the ones suffering from biases.
A recent set of surveys by Bright Line Watch are some of the first that allow us to directly compare the public with experts in political science. They have simultaneously polled political scientists and the public eight times between October 2017 and March 2020 (along with three surveys of experts alone). Their main questions concern the key components of democracy and the current state of American democracy along with a number of other issues polled less frequently. Their questions are somewhat less than ideal in that democracy is a contested concept, yet it is hard to think of a political concept that is not contested.
The analyses below are simply a first pass at these issues. I mostly consider how political scientists differ from citizens (and sometimes Trump supporters) on average. A deeper analysis would control for demography and ideology a la Caplan as well as consider time trends. To ease interpretation, I will follow Caplan in referring to differences between experts and the public as biases of the public, but as I mentioned before this interpretation might be contested.
In the first place, we can note that there are a lot of similarities between experts and the public (the numbers and graphs below are from Carey et al. 2019, though some results come from the Bright Lines website). There is a strong correlation (r=0.77) in their rankings of how essential or important 27 principles are for democracy, though the agreement is stronger on principles viewed as less important by experts (see Figure 2 below). Further, assessments of the current functioning of American democracy - both in general and in particular areas - have declined over time for both groups (see Figure 1 below). And both groups mostly agree in their ratings of democracy in a sampling of other countries. Overall, there is a good deal of alignment between experts and the public, something that is not as apparent in a comparison of economists and the public.
On to the differences. The public does tend to be more pessimistic than experts on the state of American democracy, which is one overlap with Caplan’s results. As Figure 1 shows, experts’ ratings of American democracy average around 10 percentage points higher than the public’s (on a scale of 0 to 100) over these eight surveys. Surprisingly, experts are more positive about the state of US democracy than even self-identified Trump supporters.
To explain this we might argue that political scientists see sources of stability in the maintenance of basic rights and key institutions not to mention historical legacies, where the public focuses on surface conflicts and daily scandals. This optimism may thus be connected to the institutional focus of political scientists that we discuss below. There is some confirmation of this in more specific ratings where the public tended to have a lower mean rating on the 27 individual principles (though also greater compression). The public sees worse performance than political scientists just about everywhere.
But how specific is this pessimism to the current environment? The Bright Line surveys focus on a particularly polarized and conflictual era in US politics. Can we imagine times when the public was more optimistic about US democracy than experts? One that comes to mind is the aftermath of 9/11 or other periods where there are rally-round-the-flag effects (which are arguably not so common). Another might be the beginning of the Reagan era. I would want more confirmation before I rest my hat on the public being more pessimistic than experts. Nevertheless, the Trump era does present a hard test for this bias as political scientists tend to be very liberal and anti-Trump (Grossman, forthcoming).
Looking in more detail, we might also say that the public has an anti-institutional bias relative to experts. As Figure 2 shows, experts were almost unanimous in viewing freedom of speech, freedom of the press, the right to protest, and the principle that agencies do not punish protest as essential to democracy (97-98% agreed), while only a large majority (73-82%) of the public shared these views. Conversely, the public tended to see informal norms or practices like a common understanding of facts, high participation, and seeking compromise as more important to democracy than political scientists (though both groups’ rankings of these aspects were on the low end overall). Noteworthy is that the public’s views on the importance of these principles were not polarized; Trump supporters and opponents mostly agree on what is important. This difference between experts and the public may thus be more durable than the Trump era.
Figure 2: The Importance of Characteristics for Democracy
The same anti-institutionalism goes for assessments of current US democracy. As Figure 3 shows, experts were much more positive than the public about how the US was fulfilling the key institutional principles of democracy (all parties allowed, protest tolerated, free speech, fraud-free elections, no interference with the press). The public, by contrast, was considerably more optimistic than experts on America’s performance in participation, seeking of compromise, and common understanding of facts, all arguably aspects of behavior more than institutions. There was, however, one set of institutional issues where this pattern reversed: the electoral system. On votes having equal impact and districts being biased, the public was more sanguine than experts.
Figure 3: Assessments of US Democracy Today
Compared to experts, the public thus put less weight on institutions as essential to democracy and saw the US performing less well on these institutions. Their conception of democracy was more behavioralist. While political scientists have mostly adopted the Schumpeterian/Dahlian view of democracy that rests on formal rules, citizens are closer to a Civic Culture or even Rikerian view that depends on attitudes and beliefs.
The authors of the survey also calculated what they call Bright Lines, areas where there was a divergence between the importance a group attributed to a principle and their assessment of performance on that principle. In particular, they tried to single out areas where an issue was perceived as important but the US was seen as performing poorly. They view these areas as ones where that group might draw a line in the sand against further erosion.
For experts, these areas mainly concerned equality of voting rights and checks and balances (in particular, votes having equal impact, districts being unbiased, legislative checks on the executive, and constitutional limits on the executive). By contrast, for the public bright lines mainly related to political accountability - officials not using office for private gain, sanctions for official misconduct, and investigations not being compromised by politics.
These results seem to confirm one of my expectations, which is that the public cares more about moral behavior - honesty, lack of corruption - than political scientists. Many political scientists have likely internalized Weber’s ethic of responsibility where the important thing is outcomes and politicians often have to do seemingly bad things to achieve good ends. As I often remind friends, “There are worse things than corruption.” By contrast, the public may follow Weber’s ethic of ultimate ends in their judgments of politicians. They want good Christian behavior and do not distinguish the political from other realms. One might call this an anti-consequentialist (or pro-deontological) bias.
One can see some indications of this in the survey questions that asked respondents their opinions on issues of constitutional hardball - actions that are arguably constitutional but break important norms. Though expert and mass opinions were again correlated in their ranking of how democratic these actions would be, there were some divergences. In particular, the public was more accepting than experts of refusing to raise the debt limit), while experts were much more likely to approve the abolishment of the filibuster. Experts seem to focus on actions with bad economic consequences (though they were also more likely to approve of the minority party using the filibuster to regularly block legislation).
Figure 4: Opinions on Constitutional Hardball
In short, there is some evidence that the public is more concerned with morality in politicians, while political scientists care more about consequences, even if the means to achieve them are imperfect.
Low Calibration Bias
A final difference between experts and the public is more formal. Experts tended to use the whole scale in their evaluations both of the importance of particular principles for democracy and their assessment of these principles. Expert evaluations of the importance of particular factors for democracy ranged from the 40s to nearly 100 on a 100-point scale, while the public tended to stick to the middle ground, from the 50s to the 70s. On assessments of the US, political scientists ranged from single digits to the 80s and the public from the teens to 60. Experts also changed their judgments to a larger extent over time - they became more pessimistic about US democracy - and they were more discriminating in where they became pessimistic. This makes sense as political scientists are better schooled in both conceptual formation and are more informed. Not surprisingly, their opinions were better calibrated. Tetlock (2015) argues that calibration and granularity tend to be associated with better predictions.
I can imagine a few objections to these results. In the first place, for such contested concepts like democracy, we cannot say that the expert way of viewing things is the correct one. In turn, it may not be the public that is biased. By contrast, the effects of particular factors on economic growth - the main subject of the surveys that Caplan analyzes - arguably has a correct answer (or at least more and less wrong answers). The political science way of looking at the world is backed up by some arguments that I think are relatively persuasive - i.e., Weber on the political vocation, Schumpeter and Dahl on democracy, and so on - but they are not so persuasive that we should call the public irrational or biased as Caplan does.
Second, I am not sure how much these results are peculiar to the Trump era. Most political scientists are opponents of Trump and Trump is an extremely polarizing figure. Some of the results above apply equally to Trump supporters and opponents, so there may be some generalizability. But I would want to see how evaluations change during a hopefully less polarized Biden presidency. A next step would be to conduct some Caplan-like analyses of the enlightened preferences of the public - that is, their preferences if they were more like political scientists - but the Bright Line Watch surveys have a limited number of demographic variables with which to conduct these analyses.
In sum, I believe that there are three substantive ways and one formal way in which political scientists differ from the public in their view of democracy. They are more optimistic about politics, they judge democracies more based on institutions than behavior, and they care more about outcomes than morality. If we accept political scientists as experts in this area, we could say that the public suffers from pessimistic bias, anti-institutional bias, and anti-consequentialist bias. Experts are also more calibrated in their judgments, but this is more a formal than a substantive property.
These differences also coexist with substantial similarities. Political scientists may be different, but not all that different. And it is not as easy to call the public biased on these normatively controversial issues. It is even possible that the public’s way of judging politics has its own rationale (see Fearon 1999). In the future, it would be interesting to explore differences on issues of cause and effect in politics where political scientists would arguably have a greater advantage.
Caplan, Bryan. 2011. The Myth of the Rational Voter: Why Democracies Choose Bad Policies-New Edition. Princeton University Press.
Caplan, Bryan, Eric Crampton, Wayne A. Grove, and Ilya Somin. 2013. "Systematically biased beliefs about political influence: Evidence from the perceptions of political influence on policy outcomes survey." PS: Political Science & Politics 46(4): 760-767.
Carey, John M., Gretchen Helmke, Brendan Nyhan, Mitchell Sanders, and Susan Stokes. 2019. "Searching for bright lines in the trump presidency." Perspectives on Politics 17(3): 699-718.
Fearon, James D. 1999. "Electoral accountability and the control of politicians: selecting good types versus sanctioning poor performance." In Bernard Manin, Adam Przeworski, and Susan Stokes, eds. Democracy, accountability, and representation. New York: Cambridge University Press.
Kertzer, Joshua D. 2020. "Re-Assessing Elite-Public Gaps in Political Behavior." American Journal of Political Science.
Mills, C. Wright. 2000. The sociological imagination. Oxford University Press.
Page, Benjamin I., and Marshall M. Bouton. 2008. The foreign policy disconnect: What Americans want from our leaders but don't get. University of Chicago Press.
Rhoads, Steven E. 1985. The economist's view of the world. Cambridge University Press.