Weird social science
Are we mostly studying the contemporary US?
One of the most important lines of research in recent social science is the discovery of the distinct psychology of WEIRD (Western, Educated, Industrialized, Rich, Democratic) societies (Henrich 2020, Muthukrishna et al. 2020). These scholars further claim that fields like psychology suffer from a massive selection problem by conducting their research almost exclusively on WEIRD peoples. This limited purview produces a biased and incomplete picture of human psychology and ignores the holistic and kin-based thinking that characterized most of human history.
Although their critique has been leveled at the field of psychology, it is arguably more general. I wondered how it applied across the social sciences. To what extent are different social science ignoring historical and non-WEIRD populations and how might this be limiting their understanding of how human beings organize their societies, politics, and economics?
As a first cut at the problem, I put together data on the sources of the main empirical evidence in articles published in the 2020 volume of the flagship journals of the main social science fields.1 I looked both at geography (basically America and the West versus the non-West) and time period (contemporary = 2000s, post-WWII, and pre-WWII).2 The figure below shows the percentage of articles focusing on non-Western countries and the pre-WWII period, while the table gives more complete results.
Figure 1: Percentages of articles on non-Western countries & pre-WWII period in flagship journals of social science fields (2020)
Here is my take on the results, starting with regions:
Sociology is the most Americanized field followed by Psychology, while Anthropology and surprisingly for me History were relatively un-Americanized. Economics and Political Science were somewhere in the middle. This result becomes even stronger when we include other developed Western countries. Sociology and Psychology show almost no representation of the developing and non-Western world (6% and 5% of all articles respectively). This confirms Muthukrishna et al’s (2020) criticism of psychology, but it might be leveled at Sociology as well.
Conversely, Anthropology not surprisingly covers the developing world most extensively, but Economics and Political Science perform relatively well on this dimension with about one-third of their articles focusing on non-Western countries. History’s one quarter mark is reasonable, though it is worth noting that many of these studies deal with colonialism often using the archives of the colonial power (in some cases I assigned half weight to the Western and the developing categories). History is of course handicapped by its reliance on archives which are in short supply for many developing countries.
Turning to time periods, we see the following:
All five social sciences are very contemporary. The most contemporary are Psychology and Anthropology with almost all work using evidence from the 21st century, again confirming Muthukrishna et al’s criticisms. Of course, the four-field division of Anthropology also includes historical fields like archaeology and others that can use historical data like physical anthropology or linguistics. Perhaps I was just looking at the wrong journal because those fields were underrepresented in my sample.
Conversely, History had no articles focusing on the present and by far the largest percentage investigating the pre-war period, though a majority still emphasize the postwar period. Economics, Political Science, and Sociology were relatively similar to each other in their contemporary focus (about three-quarters of articles). Political science may have been slightly better in considering the long run, though many of these pieces were driven by aggregate data.
The one empty quadrant in the figure is the upper right where we would expect historical work on the non-Western world. This area should be filled by other branches of anthropology like physical anthropology and archaeology (the ‘bones’ and ‘stones’ to go along with ‘tones’ and ‘thrones’). For some reason they were not well-represented in the issues of The American Anthropologist that I surveyed. (I might add that the figure is missing its entire top half – that is, fields that devote more than 50% of their flagship journal space to deeper history.)
What can we conclude? I’d note first that Political Science comes closest to the center of the figure (unless we extend the y-axis to 100%). So, we are obviously the best and have the most general and objective view of the human condition.
Joking aside, arguably the extreme results are driven by methods, particularly for Psychology and Anthropology. The dominant use of experiments in Psychology and ethnographic fieldwork in Anthropology means that most work will be contemporary. History’s reliance on archives also limits it to the past and to the existence of archival materials. Similarly, Political Science and Economics depend on evidentiary sources like public opinion polls and macroeconomic data that are modern inventions.
It is worth noting that some recent trends like the credibility revolution which advocate experimentally based approaches may come at the expense of work on the past (though scholars have sometimes discovered historical natural experiments). We should be alert to the tradeoffs between precision of inference and scope.
However, there may be some twenty-dollar bills lying on the sidewalk. I would have expected Sociology to have a more diversified portfolio and in general more fields could probably focus on cases and evidence outside of the very recent past and the US in order to get a more complete picture of human nature(s) and societies.
Some differences may also be driven by the organization of each discipline. Political Science arguably benefits from having a prominent subfield (comparative politics) that focuses on non-US politics and History similarly divides itself more by region or country than time period. Area studies has its uses in encouraging scholars to survey the range of human experience. Even Economics has a relatively prominent field of development. Sociology might benefit from something similar; I’m not sure why it didn’t build a major comparative discipline after WWII like political science. A colleague suggests that it was perhaps a desire to stay close to its cases.
The digitization of archives may be another trend which induces more quantitatively oriented social scientists into the past as we can see with the Broadstreet blog and the new Journal of Historical Political Economy.
I won’t speculate much about whether the differences in what we study affects our conclusions to paraphrase Barbara Geddes. Like Henrich and his collaborators, I am curious whether more study of the past or non-Western societies will reveal different psychologies, politics, and modes of social organization that challenge the universality of our current theories. My colleague Hendrik Spruyt has a new book out that looks to do exactly this.
One could put a more provocative frame on these results. A good portion of social scientists claim to be concerned with inequalities (including possibly some entire disciplines like Sociology). And yet studying the US and Western Europe means that they are mainly studying the richest people in the world. Branko Milanovic points out that two out of every three Americans are in the top decile of the world income distribution.
It is not just WEIRDness that is at stake in what we study.
Caveats: I’d note that the analysis here is inevitably limited. It simply tracks one year in one journal for each field (approximately 30-60 articles) and I am not certain how representative these journals are of their fields. Further, my coding was a bit half-assed. As I found myself wasting more and more time, I started to cut corners. I was just curious if it was worth probing further. Watch this space for updates and extensions.
These are the American Economic Review, the American Historical Review, the American Political Science Review, the American Sociological Review, the American Anthropologist, and Psychological Science.
The Western category includes mainly Western Europe plus Australia and New Zealand. I excluded articles without empirical evidence, mostly theoretical or methodological pieces and a few meta-analyses. I also excluded invited articles and articles published under special rubrics. I added the 2019 volume for History because the 2020 volume included very few regular articles. I limited Psychology to the first six months of 2020 because of the large number of articles. In a few cases, I assigned partial credit to multiple categories - e.g., a study of British colonialism in India, a quantitative study of all the countries in the world, or data that stretches from 1800 to 2000.