Abstract

We organized two evaluations of the paper: “Zero-Sum Thinking, the Evolution of Effort-Suppressing Beliefs, and Economic Development” [1]. We asked two expert reviewers from different disciplines to evaluate it, who ended up being fairly negative (especially E1) but who both made substantive critiques regarding the assumptions, conceptual framework, data analysis, and interpretation. To read these evaluations, please see the links below.

Evaluations

1. Anonymous Evaluation 1

2. Anonymous Evaluation 2

Overall ratings

We asked evaluators to provide overall assessments as well as ratings for a range of specific criteria.

I. Overall assessment (See footnote1)

II. Journal rank tier, normative rating (0-5): On a ‘scale of journals’, what ‘quality of journal’ should this be published in?2 Note: 0= lowest/none, 5= highest/best.

See “Metrics” below for a more detailed breakdown of the evaluators’ ratings across several categories.5 See here for the current full evaluator guidelines, including further explanation of the requested ratings.6

Evaluation summaries

Anonymous evaluator 1

At its core, this paper formalizes and tests a theory posing that the poor are poor because they do not work hard enough. I describe conceptual, theoretical, and empirical issues.

Anonymous evaluator 2

The paper introduces zero-sum environments to explain demotivating beliefs proposing a model and using data from Congo and the World Values Survey to support the model. It assumes demotivating beliefs to be universally incorrect, which is unsupported. In contexts with corruption or instability, hard work may not yield proportional outcomes, therefore demotivating beliefs may reflect true states of the world, thus negating the need for their model.

Metrics

Ratings

See here for details on the categories below, and the guidance given to evaluators.

Journal ranking tiers

See here for more details on these tiers.

Evaluation manager’s discussion

This paper came to the Unjournal’s attention because it is ambitious (in a good way), linking culture — or more precisely belief systems — to productivity, material welfare, and ultimately long-run growth. The authors constitute a strong interdisciplinary team, and the paper has been getting attention. We asked two expert reviewers from different disciplines to evaluate it, who ended up being fairly negative (especially E1) but who both made substantive critiques regarding the assumptions, conceptual framework, data analysis, and interpretation.

In particular both [evaluators] felt that the umbrella term of ‘demotivating beliefs’ oversimplified by amalgamating highly disparate ideas related to envy, witchcraft, and agency. While simplification and generalization can serve a purpose, in this case they felt that it (a) weakened the credibility of the assumption that such beliefs are inherently incorrect; (b) stretched thin the ability to measure such beliefs cleanly, undercutting some of the empirical results; and (c) attenuated the links from the evolutionary model to the data to real-world implications.

Please do read the reports in full, including E1’s challenges to the work of anthropologist George Fisher that helped lay the foundations for and inspired this work, as well as the role of colonialism in moving beyond zero-sum thinking. Meanwhile E2 has some concrete suggestions in terms of the World Values Survey data and the formal analysis.

Further notes27

We asked Claude.ai to highlight points of agreement and disagreement between the two evaluations, enumerated below.

Areas of Agreement:

1. Both evaluations question the assumption in the paper that demotivating beliefs are inherently incorrect. They argue these beliefs may actually reflect the true state of the world in many contexts with corruption, instability, or unfairness.

2. Both point out issues with how concepts like witchcraft and envy are measured in the empirical analysis. For example, equating non-Christian beliefs with witchcraft is seen as problematic.

3. Both suggest the correlations shown could have alternative explanations, such as demotivating beliefs being an accurate reflection of zero-sum environments, rather than validating the evolutionary model proposed.

4. Both critiques argue the paper oversimplifies by lumping together very different concepts like witchcraft, envy and views on poverty under the umbrella of "demotivating beliefs."

Areas of Disagreement:

1. Evaluation 1 has a much more critical tone overall, arguing the paper fails conceptually, theoretically and empirically. Evaluation 2, while noting significant issues, has a more measured critique.

2. Evaluation 1 directly challenges the foundation of the paper by extensively critiquing the original ideas of anthropologist George Foster that inspired it. Evaluation 2 focuses its critique on the assumptions and analysis in the current paper.

3. Evaluation 1 takes great issue with the paper's discussion of colonialism enabling Europe to grow beyond zero-sum thinking. […]

4. Evaluation 2 makes several specific suggestions for improving the analysis, such as using a hierarchical model and providing more details on the WVS data. Evaluation 1 does not see a path to sufficiently improve the paper.

In summary, while both evaluations identify some similar core conceptual and empirical issues in the paper, Evaluation 1 is harsher in its critique and sees the flaws as insurmountable, while Evaluation 2, though still critical, provides more constructive feedback and seems to see potential for improvement.

Author engagement

As always, we informed the authors in advance that we would be commissioning their paper for evaluation. We shared these evaluations with them and invited their response. We have not heard from them. If they choose to respond, we will integrate the response here.