Abstract

We organized two evaluations of the paper: "Existential Risk and Growth" [1]. The evaluators see the paper as a substantively important model informing a critical question, but have concerns about the degree to which the assumptions and model structure are sufficient for actually answering key questions. To read these evaluations, please see the links below.

Evaluations

1. Seth G. Benzell

2. Ioannis Bournakis

Overall ratings

We asked evaluators to provide overall assessments as well as ratings for a range of specific criteria.

I. Overall assessment: We asked them to rank this paper “heuristically” as a percentile “relative to all serious research in the same area that you have encountered in the last three years.” We requested they “consider all aspects of quality, credibility, importance to knowledge production, and importance to practice.”

II. Journal rank tier, normative rating (0-5):1 On a ‘scale of journals’, what ‘quality of journal’ should this be published in? (See ranking tiers discussed here.) Note: 0= lowest/none, 5= highest/best.

See “Metrics” below for a more detailed breakdown of the evaluators’ ratings across several categories. To see these ratings in the context of all Unjournal ratings, with some analysis, see our data presentation here.2

See here for the current full evaluator guidelines, including further explanation of the requested ratings.3

Evaluation summaries

Seth G. Benzell

Authors find: (1) existential risk follows an inverted-u shape in both technology level and over time (2), technological accelerations reduce cumulative existential danger. These findings are novel and important, implying a “time of perils” after which safety is (asymptotically) assured. Two critiques: (1) the representative agent of the model is neither a normative goal nor a positive prediction, making implications nebulous (2) omission of savings from the model eliminates a potential mechanism for intertemporal consumption smoothing, which may impact demand for safety.

Ioannis Bournakis

The paper examines how hazard rates change with varying speeds of technological advancement, questioning the wisdom of slowing progress. It finds a trade-off between technological progress and safety as countries prosper under optimal policies, illustrating a Kuznets curve where hazard rates decrease post-technological advancement. Despite its critical inquiry into whether technology jeopardizes society, it overlooks integrating a benefit function with hazard assessment. This omission limits the ability to assess technology's overall impact. With adjustments, it merits publication in a 3-star journal, likely achieving acceptance after revisions.

Metrics

Ratings

See here for details on the categories below, and the guidance given to evaluators.

Journal ranking tiers

See here for more details on these tiers.

Evaluation manager’s discussion (David Manheim)

The evaluators see the paper as a substantively important model informing a critical question, but have concerns about the degree to which the assumptions and model structure are sufficient for actually answering key questions. This is in large part a drawback of this class of approach, rather than the paper itself, and it seems useful as a base for further research on the questions of how to consider the “time of perils” hypothesis.

As the evaluations note, their criticisms do not imply that the paper itself is not good or useful; the same applies to the below comments on the generally excellent and helpful evaluations.

Regarding the first evaluation, I think the question of savings is potentially important, and could potentially be addressed, at least at a high level. On the other hand, the questions about whether the model describes the actual dynamics of decision-making in a race scenario are certainly valid, but the paper is still useful. For example, the model in the paper is helpful in describing what policymakers who respect social preference should be aiming for as an optimal outcome, which can inform what we should normatively aim for. As a secondary point, it seems unhelpful to posit that if society had very different social preferences, the result would not apply - this paper is a case where the assumptions generally match reality, so they are helpfully simplifying.

Regarding the second evaluation, balancing risk and reward is certainly critical, but it seems unclear how this would affect the results, given the almost axiomatic assumptions that AI would be beneficial if not harmful. The other criticisms seem correct, but not substantively useful, considering that the paper is not attempting to provide a way to gather empirical evidence, or address the understood shortcoming of using exogenous growth.

Why we chose this paper

From our prioritization notes: This paper addresses a very important current policy discussion related to how to respond to technological advances when they pose large-scale risks with a simple conceptual model. It seems like a possible critical crux between different approaches to technological advances, e.g. progress studies/accelerationism versus safety, risk mitigation, and much of actual tech policy. Given that the arguments are somewhat technical, and the usefulness of the conceptual approach is unclear, it seems likely to benefit from rigorous review of the technical arguments and assumptions, and thought about whether and how the model should inform policy considerations.

Evaluation process

The authors engaged strongly with this process. Phil Trammell kept us informed about the updates the authors were making, and let us know as soon as these were released. We appreciate the authors’ engagement with our process, including their substantive response to the evaluators’ comments.