Skip to main content

Evaluation Summary and Metrics: “The Environmental Effects of Economic Production: Evidence from Ecological Observations”

Evaluation Summary and Metrics: “The Environmental Effects of Economic Production: Evidence from Ecological Observations”

Published onJul 17, 2023
Evaluation Summary and Metrics: “The Environmental Effects of Economic Production: Evidence from Ecological Observations”
·

Preamble

Paper: “The Environmental Effects of Economic Production: Evidence from Ecological Observations” (2021).

Authors: Liang, Y., Rudik, I., & Zou, E.

This paper was selected as part of our (NBER) direct evaluation track.

We organized two evaluations of this paper (1. Elias Cisneros, 2. Anonymous). The authors also responded. To read the evaluations and the response, click the links at the bottom.

Evaluation manager’s notes

Why we chose this paper

This NBER paper uses an extensive, longitudinal dataset comprising ecological sampling information from thousands of locations across the United States, and using a range of correlational and quasi-experimental methods, it generates robust evidence that economic productivity has a negative impact on biodiversity. Furthermore, by examining the contribution of two potential mechanisms through which these impacts are realised - namely air pollution and land-use change - the study provides the start of an explicit diagnosis, and most importantly, a potential way forward for mitigating the negative impacts of economic activity on biodiversity. This is a very important discussion that must remain central to global environmental and development policy discussions.

Evaluators were asked to follow the general guidelines available here. In addition to written evaluations (similar to journal peer review), we ask evaluators to provide quantitative metrics on several aspects of each article. These are put together below.

The paper can be found here.

How we chose the evaluators

For this paper we organised two evaluations. Both evaluators are environmental economics, with strong econometric skills. We chose them explicitly so they could comment on the econometrics, as well as on the overall research question.

Evaluators were asked to follow the general guidelines available here. In addition to written evaluations (similar to journal peer review), we ask evaluators to provide quantitative metrics on several aspects of each article. These are put together below. For this paper we did not give specific suggestions on ‘which aspects to evaluate’.

Summary of evaluations

The evaluations are very positive overall, and both evaluators agree that is highly relevant to Global Priorities. They also agree that the authors address the limitations of the datasets rigorously, and that the analyses are well-considered and robust. Some modest restructuring of the paper is proposed to increase readability. In addition, the evaluators propose some additional analyses that could potentially bolster some of the findings - specifically, they propose that the authors might examine the heterogeneity of the pollution regulation-biodiversity association; to potentially also assess impacts of productivity on bird taxa; and additional quasi-experimental analyses are suggested that could further the analysis of both the mechanisms (land-use policies, air pollution) explored in this study.

Metrics (all evaluators)

Ratings

Eval. 1 (of 2)

Elias Cisneros

Eval. 2: Anonymous

Category

Rating (0-100)

Confidence:
High = 5, Low = 0

Rating (0-100)

Confidence:
High = 5, Low = 0

Overall assessment

88

5

70

3

Advancing knowledge and practice

90

4

70

3

Methods: Justification, reasonableness, validity, robustness

75

3

70

3

Logic & communication

80

4

75

4

Open, collaborative, replicable

90

5

65

2

Engaging with real-world, impact quantification; practice, realism, and relevance

95

5

60

3

Relevance to global priorities

95

4

80

4

Predictions

Eval. 1 (of 3): Elias Cisneros

Evaluator 2: Anonymous

Prediction metric

Rating (0-5) (low to high)

Confidence (0-5)
High = 5, Low = 0

Rating (0-5)

Confidence (0-5)
High = 5, Low = 0

What ‘quality journal’ do you expect this work will be published in?

Note: 0= lowest/none, 5= highest/best

4

5

4

3

On a ‘scale of journals’, what ‘quality of journal’ should this be published in?

Note: 0= lowest/none, 5= highest/best

4

5

4

5

Comments
0
comment
No comments here
Why not start the discussion?