Abstract

We organized two evaluations of the paper: “Biodiversity Risk”[1]. To read these evaluations, please see the links below.

Evaluations

1. Anonymous evaluation 1

2. Anonymous evaluation 2

Overall ratings

We asked evaluators to provide overall assessments as well as ratings for a range of specific criteria.

I. Overall assessment: We asked them to rank this paper “heuristically” as a percentile “relative to all serious research in the same area that you have encountered in the last three years.” We requested they “consider all aspects of quality, credibility, importance to knowledge production, and importance to practice.”

II. Journal rank tier, normative rating (0-5):1 On a ‘scale of journals’, what ‘quality of journal’ should this be published in? (See ranking tiers discussed here.) Note: 0= lowest/none, 5= highest/best.

See “Metrics” below for a more detailed breakdown of the evaluators’ ratings across several categories. To see these ratings in the context of all Unjournal ratings, with some analysis, see our data presentation here.2

See here for the current full evaluator guidelines, including further explanation of the requested ratings.3

Evaluation summaries

Anonymous evaluation 1

The paper evaluates whether biodiversity risk is priced in asset markets and breaks new ground in doing so. It concludes that biodiversity risk is priced and provides open-source metrics for measuring biodiversity risk based on media reporting and firm-level biodiversity risk exposure based on 10K reports. The paper's strength is that it uses primary data that is well-documented and offers it for further research. One important weakness is that both the statistical and economic significance of the result are not sufficiently discussed.

Anonymous evaluation 24

This paper seeks to measure biodiversity risk. It does so in several ways:

• Via a news based measure 

• textual analysis of corporate disclosures 

• a survey of finance professionals, regulators and academics 

The authors then apply their measures in an asset pricing and portfolio context and find that markets are pricing biodiversity risk to some degree but their survey respondents believe that market pricing of biodiversity risk is incomplete. 

Metrics

Ratings

See here for details on the categories below, and the guidance given to evaluators.

Journal ranking tiers

See here for more details on these tiers.

Evaluation manager’s discussion

“Biodiversity Risk” clearly adds to our understanding of the impact of Environment Social and Governance concerns on financial returns, with a focus on the emerging area of risks owing to changes in biodiversity. Moreover, it offers a novel measure of perceived biodiversity risk which can be used by future studies, perhaps in some form of shift-share (/Bartik) instrument approach.

While this paper offers novel insight, there are a few areas in which it could be improved, or more explicitly recognise its shortcomings. As both evaluations highlight, it is somewhat ambiguous as to what the paper is identifying as biodiversity risk. First, both evaluations highlight that the measure of biodiversity risk is perceptions, rather than on-the-ground changes in biodiversity risk. Indeed, large changes in actual biodiversity risk - at least physical risk - in such timeframes seem somewhat unlikely. Second, both evaluations (and particularly the second) mention the concern that measured biodiversity risk may well be picking up climate risk, and that this could be more comprehensively explored in the paper. Third, evaluation 2 highlights that nuance could be added to the discussion by disentangling physical and transition risks.

Personally, I wonder to what extent the lack of nuance in defining biodiversity risk per se within the paper is in part limited by the dictionary approach that is currently being used, which I think could well result in overly broad definitions of what constitutes biodiversity risk. If this overly broad classification is orthogonal to other factors, then the noise introduced would “only” result in estimates biased towards zero. However, there may be confounding effects introduced if those biodiversity-related terms are being used in different contexts at systematically different rates (i.e. disproportionately by a particular subset of firms or sectors).

Finally, both evaluations suggest to me that the policy implications of this research could be better explored. For instance, what does this research suggest would be required within the financial sector in order to see better pricing of biodiversity risks? Is that even possible within financial sector regulation, or is regulation on the actual firms and their damages required? Indeed, it seems that more could be discussed within the paper about the externality regarding which firms/sectors are exposed to biodiversity risk, versus which are responsible for damages to biodiversity and increasing biodiversity risks. Similarly more could be said about the different timeframes over which damages to, and the negative impacts of, biodiversity risk are likely to occur, and the potential implications of this.

Unjournal Process

How we chose the evaluators

We looked for evaluators with:

1. Expertise in text analysis.

2. Expertise in measuring company performance indicators.