Abstract

Overall, I find this paper competently executed. I like the inductive approach to theorizing after the findings, the multiple tests for mechanism, extensive robustness checks, and the use of transaction-level data to check what participants used the money for. The paper has external validity limitations, and can be framed better since the unconditional cash transfers were given during the Covid pandemic. [I suggest] several further tests to test the main mechanism, and improve the mediation analysis.

Summary Measures

We asked evaluators to give some overall assessments, in addition to ratings across a range of criteria. See the evaluation summary “metrics” for a more detailed breakdown of this. See these ratings in the context of all Unjournal ratings, with some analysis, in our data presentation here.1

Overall assessment (See footnote2)

Journal rank tier, normative rating (0-5): On a ‘scale of journals’, what ‘quality of journal’ should this be published in?3 Note: 0= lowest/none, 5= highest/best.

Written report4

Overall, I find this paper competently executed. I like the inductive approach to theorizing after the findings, the multiple tests for mechanism, extensive robustness checks, and the use of transaction-level data to check what participants used the money for. Overall, I have several suggestions for improving the paper:

1. The paper is framed, and the contribution posed mainly about unconditional cash transfers (UCT). However, the experiment was conducted during the COVID-19 pandemic. Therefore, the paper is also about UCT during a pandemic. The authors are probably the first ones to study this, but this contribution should be made upfront. The authors, therefore, should relate to further literature on cash transfers during the pandemic.
   

2. Related to the above, the paper has limited external validity given the end of the pandemic/conducting the experiment under very particular circumstances. This is, of course, beyond the control of the authors, so an alternative approach is to recognize upfront the context/circumstances under which the experiment was conducted and not wait until the conclusion.

3. As the paper speculates, cash treatments might have been too small to reach any positive effect. The problem is that [the potential for] testing higher-value transfers is limited, given the costs. UCTs may be more viable in developing contexts, where the values can be larger compared to annual income, as the authors suggests.

4. The main mechanism behind the potential negative effects of the treatment suggests that the UCT can remind receipts of the debts that they can't cover. Beyond the model, this could be explored further empirically; according to the stock of debt pre-treatment, do people with higher debt stock have more negative effects? Some heterogeneous effects with levels of debt could be done.
   

5. The auxiliary 15 participants' interviews could be improved by improving the sample size and ideally having representative groups in treatment and control groups. If the size of the interviewees can't be scaled up, given that significant time has passed since the experiment, at least the authors should do their analysis separating interviews from the treatment and control group.

6. The mediation analysis can be improved in two ways:

   • The mediators can be further disaggregated; the variables, Money Mind – Cost, Overwhelmed Others' Needs, and Spending Stress can be used in Table H.11 separately to see if main treatment effects change, under using only one of those.

   • Most importantly, the mediation analysis can improve beyond just controlling for mediators and checking how the mani effects change. For example, see the proposal by Acharya, Blackwell, and Sen (2016)[1].
     

7. Did the authors try exploring het[erogeneous] effects according to the previous level of income? The treatment effects might depend on where the participants were in the income distribution/how big the cash transfer is with respect to their income.

References:

[1]Acharya, Blackwell, and Sen (2016), American Political Science Review , Volume 110 , Issue 3 , August 2016 , pp. 512 - 529

Evaluator details

1. How long have you been in this field?

   • 5 years

2. How many proposals and papers have you evaluated?

   • Over 20