Abstract

This is an evaluation of "Universal Basic Income: Short-Term Results from a Long-Term Experiment in Kenya".[1]

Summary Measures

We asked evaluators to give some overall assessments, in addition to ratings across a range of criteria. See the evaluation summary “metrics” for a more detailed breakdown of this. See these ratings in the context of all Unjournal ratings, with some analysis, in our data presentation here.1

Overall assessment: We asked evaluators to rank this paper “heuristically” as a percentile “relative to all serious research in the same area that you have encountered in the last three years.” We requested they “consider all aspects of quality, credibility, importance to knowledge production, and importance to practice.”

Journal rank tier, normative rating (0-5): “On a ‘scale of journals’, what ‘quality of journal’ should this be published in? (See ranking tiers discussed here)” Note: 0= lowest/none, 5= highest/best”.

See here for the full evaluator guidelines, including further explanation of the requested ratings.

Written report

This is an extremely impressive (and costly) RCT that will likely be of use to policymakers when designing the structure of benefits.

This study uses a RCT to evaluate three different timings of the same amount of unconditional cash transfer. In one arm, the full amount was transferred as a lump sum; in another, the amount was transferred as a UBI over two years, and in the third, the same amount was transferred over a two-year period, but with the promise of ten years of additional (similarly valuable) transfers. Each participant received $3,000 in transfers over the study period (with the long-term arm expecting to receive an additional $15,000 over the duration of the study). The study seeks to evaluate how the timing of transfers affects the results of the transfers - for instance, if the promise of a UBI makes recipients less likely to work because they can rely on future transfers, or if a lump sum allows for costly upfront investments in productive assets.

The size of the RCT (and cost of the entire program) bears some emphasis; there are some 23,000 treated adults (pg. 11); the cost of this RCT was somewhere in the neighborhood of $70 million USD. As such, it likely seeks to inform governments about policymaking, since it is more similar to scale to a government program than a standard economics RCT. If a government seeks to do UCTs - as indeed, many governments did during the pandemic, and continue to explore beyond the pandemic - it is important to understand how the structure of transfers will impact their results. It will also inform GiveDirectly, a NGO that provides UCTs as it continues to scale up its giving. (GiveDirectly moved approx. $30 million a year before the pandemic; they now move closer to $200 million a year, and continue to refine their model to best support those in extreme poverty.)

One can also imagine it being influential even when considering targeted benefits. Cash benefits can be structured as a one-time transfer, or can pay out over time; since this study shows that people in extreme poverty had larger income gains from receiving a single large transfer, one could imagine governments shifting towards one-time payments rather paying out a benefit over time.

The RCT itself is well done. The analysis is clear; there is a pre-analysis plan that is linked to in the text. The design itself makes a great deal of sense; these are important questions and it seems useful to have a well-powered RCT seek to answer them.

Since every adult in a village receives a transfer, one can examine general equilibrium effects; for instance, the authors seek to answer if prices are higher in villages where everyone is somewhat better off than in control villages. (They do not find that they are, though they note that they are simply not rejecting the null; this is not a precise zero.)

There are some familiar results here, likely expected to any reader familiar with the cash transfer literature. There is no evidence that people work less if provided with a UBI (though it is not clear to me that this would be true in a case where the UBI was large enough to cover all basic needs), nor do people particularly use UCTs for temptation goods.

There are also less familiar results. They find that the economic effects of the lump sum transfer are similar in size (and for some outcome variables, larger) to that of the long-term UBI - even though the lump sum transfer recipients have received all that they will have, the long-term UBI recipients have the security of 10 more years of transfers. This, to [me], suggests that recipients are extremely credit-constrained - and that this is not alleviated by the UBI. Even when they receive a UBI, this does not allow them to make the lumpy investments needed for long-term returns.

Given the size of the study, and the stature of the authors, I do not feel I need (or indeed, am qualified to give) advice to the authors on how they might place in a top economics journal. I suspect the paper will publish well; fewer other RCTs on UCTs have this sample size and are able to truly answer questions about implementation.

If I were to quibble with this study, it would be on the presentation of results. There are many outcome variables here, and three different arms; I found it difficult to follow which results belonged to which arm. It was difficult to keep them all straight. I would have much appreciated a summary table comparing the size of results (and also noting if it was possible to reject the hypothesis that the effect sizes were the same). Ideally, such a table would group types of outcome (e.g. consumption, assets, “softer” measures) and perhaps even be color-coded with relative effect size.

Similarly, the introduction brings up a policy-relevant question that the paper does not address. On page one of the paper, the authors mention that “one could ask whether the transfer raises the incomes of the recipients by at least as much as the amount of the transfer”. While it is possible to calculate if this is true from the results in the paper (e.g. from income and asset changes), this summary is not presented in the paper. I think it would be particularly helpful for policymakers to directly compare the (income + asset) changes to the cost of the UCT. Policymakers - especially in developing countries - operate under significant constraints; this would likely help them better evaluate (expensive) UCT programs in comparison to other programs.

Evaluator details

1. How long have you been in this field?

   • five years

2. How many proposals and papers have you evaluated?

   • This is my first referee report, but I regularly make project funding decisions for my day job.