Evaluation of "Intergenerational Child Mortality Impacts of Deworming: Experimental Evidence from Two Decades of the Kenya Life Panel Survey" for The Unjournal.
This is a highly credible and important analysis of the intergenerational effects of a Kenyan deworming intervention. Such effects are important to document because they could meaningfully affect the cost-benefit analysis of such programs and also teach us about the mechanisms through which parents transmit advantage to their children. However, the analysis would be more credible if the authors were transparent about their deviations from the pre-analysis plan, which include the addition of (and exclusion of) outcomes and changes to the empirical specification. Also, more information on the “first stage”—the impact of randomization on receipt of deworming treatment—would be helpful.
We asked evaluators to give some overall assessments, in addition to ratings across a range of criteria. See the evaluation summary “metrics” for a more detailed breakdown of this. See these ratings in the context of all Unjournal ratings, with some analysis, in our data presentation here.1
Rating | 90% Credible Interval | |
Overall assessment | 90/100 | 80 - 100 |
Journal rank tier, normative rating | 4.5/5 | 4.0 - 5.0 |
Overall assessment (See footnote2)
Journal rank tier, normative rating (0-5): On a ‘scale of journals’, what ‘quality of journal’ should this be published in?3 Note: 0= lowest/none, 5= highest/best.
See here for the full evaluator guidelines, including further explanation of the requested ratings.
The primary claim is that exposure to additional years of deworming treatment in primary school results in lower child mortality of the children of those exposed. The evidence provided is from analysis of an intervention randomized at the school level that compares the mortality of the children of those exposed earlier vs later via the intervention. The outcome data is derived from long-term follow up survey data of these experiment participants. I discuss the importance of this claim in detail in my report.
I believe with fairly high certainty the general claim that mortality of the children of those randomly exposed to the intervention fell and believe it likely falls within the confidence intervals implied by the standard errors reported in the paper. However, some uncertainty may remain due to the fact that not all sampled participants completed the follow-up surveys.
This manuscript examines the intergenerational consequences of a public health deworming intervention conducted at the school level at 75 schools in Kenya between 1998 and 2001 called the Primary School Deworming Project (PSDP). The PSDP gave free de-worming drugs to an essentially randomly-selected 50 of the 75 schools earlier (in 1998 and 1999), with the remaining 25 receiving them later (in 2001). As a result, the 50 early schools received more years of the deworming intervention than the later treated schools.
The authors take advantage of this intervention to investigate whether the beneficial effects of de-worming for the parents’ generation result in better health for their children. To measure these intergenerational effects, the authors use high-quality panel data collected via the Kenya Life Panel Survey (KLPS). The primary outcomes they consider are infant mortality and under age 5 mortality. The analysis shows that the children of parents induced to receive more years of deworming medication had children with significantly lower mortality rates. Lower mortality rates were observed both for the children of female parents and male parents. This result is surprising as we might hypothesize different mechanisms through which a mother versus a father might confer intergenerational health advantages.
The analysis then explores potential mechanisms for this reduced infant mortality, including measures of parental health, education, living situation, fertility, and health care, and finds some evidence that all of these may have moved, suggesting that the portfolio of improved outcomes resulting from the deworming program may be responsible for the improved child mortality outcomes observed.
This is a terrific paper documenting important results. The results are highly credible given the randomized design. The two most important implications are: first, that these de-worming interventions could have higher returns than previously believed, because the benefits extend across generations. Second, it is well known that higher socio-economic status parents tend to confer this advantage to their children. It is also well-documented that early life health is predictive of adult health, educational attainment, and earnings. So, these results suggest that health may be an important channel driving the intergenerational transmission of advantage.
I had one major comment regarding the paper and a few minor points, which I make below.
Deviations from pre-specified analysis. High-quality experiments typically publicly pre-specify their analysis plans in order to avoid the appearance of searching across specifications and outcomes for significant effects and selectively reporting those estimates. However, over time, researchers may change their opinion about the best way to approach a topic or the appropriate emphasis to place on certain estimates versus others. Researchers are, naturally, free to deviate from or amend pre-specified analysis plans over the course of their investigation. However, it is appropriate, when doing so, to make note of meaningful deviations somewhere in the manuscript, ideally with an explanation for why these deviations occurred.
This paper cites AEA-registered plan AEARCTR-0001191 as its pre-analysis plan (PAP). I read these analysis plan documents and noted several meaningful differences between this paper and the pre-specified analysis. The most meaningful deviations I noticed were:
Second generation child health outcomes were pre-registered as “secondary” outcomes related to a primary analysis of participants’ fertility rather than focal outcomes themselves. Only under 5 mortality, and not infant mortality, was included as a possible outcome in the PAP. The first child health outcome listed in the PAP is the “child health index,” described as “mean-effects index based on the respondent-reported subjective health of their child, and a reverse-coded sum of indicators for common health symptoms children can experience.” But, this child health index does not appear in the paper.
The PAP proposes a secondary hypothesis test that uses an F test to examine the joint significance of the original de-worming treatment and the cost-sharing treatment but I did not see this in the paper. In addition, the PAP states that the authors will report multiple hypothesis adjusted p-values, but the manuscript does not contain them.
The authors do not present formal statistics on the “first stage”--i.e., how much more likely were participants who attended a group 1 or group 2 school to receive any de-worming medication compared to those who attended a group 3 school, and how many more years of de-worming medication did they actually receive. The paper states that group 1 and group 2 schools received “on average” 2.41 additional years of exposure to the PSDP, but take-up was about 75%, with some amount of treatment reaching the control group. So, we might think […] the number of additional medication years was lower, perhaps closer to 1.7 or 1.8 years. It’s also likely the difference in the likelihood of receiving any de-worming treatment during these adolescent years is small between the two groups. More formal analysis on this would be helpful in interpreting the magnitude of the reduced form differences in child health outcomes across the treatment and control groups.
Table A1, documenting the low attrition rates of the KLPS survey waves, is very impressive. Very few studies are able to achieve such a high follow-up rate over such a long period. One additional helpful piece of information would be to show differences in baseline characteristics across respondents and non-respondents by treatment arm.
In a footnote, the manuscript mentions that marriage quality may be a mechanism, and is an area for future research. Given that the results show that both mother’s and father’s health is equally important in generating intergenerational health benefits, this seems like a relevant outcome to consider for this paper. E.g., if fathers exposed to the deworming program matched with healthier mothers, it is possible that the (untreated) mother’s health is the mechanism responsible for the improvement infant and child mortality outcomes.
How long have you been in this field?
10+ years
How many proposals and papers have you evaluated?
100+ (I have not tracked)
Evaluation of "Intergenerational Child Mortality Impacts of Deworming: Experimental Evidence from Two Decades of the Kenya Life Panel Survey" for The Unjournal.
Creative Commons Attribution 4.0 International License (CC-BY 4.0)
