Evaluation Summary of The Long-Run Effects of Psychotherapy on Depression, Beliefs, and Economic Outcomes
We organized two evaluations of the paper: “The Long-Run Effects of Psychotherapy on Depression, Beliefs, and Economic Outcomes”. To read these evaluations, please see the links below.
We asked evaluators to provide overall assessments as well as ratings for a range of specific criteria.
I. Overall assessment: We asked them to rank this paper “heuristically” as a percentile “relative to all serious research in the same area that you have encountered in the last three years.” We requested they “consider all aspects of quality, credibility, importance to knowledge production, and importance to practice.”
II. Journal rank tier, normative rating (0-5):1 On a ‘scale of journals’, what ‘quality of journal’ should this be published in? (See ranking tiers discussed here.) Note: 0= lowest/none, 5= highest/best.
| Overall assessment (0-100) | Journal rank tier, normative rating (0-5) |
Anon. Evaluator 1 | 77 | 4.0 |
Anon. Evaluator 2 | 92 | 4.7 |
See “Metrics” below for a more detailed breakdown of the evaluators’ ratings across several categories. To see these ratings in the context of all Unjournal ratings, with some analysis, see our data presentation here.2
See here for the current full evaluator guidelines, including further explanation of the requested ratings.3
This paper’s most important contribution is the evidence for sustained positive impacts on mental health after 5 years (one of the longest follow-ups in the field), after short, lower-cost (community based) therapy. As critical comments, I don't believe the 'depressive realism' task was well selected, I see opportunities to better contextualize the effects for the reader as well as opportunities to improve figures and make stronger policy recommendations.
This is a high quality paper that extensively studies the effects of therapy. I make a few small suggestions for analyses to explore the economic effects and the perceived efficacy of therapy.
See here for details on the categories below, and the guidance given to evaluators.
Evaluator 1 Anonymous | Evaluator 2 Anonymous | ||||
|---|---|---|---|---|---|
Rating category | Rating (0-100) | 90% CI (0-100)* | Comments | Rating (0-100) | 90% CI (0-100)* |
Overall assessment4 | 77 | (70, 84) | 92 | (81, 100) | |
Advancing knowledge and practice5 | 83 | (76, 90) | 6 | 90 | (80, 100) |
Methods: Justification, reasonableness, validity, robustness7 | 75 | (68, 82) | 8 | 90 | (83, 97) |
Logic & communication9 | 77 | (74, 80) | 79 | (69, 89) | |
Open, collaborative, replicable10 | 35 | (20, 50) | 11 | 88 | (82, 95) |
Real-world relevance 12 | 83 | (76, 90) | 13 | 86 | (79, 93) |
Relevance to global priorities14 | 85 | (80, 90) | 92 | (82, 100) |
See here for more details on these tiers.
Evaluator 1 Anonymous | Evaluator 2 Anonymous | |||
|---|---|---|---|---|
Judgment | Ranking tier (0-5) | 90% CI | Ranking tier (0-5) | 90% CI |
On a ‘scale of journals’, what ‘quality of journal’ should this be published in? | 4.0 | (3.5, 4.5) | 4.7 | (4.4, 5.0) |
What ‘quality journal’ do you expect this work will be published in? | 4.0 | (3.5, 4.5) | 4.6 | (4.1, 5.0) |
See here for more details on these tiers. | We summarize these as:
| |||
The paper was brought to our attention by a suggestion from the Happier Lives Institute.
One of the evaluators was a member of our Field Specialist team, but they were not part of the decision to prioritize this paper.
Evaluation of The Long-Run Effects of Psychotherapy on Depression, Beliefs, and Economic Outcomes
Creative Commons Attribution 4.0 International License (CC-BY 4.0)
