Wherein we learn, completely to our surprise, that attempts to be helpful are appreciated …
Such is the generosity of the greater part of young men, that, so far from being disposed to neglect or despise the instructions of their master, provided he shows some serious intention of being of use to them, they are generally inclined to pardon a great deal of incorrectness in the performance of his duty, …
Adam Smith, on students
This article just got published in one of the AEA offsprings called American Economic Journal: Economic Policy.
The authors report the results of a large-scale pre-registered (see fn 9) intervention during the spring of 2016 and the fall of 2017 at a broad-access, four-year institution in Northern California. At that institution the completion rates for Latinx and Black students are reported to be particularly low (p. 123).
“We randomly chose 30 large undergraduate courses (serving over 120 students) in each respective term and identified the instructor of record. … we recruited these professors by sending personalized letters signed by both the provost and dean of undergraduate students. In total, 22 faculty members across 20 different course subjects participated in the study, with nearly 3,000 total students in the treatment and control groups. All participating faculty were given templates of emails that they were encouraged to personalize to their own courses. … all emails, had to meet three basic criteria: (1) they had to be personalized to the student; (2) they had to acknowledge a student’s performance in the course so far; and (3) they had to provide feedback about what students could do to improve grade performance and/or seek additional help. Participating professors received a $500 payment to be part of the experiment, with an additional $100 gift card for completing a survey at the end of the semester.” (pp. 123–4)
It is not clear how many of the participating faculty were casuals (presumably a large percentage) but given that the recruitment letters were sent by the provost and the dean of undergraduate students, it would be naive to assume that there were not massive experimenter demand effects here. The fact that almost half of the faculty discontinued (reportedly!) their personalized attention after the experiment (p. 136) is an indication. (Given what I know about social-desirability effects, I conjecture that the percentage is in fact much higher.)
Towards the very end, the authors argue that the cost of the estimated 24 additional college graduates their intervention produced in the treatment group was approximately $2,500 per graduate, and hence pass their benefit-cost test. The costs are computed from the $$$ spent on incentive payments for participating faculty and the almost $50,000 dollars spent on research assistants. (See fn 29) They do not take into account the opportunity costs on the part of participating faculty of writing those individualized messages. We don’t know what they are.
Overall the effect of the intervention seems, in the grand scheme of things, minor and hardly surprising. Cue the opening Adam Smith quote.
Consider making your opinion known by applauding, or commenting, or following me here, or on Facebook or on LinkedIn.
