Bergman et al. (2023) study the impact of algorithmic tracking on course placement for students in seven community colleges in New York. They develop placement algorithms based on historical data on student performance, predicting students’ likelihood of passing college-level math and English courses. The paper evaluates these algorithms against the results when course placement is based solely through testing using an experimental design where students are randomly assigned to both placement systems and examining the resulting placements and pass rates.
Pays for Itself
The paper considers two components of cost for the algorithmic tracking program: the direct costs of implementing and operating the algorithmic placement system and the indirect costs associated with changes in the amount and types of credits attempted by the students. To estimate the total costs of the algorithmic tracking program, the paper estimates the cost for the control group (assigned to courses based on testing) and costs for the treatment group (assigned to courses using the algorithm) and calculates the total cost of the algorithmic tracking program relative to the traditional placement method.
The cost of implementing and operating the algorithmic placement system for five years is $958,810, compared to $174,240 for the test score placement system. Given a sample of 5,808 students, this implies a incremental direct cost per student of $140.
Using IPEDS information on the cost per credit for college-level and remedial courses, the paper calculates indirect costs per student of $5,040 for the algorithmic placement system, versus $5,420 for the test score placement system. This implies an indirect cost reduction of $380 per student given the estimated reduction in remedial credits and increase in attempted college-level credits. The cohort-weighted average for tuition and fees is 39% of total expenditures per credit, with the remaining 61% of costs per credit borne by the government (Barnett et al. 2020). This results in an indirect cost reduction of $230 per student for the government.
Adding together direct and indirect costs, the algorithmic placement system costs in total $90 less per student compared with the test score placement system.
As a result of the algorithmic placement system, students attempt 0.737 fewer net credits: 1.095 fewer remedial credits and 0.358 more college credits in math and English. Using IPEDS data for the sample colleges, the overall cost per college-level and remedial credit is approximately $520 (Barnett et al. 2020). However, students do not pay all of the costs associated with each attempted credit. Using IPEDS data again, the cohort-weighted average for tuition and fees is 39% of total expenditures per credit (Barnett et al. 2020). As a result, students save $150 (0.737 x $520 x 0.39) under algorithmic tracking relative to the test score placement system.
Bergman et al. (2023) note that, by assuming there is no effect on utility from attempting college-level courses or avoiding remedial courses, this is a conservative estimate of a student’s willingness to pay for algorithmic placement.
Bergman et al. (2023) conclude that, given negative net costs and a positive willingness to pay, the MVPF of algorithmic tracking for community college course placement is infinite.
Bergman, Peter, Elizabeth Kopko, and Julio E. Rodriguez (2023). “A Seven-College Experiment Using Algorithms to Track Students: Impacts and Implications for Equity and Fairness.” NBER Working Paper 28948. http://www.nber.org/papers/w28948.
Barnett, Elisabeth A., Elizabeth Kopko, Dan Cullinan, and Clive Belfield (2020). “Who Should Take College-Level Courses? Impact Findings from an Evaluation of a Multiple Measures Assessment Strategy.” Center for the Analysis of Postsecondary Readiness. https://ccrc.tc.columbia.edu/publications/multiple-measures-assessment-impact-findings.html