Higher education institution rankings have enjoyed an enormous rise in popularity over the past decades, and are considered a key source for evaluation by the public. However, there is mounting evidence that these rankings are methodologically flawed and yield little insight about the quality of an institution to inform prospective students or their parents.
Some argue that rankings magnify economic inequality, as they rest predominantly on selectivity without gauging student effort to succeed. Criticism stems in part from year-to-year changes in ranking methodology, giving rise to efforts to track such changes. Current rankings engender varied limitations, key among which is reliance on input factors that describe an institution, most prominently the institution's admission selectivity, preparation level of enrolled students, faculty resources/research, campus diversity, and perceived reputation, to name a few, without centering the ranking on student outcomes after attending an institution.
Current approach to college rankings
While some rankings use student outcome measures (e.g. student graduation rate), none of the popular rankings employ a methodology that gauges an institution's value-added contribution to their students' post-attendance success.
The Wall Street Journal/Times Higher Education (WSJ/THE) Ranking, relying on the largest number of weighted ranking factors, determines 60 percent of an institution's rank for its global ranking on the basis of faculty research and publication citations alone, without consideration for post-graduate student outcomes. And while its separate US College Ranking determines 19 percent of an institution's rank on the basis of students' expected post-graduate income and student debt repayment, it allocates 20 percent of the total rank to validity-plagued student self-reported 'engagement' measures, with the remaining weight determined strictly by input factors (student enrollment and faculty resources).
Georgtown University's Center on Education and the Workforce is moving in the right direction with the November 2019 release of its College ROI, ranking institutions on students' future earnings in relation to net cost of attendance and cumulative college debt. However, the ranking does not control for institutional resource and student inputs in determining an institution's position, thus stopping short to provide a value-added assessment of an institution's contribution to the student's post-attendance financial success.
Enter College ROI: The value-added college ranking
In contrast to popular ranking surveys, the rankings here estimate four key academic progress and post-enrollment/graduation student financial outcomes after taking into account student and institutional resource input factors. This normalizes the ranking across the wide range of institution types, offering the consumer the opportunity to compare highly selective research universities that are financially well endowed with institutions that are less selective with limited resources. Thus, an institution's rank is determined in light of a broad range of input factors that influence the measured student outcomes in order to gauge the value-added contribution of the institution. The institutional value added is based on a composite score, weighted on all four measured outcomes, that determines an overall rank as well as rank across each of the four outcomes.
The four outcomes include the following:
- First-year student persistence (freshman retention rate)
- Six-year graduation rate of Bachelor-degree-seeking students
- Federal student loan repayment rate
- Intergenerational income mobility rate.
The latter two outcomes respond to the demand for ranking institutions on the financial well being of its students after attendance. As research shows, one in four college graduates earns no more than a high school completer, thus limiting both the ability to pay down debt and fare better than the parents. All four outcomes look at undergraduate-program students, excluding students enrolled in only graduate-level programs.
Description of data sources, institutional resource input factors, measured outcomes, and the statistical estimation methods employed and data glossary are available from the Office of Institutional Research.