Introducing College ROI: The Value-Added College Ranking Based on Students Graduating, Earning, and Repaying

The Limitations of Current Rankings

Higher education institution rankings have enjoyed an enormous rise in popularity over the past decades, and are considered a key source for evaluation by the public.  According to one source, US News & World Report's 2014 ranking website drew 2.6 million unique visitors, with 18.9 million page views, during the rankings release day alone.  However, there is mounting evidence that these rankings are methodologically flawed and yield little insight about the quality of an institution to inform prospective students or their parents (see here, here, or here).  Some argue that rankings magnify economic inequality, as they rest predominantly on selectivity without gauging student effort to succeed. Criticism stems in part from year-to-year changes in ranking methodology, giving rise to efforts to track such changes.  Current rankings engender varied limitations, key among which is reliance on input factors that describe an institution, most prominently the institution's admission selectivity, preparation level of enrolled students, faculty resources/research, campus diversity, and perceived reputation, to name a few, without centering the ranking on student outcomes after attending an institution.  And while some rankings use student outcome measures (e.g. student graduation rate), none of the popular rankings employ a methodology that gauges an institution's value-added contribution to their students' post-attendance success.  The Wall Street Journal/Times Higher Education (WSJ/THE) Ranking, relying on the largest number of weighted ranking factors, determines 60 percent of an institution's rank for its global ranking on the basis of faculty research and publication citations alone, without consideration for post-graduate student outcomes.  And while its separate US College Ranking determines 19 percent of an institution's rank on the basis of students' expected post-graduate income and student debt repayment, it allocates 20 percent of the total rank to validity-plagued student self-reported 'engagement' measures, with the remaining weight based strictly on input factors (student enrollment and faculty resources). 

Institutional Value Added and Student Post-Attendance Financial Condition

In contrast to popular ranking surveys, the rankings here estimate four key academic progress and post-enrollment/graduation student financial outcomes after taking into account student and institutional resource input factors.  This normalizes the ranking across the wide range of institution types, offering the consumer the opportunity to compare highly selective research universities that are financially well endowed with institutions that are less selective with limited resources.  Thus, an institution's rank is determined in light of a broad range of input factors that influence the measured student outcomes in order to gauge the value-added contribution of the institution.  The institutional value added is based on a composite score, weighted on all four measured outcomes, that determines an overall rank as well as rank across each of the four outcomes.  

The four outcomes are:  First-year student persistence (freshman retention rate), 6-year graduation rate of Bachelor-degree-seeking students, federal student loan repayment rate, and intergenerational income mobility rate.  The latter two outcomes respond to the demand for ranking institutions on the financial well being of its students after attendance.  As research shows, one in four college graduates earns no more than a high school completer, thus limiting both the ability to pay down debt and fare better than the parents.  All four outcomes look at undergraduate-program students, excluding students enrolled in only graduate-level programs.  Description of data sources, institutional resource input factors, measured outcomes, and the statistical estimation methods employed can be found here, supported by a data glossary

Interactive Value-Added Ranking Report

The first page of the following interactive dashboard provides the overall rank across all four outcomes using a composite score for each institution with available data (currently over 550 institutions). Subsequent pages, with page link at bottom, rank institutions on each individual outcome in tabular and geographically mapped format.  Click on the expansion arrow (bottom right) to view in full-scale mode.  For further questions, click here.  A video presentation of the ranking system is available here, with the accompanying presentation slides.