Online Education

History and Context for U.S. News & World Report Rankings

U.S. News & World Report has published rankings for colleges, universities, high schools, and global universities since 1983. Over the years, these rankings have become important to students, parents, governing boards of institutions, administrators, and employers in gauging the quality of institutions and academic programs. Top-ranked institutions receive national attention as leaders in higher education. With the advent of online programs, U.S. News & World Report launched the U.S. News & World Report’s Best Online Program rankings in 2012. Numerous competitors have released ratings in the online realm in recent years, but none is held in higher regard than the U.S. News & World Report online rankings.

In 2015, the U.S. Department of Education launched the “College Scorecard.” On this website, students can search for programs and degrees at 2- and 4-year institutions. The database allows students to compare institutions on the following key metrics: average annual cost, financial aid and debt, graduation and retention rates, and salary after attending. It should be noted that the College Scorecard does not rank  universities.

In recent years, many start-up companies have also entered this space. The methodology for calculating rankings often varies across these entities (for example, the period of time during which the rankings are calculated is not consistent). Some examples include:

  • Guide to Online Schools. According to the Guide to Online Schools, website rankings are based on data gathered from the official website of the college, the National Center for Education Statistics, the U.S. Department of Education, and GradReports.com (a sister company). The factors taken into account with the rankings are: tuition, recommendation rates, default rate, accreditation, for-profit vs. non-profit status, and student services. The website offers three rankings: best value colleges, most affordable colleges, and top colleges.
  • Open Education Database. Open Education Database uses the following metrics to rank online programs: number of full-time faculty per part-time faculty member, institutional financial aid rate, acceptance rate, retention rate, graduation rate, years accredited, default rates, and job placement rates. Programs are ranked in 11 areas of study.

We note that some of these companies may have contractual arrangements with individual institutions to promote their enrollments which call into question the credibility of their rankings.

 

U.S. News & World Report Online Program Rankings

The 2016 U.S. News & World Report Best Online Programs rankings included online bachelor’s degree programs, as well as the following online graduate degree programs: MBA, business, criminal justice, education, engineering, information technology, and nursing. Participation in the online rankings has grown rapidly from just a handful of universities in 2012. In 2016, U.S. News & World Report received data from over 1,250 online degree programs, about 50 more than the previous year. It should be noted that online programs, rather than universities, are ranked, and no distinction is made between the for-profit and not-for-profit institutions.

Responses to an extensive questionnaire with more than 100 questions were required for consideration in the 2016 edition of the online rankings. U.S. News & World Report considers many factors in calculating the rankings, including program reputation, class size, assessments, technology infrastructure, student support services, faculty credentials, technical staff available to faculty, student indebtedness, time to degree deadline, graduation rates, and one-year retention rates.

Even though over 1,250 institutions participated in the 2016 U.S. News & World Report rankings, many prominent institutions that offer online programs did not. The University of Maryland University College (UMUC) is one of the largest providers of online programs. In 2014 it offered more than 4,100 bachelor’s degrees, and it enrolled nearly 48,000 students. Yet UMUC did not participate in the 2016 U.S. News & World Report rankings, as reported in a Washington Post article.

 

Methodology for Best Online Bachelor’s Programs

For the 2016 rankings, U.S. News & World Report assessed online bachelor’s programs based on the following four categories (relative weight for each category is mentioned in parentheses) and constituent factors:

Table 1: U.S. News & World Report ranking categories and factors

Category (weight %) Factors
Student engagement (40%) Quality of learning experience; opportunities to engage with instructors and peers; retention rates; graduation rates; class size; time to degree completion
Faculty credentials and training

(20%)

 Faculty with terminal degree; faculty who have more than a bachelor’s degree; faculty with a master’s degree or higher; tenured faculty; preparedness to teach distance learners; years of experience teaching distance learners; technical support staff available to faculty
Student services and technology

(20%)

 Use of diverse online learning technologies; support services that provide learning assistance, career guidance, and financial aid resources
Peer reputation (20%) To account for “intangible” factors affecting program quality that are not captured by statistics, U.S. News & World Report surveys high-ranking academic officials and employers. Thus, for example, degrees from programs that are well respected by academics may be held in higher regard among employers.

 

A program’s score for each factor is calculated using data that the program reported to U.S. News & World Report.

 

Category Scores and Rankings

U.S. News & World Report multiplies each factor score by the weight it has selected for the factor, then sums these values to compute the four separate category scores. Each of these category scores is rescaled so that the top-scoring school receives a display score of 100, and the bottom-scoring school receives a display score of zero. Programs with overall scores in the bottom 25% are categorized as “Rank Not Published.”

The methodology for calculating rankings for online master’s degree programs is similar, with some changes to categories and factors within categories.

U.S. News & World Report also provides rankings for online programs for veterans, for which the following conditions need to be met:

  • Schools included in online programs for veterans-focused rankings first must be ranked among the top 75% of schools in U.S. News & World Report Best Online Programs rankings.
  • Programs included in the veterans-focused rankings also have to belong to an institution that is certified for the GI Bill and participates in the Yellow Ribbon Program.
  • Online programs at public institutions that do not participate in the Yellow Ribbon Program are still eligible to be ranked if they charge in-state tuition, which can be fully covered by the GI Bill, for all veterans applying from out of state.

 

Recommendation and Strategies

Although responding to the U.S. News & World Report rankings questionnaire can be a cumbersome process involving upwards of 50 hours of combined effort across various departments, it can also be a valuable exercise in itself for an institution. The development of an institutional strategy for preparing responses to the U.S. News & World Report questionnaire is critical to the institution that hopes to increase its standing in these rankings. Some institutions choose to commission services from a third-party vendor to complete the questionnaire on their behalf. Typically, this option is chosen because of the belief that the vendor will have more experience with the methodology and be able to provide the most strategic answers leading to successful outcomes. While acknowledging that this approach may find favor with some institutions, we are going to more fully explore the strategies used by institutions that prepare responses to the questionnaire in-house.

First, institutions should acknowledge that the U.S. News & World Report questionnaire requires information to be gathered from across campus. Offices of instructional research, financial aid, undergraduate and graduate offices, faculty teaching and learning centers, information technology, technology support services, student tutoring and writing centers, and admissions offices all hold components of the data that will need to be compiled in the application.

The next step is to review the information gathered and to determine the consistency of data being reported on a single program submission. Several questions in the U.S. News & World Report questionnaire overlap or have multiple sections. Often, a “yes” response will then require follow up responses to further clarify the initial answer. Narrative data should clarify and exemplify the program being reported. Institutions should not simply restate data that has been reported in other sections of the questionnaire.

Then, data should be compared between program submissions. For example, if student tutoring is available for students in the business program, is it also available for engineering students? Are certain services only available to undergraduates or are all services available in both undergraduate and graduate programs?

Finally, all data should be reviewed by an individual who can check for inconsistencies or errors in data reporting. Program data should only be submitted after as many questions as possible have been given a response and the institutional representative feels confident that the data is complete and adheres to a high level of data reporting integrity.

 

2016 Best Online Bachelor’s Programs

A broad spectrum of institutions was represented in the top 50 of the 2016 U.S. News & World Report Best Online Bachelor’s Programs rankings. Table 2 provides descriptive statistics on the following factors for institutions that were ranked in the top 50: year founded, numbers of students, and tuition. Interestingly, among the top 50 institutions in the Best Online Bachelor’s Programs rankings, 10 institutions enrolled less than 200 students while 27 enrolled more than 1000 students. The distribution of the number of students at these institutions is negatively skewed (mean: 2618 and median: 1265). The earliest program was founded in 1993 and the most recent in 2012, again a wide range. Tuition at these top 50 ranges from $92 to $766 per credit unit.

 

Table 2: 2016 top 50 Online Bachelor’s Programs: Year founded, number of students, tuition

Measure Year founded Number of students Tuition (per credit unit)
Min 1993 35 $92
Max 2012 16788 $766
Mean 2003 2618 $390
Median 2003 1265 $390

 

In the 2016 Best Online Bachelor’s Programs rankings, 10 schools were not ranked because they reported less than 10 students or they were less than one year old at the time of data collection.

Per marketing communication from U.S. News & World Report, programs that were ranked in the top 50% in their category earned eligibility to display a U.S. News & World Report “badge” on their websites, in advertisements, and in other promotional media. Furthermore, seven master’s-level Best Online Programs rankings will be featured in U.S. News & World Report’s Best Graduate Schools 2017 Guidebook, while the Best Online Bachelor’s Programs rankings will appear in its Best Colleges 2017 Guidebook.

 

Interview with Robert Morse, Chief Data Strategist at U.S. News & World Report

  1. When and why did U.S. News & World Report decide to have online program rankings? The first edition of U.S. News & World Report’s Best Online Program rankings was released in 2012. U.S. News & World Report believes online learning is fast becoming an integral part of all types of education, including higher education, and that consumers are hungry for information related to online degrees. In our fifth year of collecting data from distance education programs, we have been able to expand the range and depth of our information. We have assessed more than 1,200 online degree programs, up 70 percent from the inaugural list. A defining characteristic of these rankings is that they are of programs, not schools. As the methodologies section of the report  makes clear, these are degree-granting programs at regionally-accredited institutions that have few, if any, in-person attendance requirements. This allows for apples-to-apples comparisons. It also means many institutions that perform well in U.S. News & World Report’s other education rankings are ineligible for the Online Programs rankings. MOOCs are not included, nor are blended learning programs, which have significant in-person attendance requirements paired with their online delivery components. Another defining characteristic is U.S. News & World Report makes no distinction between the for-profit and not-for-profit sectors. Each program is rated on a level playing field.

  2. How do you compile the rankings from the questionnaires? What factors should institutions keep in mind when preparing an application? When completing their questionnaires, schools should attempt to respond to questions as thoroughly as their data and history allow. The rankings are calculated based mostly on program-level data schools report to U.S. News & World Report, such as best practices for online course delivery, graduation and retention rates, and services made available to students remotely. But respondents must answer the questions to get credit. U.S. News & World Report designed its survey questions following discussion with schools—both through one-on-one interviews and larger forums, including a couple that UPCEA kindly facilitated. We continue to solicit and incorporate feedback each year on how we can make it easier for respondents to collect and report data on their distance education programs. Schools may email [email protected] with questions or input.

  3. How many hits do U.S. News & World Report online program ranking pages get? How do these numbers compare to page hits for rankings of brick and mortar programs (are they increasing year over year)? As a matter of policy, U.S. News & World Report does not disclose web traffic figures for any of its education ranking pages. We have been satisfied with the product enough to commit to investing further in it. In 2015 we expanded the product by publishing our first ever rankings of online master’s degree programs in criminal justice/criminology, and split our ranking of online graduate business programs into two separate rankings of MBA degrees and non-MBA business degrees. U.S. News & World Report has also increased its homegrown editorial content on non-traditional learning. A new blog, Online Learning Lessons debuted in September and provides consumer oriented tips ranging from time management skills to how to study overseas. Institutions are increasingly seeing that the Best Online Program rankings are worth their while, too. In the 2012 inaugural data collection, schools submitted 710 completed surveys on their online degree programs. That number has risen each year through 2015, when 1,207 surveys were submitted. This growth far outpaces growth in the industry based on data we have seen.

  4. What do you consider to be the most interesting trends since the launch of the rankings—for example, growth in the number of institutions submitting data; change in profile of institutions that apply? An interesting trend is the extent public institutions are serving non-traditional learners. For example, 16 of the 25 highest rankings online bachelor’s degree programs in 2015 were housed at public institutions. These programs perform well in the rankings because they often have solid national reputations, are affordable to students and their large sizes give them the economies of scale needed to make the large up-front investments needed in training and infrastructure for establishing online programs. Another notable trend is with the passage of time, not only are more schools participating, but they are better at submitting statistical data on key ranking indicators. U.S. News & World Report has also become more experienced at assessing their data. For these reasons, among others, there has become greater stability in the Best Online Programs rankings.

  5. How influential are U.S. News & World Report online program rankings with student decision making? What percentage of the market share does U.S. News & World Report occupy? What is the competition? As far as we know, U.S. News & World Report is the only organization ranking online programs in an analytically rigorous way. There are a few websites that publish lists of top schools, but these lists rely on a small number of institutional characteristics that, U.S. News & World Report feels, do not offer a comprehensive view of online programs. There are other online program rankings that do not incorporate data at all in their evaluation.

  6. What is the typical profile of learners that use U.S. News & World Report Online Program rankings? The majority of users of the Best Online Programs webpages have at least earned some college credit. They are most often in the 20s to 40s age range. They are much more likely to be working professionals looking to advance in or change their careers rather than be prospective first-time college students.

  7. How many institutions submitted for 2015 rankings? How many of these institutions were ranked? In 2015, U.S. News & World Report ranked 1,207 online degree programs.

Reflections

A strategic approach to preparing responses to the U.S. News & World Report questionnaire will provide a strong foundation for the reporting of institutional data and also provide an overview that will highlight those areas whee there is room for improvement. The UPCEA Hallmarks of Excellence in Online Learning and the Online Learning Consortium Quality Scorecard can assist institutions in strategically addressing these gaps, thereby helping position the institutions for the next year’s U.S. News & World Report online rankings.

 

Ray Schroeder is Professor Emeritus, Associate Vice Chancellor for Online Learning at the University of Illinois Springfield and Director of the Center for Online Leadership at the University Professional and Continuing Education Association (UPCEA). Schroeder publishes and presents nationally in online and technology-enhanced learning. Ray publishes five daily blogs on various aspects of news, research, and trends in technology-enhanced learning in higher education. He is recipient of the 2002 Sloan-C award for the “Most Outstanding Achievement in ALN by an Individual” and the inaugural 2010 recipient of the Sloan Consortium’s highest Individual leadership award – the A. Frank Mayadas Leadership Award. Ray received the 2011 University of Illinois Distinguished Service Award. He is an inaugural Sloan Consortium Fellow, the 2012 Innovation Fellow for Digital Learning by the UPCEA, the 2016 United States Distance Learning Association Hall of Fame Award recipient, and the University of Wisconsin Mildred B. and Charles A. Wedemeyer Excellence in Distance Education Award recipient for 2016.

Khusro Kidwai is Assistant Dean for Distance Learning at Northwestern University’s School of Professional Studies and adjunct faculty in the MS in Information Design and Strategy program. Prior to joining Northwestern University, he was the Director of Online Teaching and Learning at the University of Southern Maine (2010-2012) and Research Associate at the John A. Dutton e-Education Institute at The Pennsylvania State University (2009-2012). In 2013 he was invited to join the board of directors of the International Board of Standards for Performance, Training, and Instruction (IBSTPI). He currently serves as the Vice President for the Board. He received his Ph.D. degree from the Instructional Systems program at The Pennsylvania State University.

Vickie Cook is Director of Online Learning at University of Illinois Springfield and is an associate research professor in the College of Education at UIS. She currently serves UPCEA as an Online Learning Strategist, serves on the Leadership and Strategy network for UPCEA and provides professional development for grant projects with the Council of Public Liberal Arts Colleges (COPLAC). She is the past-president of the Illinois Council on Continuing and Higher Education, provides over fifteen professional development workshops each year for conferences, colleges, and universities around the country, and provides annual professional development with a private school in Mexico City to improve student achievement through teacher in-service.