Understanding the U.S. News University Rankings

As high school seniors commit to universities this spring, many look to the U.S. News & World Report’s college rankings. The annually released report, now in its 37th year, has long been relied upon to find the best future for oneself.

In recent weeks, however, questions are being raised about the integrity of the list. Below, we look at the data, methodology, and sources that U.S. News utilizes in order to understand both the report itself and its criticisms.

The U.S. News provides a ranked list of 392 undergraduate institutions, as well as their out-of-state tuition cost and enrollment number.

The relationships between each of these features (rank, tuition, and enrollment) emerge when we visualize the data.

   

There is a clear trend, that colleges with higher tuition costs also have a higher rank. In general, public institutions have higher enrollment and lower tuition costs when compared to similarly ranked private ones.

The comparison of university quality, cost, and size is a valuable resource for high school students. In order to form this list, U.S. News uses nine categories of factors, each given a certain importance, or weight. 

Problems arise, however, when considering the sources of the data. U.S. News writes in their methodology explanation that, “Only thoroughly vetted academic data from our surveys and reliable third-party sources are used to calculate each ranking factor.”

But relying upon university-provided data is risky. Institutions desire a high rank, and can easily modify the information they submit about themselves. This has happened several times in the past, as reported on by The New York Times and Inside Higher Ed.

Recently, a Columbia University math professor disputed Columbia’s No. 2 ranking. Dr. Michael Thaddeus dove deep into Columbia’s score for each of the factors, and determined that the numbers the university reported didn’t line up.

The factors that Dr. Thaddeus analyze have varying significance in U.S. News' calculations. The weights can be seen here:

The weights of each of these factors helps us understand the significance of the fudged values. Even though the reported 96% graduation rate that Columbia reported doesn't seem too different from the true 92%, because the factor is weighted so significantly in the rankings (35%), it drastically improves Columbia's place. Dr. Thaddeus explains in his report, "Using the first figure, Columbia is in 6th place, surpassed only by Harvard, Notre Dame, Princeton, Yale, and Duke. Using the second, it slips all the way to 26th place."

According to The New York Times , Columbia still stands by its data, as there is no standardized method for calculating these values. This lack of standardization is precisely why university rankings are misleading and inaccurate. While at a surface level the lists might appear helpful for decision-making, the unreliable data proves to create an unreliable report.