A Change in Methodology: Pepperdine Drops In Rank

A national university offers a “full-range” of undergraduate programs, along with masters and doctoral programs, according to U.S. News & World Report. These schools also participate in “groundbreaking research” and range from large state universities to smaller private institutions such as Pepperdine.

This year, the rankings had more significant methodology changes than in years previous, said Kendra Killpatrick, associate dean of Seaver College and professor of Mathematics. These changes removed some of the metrics that Pepperdine — and universities similar to Pepperdine — typically score well in.

“Schools that focus on that student-faculty interaction, teaching, high touch, some of those metrics were removed,” Killpatrick said. “And then, they’ve added some metrics — particularly 4% — now, rating faculty scholarship metrics, publications, citations that favor a larger research institution where scholarship is more the focus of their instructional faculty.”

U.S. News & World Report eliminated the following factors: “proportion of graduates who borrowed federal loans, high school class standing, alumni giving rate, terminal degree faculty and class size,” according to U.S. News & World Report.

The top-rated schools remained steady, but this was not true for schools lower down — where Pepperdine is — according to U.S. News & World Report.

“This is mostly due to changes in the ranking factors – how these schools scored on factors that were eliminated and criteria that were newly implemented,” according to U.S. News & World Report.

The weight for each part of the U.S. News & World Report’s ranking for 2023-24. Infographic by Samantha Torre

The weight for each part of the U.S. News & World Report’s ranking for 2022-23. Infographic by Samantha Torre

U.S. News & World Report added the research metric this year, which accounts for 4% of a university’s overall score, according to U.S. News & World Report.

U.S. News & World Report extracts metrics from a third-party system, Elsevier, which measures citation per publication — how much each publication a faculty member releases is cited — field impact and amount of publications in the top-cited journals, said Gabriella Yu, manager of Data and Analytics.

Large public institutions that have significant funding or strong research universities where faculty papers are cited often will do well in this ranking, Yu said.

“They just have a very different focus than liberal arts colleges and private universities,” Yu said.

When considering research, Kilpatrick said U.S. News & World Report considered research across all fields — not just STEM.

This research is also university-level data, so the U.S. News & World Report considered research from all five of Pepperdine’s schools, Yu said. However, the exact correlation is not that clear.

“Depending on the focus of different grad programs and grad schools at different universities, of course, the focus [of research] will be different,” Yu said.

In previous years, universities would report what percent of classes had under 20 students, Killpatrick said. Last year, 71% of Pepperdine’s classes were this size. This year, the number has grown to 74%.

In that metric, Pepperdine ranked second out of the top 100 universities and 40th overall when including smaller universities that do not rank in the top 100, Killpatrick said.

U.S. News & World Report removed the class size category this year, Killpatrick said. The class size category used to count for 8% of a university’s overall score.

“That is one [category] where we do very well on and continue to do well on even though it doesn’t count in the rankings,” Killpatrick said. “It counts in terms of what we feel is important. That is the real student experience — how big are those classes? So, that’s something that’s been taken out. That really affects us.”

Graduation rate and retention rate were two categories that did not go through significant changes in methodology, aside from how much they weigh in the score, Yu said. For the six-year graduation rate, the previous year’s weight was 17.6%; this year’s is 21%. The first-year average retention rate was weighted at 4.4% last year, and is now weighted at 5%.

Graduation rate is weighed at 21% for universities without useable standardized test scores, according to U.S. News & World Report.

“For schools with less than 50% of both their fall 2022 and fall 2021 entering classes reporting SAT/ACT scores during the admission process — including test-blind schools — US News did not assess their test scores at all,” Yu wrote in a Sept. 21 follow-up email to the Graphic. “Instead, the 5% weight was reallocated to graduation rate.”

For the class of 2015, Pepperdine had a 90% first-year retention rate, according to OIE data. The six-year graduation rate for that same class — which would be the 2022 graduation year — was 83%. For the class of 2014, the retention rate was 94%, and the graduation rate was 89%.

This year, U.S. News & World Report included new factors such as changing the definition of social mobility to include first-generation graduation rates, along with Pell Grant graduation rates, according to U.S. News & World Report. This had the intention of “place[ing] more emphasis on social mobility and outcomes.”

“These new first-generation rankings factors are based on graduation rates of federal loan recipients who entered college between fall 2011 and fall 2013,” according to U.S. News & World Report.

For this classification, neither parent can have a record of attending college, or the information is unknown, based on the FAFSA form, according to U.S. News & World Report.

“We [Pepperdine] do better on those metrics: our Pell Grants graduation rate is higher, our Pell proportion is higher,” Yu said. “And our Pell Grant students’ graduation performance is always strong. Even our first gen students’ graduation rates are consistently either the same level or even better than the overall average of our students.”

The six-year graduation rate for first-generation students was 82% for the fall 2015 incoming class, according to the OIE report. For Pell Grant recipients, the graduation rate was 87%.

There are two metrics on the Pell Grant category this year, Yu said. The first is raw graduation rate, which is a four-year average this year — and a comparative rate comparing Pell Grant recipients with non-Pell Grant Recipients.

“For both of the metrics, we significantly improved,” Yu said. “For Pell Grant graduation rate, which is the raw graduation [rate], we improved from 285 in that rank category to 36 this year, and this 36 is among all 436 national universities.”

While Pell Grant students perform very well at Pepperdine, the proportion of Pell Grant students at private universities is lower than the proportion at state universities, Yu said.

In previous years, U.S. News & World Report used to multiply these two together, which had a different effect, Killpatrick said. Now, the proportion of Pell Grant students is an add-on percentage.

“They [U.S. News & World Report] take your [University’s] graduation rate — and ours is in the 80s — and then they add on to that your percentage of your student body that has a Pell grant,” Killpatrick said. “For us, that’s about 20%; so, at a larger school, that add-on percentage could be quite a bit bigger.”

Yu said, among top 100 schools, Pepperdine is No. 22 in terms of the comparative rate.

“We’re strong in these two categories that are supposed to measure social mobility, which again, tied to the best values, can highlight how the resources within Seaver have been pouring in to support our students from different backgrounds,” Yu said.

Peer Assessment consists of 20% of the ranking, Killpatrick said. U.S. News & World Report uses a survey sent out to university presidents, provosts and admissions officers. This score has remained steady from previous years. This year, Pepperdine achieved a 3.5 out of 5.

“We do particularly well with deans of admission, who really know other schools and sort of know the weeds of what kinds of programs and what kinds of experiences students get at different schools,” Killpatrick said.

This year, Killpatrick said, the methodology changes were bigger than what universities have seen before. Usually, there are little shifts such as a tweak in percentage or way in which a category was calculated.

“I would encourage students to look at the breakdown of that ranking,” Killpatrick said. “And we’re talking about looking at those things that measure what Seaver College is good at, like why you came to Seaver College.”

In many of those metrics, Seaver has improved, Killpatrick said.

“I would say to students, that on those metrics that we find important, the news is very encouraging,” Killpatrick said.

The University is committed to its Christian mission and academic excellence, Killpatrick said. Graduation rates, retention rates and class sizes are among what is important for Seaver college, and Seaver college will continue to examine that data.

“Our ultimate goal is to provide students with a transformative educational experience as they pursue lives of purpose, service, and leadership while consistently improving our position and value among the top Christian universities in the world,” Gash wrote.

Junior Jade Gonzalez said, when she first came to Pepperdine, the University was ranked 49th before it fell to 55th for 2022-23 and now, to 76th/

“It made me feel a little embarrassed,” Gonzalez said.

From the University, Gonzalez said she would like to hear an explanation as to why the ranking fell. She cited Pepperdine’s rise in cost as a reason for her frustration. Gonzalez said after the change in rank, a state school she had turned down to attend Pepperdine has risen above Pepperdine in the rank.

“I have had so much financial aid where you [the University] keep raising it, I can’t go here,” Gonzalez said. “I’m currently in the process of an appeal because I can’t go here, and I’m frustrated that that is going on. It seems as if it’s easier and easier to get in, but you’re costing more and more money so it’s not as exclusive.”

For Gonzalez, going back up to the 60s would be a good start for next year, she said, but the substantial drop in rankings is making her question the quality of Pepperdine’s education.

“If I’m not getting this education that I’m paying so much for, working so hard for, like, what am I doing?” Gonzalez said.

Senior Alex Consentino said, while the ranking was not necessarily a big factor in his decision to apply, he did appreciate the level of “prestige” that came from being a top 50 university.

“It’s just kind of disappointing to see,” Consentino said. “Somewhere that I thought was a very prestigious institution going backwards.”

Consentino said he researched the reason for the drop through U.S. News & World report and said, though the University is trying, there is a disconnect between what students want and what the University believes students want.

Additionally, Consentino said he would have liked to see the University acknowledge the drop through an email or social post.

“Just say, ‘Hey, listen, this happened,” Consentino said. “What do you guys think we should do better? Because it’s, obviously stemming from the fact that students are unhappy being here. A lot of kids transfer.”

Pepperdine’s top 50 ranking was a big reason why senior Sabrina Musharbash chose the University, Musharbash said.

“Especially when you get into a bunch of colleges and get good financial aid packages to a bunch of colleges, the rank becomes a huge thing,” Musharbash said.

After the change, Musharbash said she saw colleges she had turned down ranked above Pepperdine.

“I just feel like Pepperdine was so much more competitive when I first got in,” Musharbash said. “But now, I just see the worth of my degrees slowly declining. And I haven’t gotten the degree yet.”

Talking about the reasons for the change feels like the University is making excuses, Musharbash said.

“No one’s going to be looking at the numbers like ‘It’s OK because they [U.S. News & World Report] changed the way they look at class sizes,'” Musharbash said. “At the end of the day, we’re still 70-something.”