Bloch School leader: No. 1 ranking lacked credibility

In a week that saw it lose four years of Princeton Review rankings as punishment for fabricating data in rankings applications, the University of Missouri-Kansas City’s business school still had something to trumpet: The No. 1 ranking of its entrepreneurship program.

Dean David Donnelly assured students Wednesday that a recent audit “validated” that recognition for the signature program at the Henry W. Bloch School of Management.

But documents obtained Thursday by The Kansas City Star through an open-records request show that, as far back as last summer, even the head of that program was expressing doubts about that ranking.

In an Aug. 5 email to Donnelly, Jeff Hornsby, managing director of the Bloch School’s Regnier Institute for Entrepreneurship and Innovation, questioned the university’s continued defense of the No. 1 ranking study in the Journal of Product Innovation Management, or JPIM.

He sent the message after a Kansas City Star story on July 26 raised questions about the ranking’s validity and brought the university unwelcome publicity.

“I feel we need to start questioning the results as well,” Hornsby wrote. “As I have stated many times to the PR team and in meetings, we need to be careful defending these outcomes. I personally believe after looking at all the information that this is the only avenue.

“From the onset of the rankings, very few from our Bloch faculty gave the rankings credibility and no one in the academic community gives the JPIM results any credibility. Our continued defense continues to deteriorate our credibility and legitimacy.”

Hornsby had good reason to be concerned. Another batch of emails, these from an academic journal expert, pointed to the study’s “rather arbitrary and subjective” methodology and the authors’ undisclosed connections with a top Bloch School official.

Yet UMKC administrators did not back away from the ranking and continue to defend it, despite the recommendation from Hornsby.

Hornsby wasn’t the only faculty member whose counsel was for UMKC to take a less-defensive approach in dealing with potentially unflattering news about the Bloch School.

Records show that as word of The Star’s investigation spread last spring, business professor Stephen Pruitt urged Donnelly to put a more positive spin on the story by admitting any shortcomings before a story was published.

“I think most people believe in redemption,” Pruitt said in a May 8 email. “I know I certainly do.”

Donnelly and other university officials did stress the positive in interviews with reporters, but downplayed flaws in the Princeton Review data and denied having any skepticism about the JPIM ranking.

Asked why the university didn’t follow Hornsby’s advice, UMKC issued a statement Friday:

“A number of people across UMKC shared their opinions regarding discussions about the JPIM article. Most of these discussions were in person or by phone. The opinions expressed in email represented only a few of many varied points of view and were a small part of the overall discussion.”

Hornsby also issued a statement Friday:

“I voiced my academic opinion to the Dean and he told me that an external audit was going to be conducted and to wait for the outcome of the audit which I agreed to do. ... We have excellent students and faculty who are working hard in the Bloch School. ... We are focused on moving forward.”

Donnelly did not respond to a request for comment. Pruitt said he had no comment.

The Star’s July 26 article quoted experts who challenged the study’s methodology and authorship. They said the study may have been structured in such a way as to ensure that the Bloch School received the top ranking over better-known universities such as Harvard, Stanford and MIT.

While UMKC officials deny that the study was flawed and cite other experts to back that up, the appearance of a potential conflict of interest was beyond dispute.

The paper was written by two UMKC visiting scholars from China. They had help from the UMKC faculty member, Michael Song, whom the study ranked as the top scholar worldwide in the field of innovation management. And it was the program Song headed at the time, and Hornsby now runs, that was rated No. 1 in the world.

Yet none of those ties or Song’s involvement was disclosed, not even to the editor of the journal.

Those ties, however, were not a secret at the Bloch School. Faculty members were skeptical of the JPIM article’s finding as early as late 2011, when UMKC invited 300 guests to hear the announcement of the No. 1 ranking.

UMKC and the editors of the journal continue to maintain that the study’s findings stand on their own, regardless of Song’s involvement. The independent audit by PricewaterhouseCoopers released last week quoted Song as saying he helped edit the ranking paper and “may have written parts related to the strategy portion of the paper.”

Song previously told The Star that he wasn’t sure that he’d even seen the paper before it was published.

Of the top 10 universities mentioned in that section, UMKC was the only one that got more than cursory treatment and Song was the only person mentioned by name.

Song has not been available for comment since the release of the PricewaterhouseCoopers audit, which said he was one of two faculty members responsible for submitting false data to the Princeton Review.

The University of Missouri System’s Board of Curators commissioned the audit last summer at the request of Gov. Jay Nixon. The governor said an independent review was necessary in light of The Star’s investigation.

In Columbia, where the Board of Curators was meeting Friday, chairman Donald Cupps said its members had “reviewed the report extensively.”

“We’ve sent the expectations of what we expect,” Cupps said. “Some things were troubling, but the campus is taking actions to correct the problems. … I won’t characterize what I found troubling or not troubling. I think the report speaks for itself.”

The wrong question

Questions about the JPIM study first arose shortly after its publication in March 2012.

Richard Arend, a Bloch School professor, challenged its findings in letters to the Chinese scholars and to officials at the journal and its publisher.

In response to Arend’s questions, the publication board of JPIM assembled a three-member panel to evaluate the paper. Its members found no reason to question its validity, although one pointed out that the outcome might have been different depending on the variables.

Indeed, one of the study’s authors acknowledged in an earlier email obtained by The Star that UMKC would not have even made the study’s top 10 list if a more common methodology had been used.

The Star, meanwhile, assembled its own panel of experts during its investigation last year. One of them was Ben Martin, an editor of Research Policy, a leading business management journal.

Martin told The Star that the ties between the authors and the university should have been disclosed so that readers could judge for themselves whether to trust the paper’s findings.

As Star reporters prepared to write their story, they informed UMKC that they had consulted Martin.

UMKC’s director of media relations, John Martellaro, then contacted Martin, according to internal UMKC emails that The Star obtained Thursday. He wanted Martin’s reaction to the review by JPIM’s panel.

Martin’s conclusion: The JPIM panel considered the wrong question. It wasn’t a matter of bad methodology so much as poor disclosure, he wrote.

In his July 3 response to Martellaro, Martin said that the authors of the study were obliged to state their associations with UMKC. He cited the policy of John Wiley & Sons, which publishes the journal the article appeared in. It states:

“Editors, authors, and peer reviewers have a responsibility to disclose interests that might appear to affect their ability to present or review data objectively. ... If there is doubt about whether conflicts are relevant or significant, it is prudent to disclose.”

In an open letter sent last summer, authors Pianpian Yang and Lei Tao said they felt that disclosing their ties to UMKC would have been inappropriate. They did not respond to a request for comment Friday.

Martin said in his email that one reason members of JPIM’s independent review panel found no reason to question the No. 1 ranking study was that they were asked whether there was “a fatal flaw” in the article’s methodology. Neither he nor they found one.

“However, there were,” Martin wrote, “a number of subjective and debatable or even questionable methodological decisions that others might disagree with.”

The authors counted the number of articles that researchers had published in a select set of journals and over a time frame that matched Song’s career.

Martin wrote that the study’s outcome depended “on the precise methodology chosen by the authors” and included “arbitrary and subjective” decisions such as the choice of journals.

“Given this,” Martin wrote, “the authors should have declared any personal, professional or institutional interest that a reasonable reader might perceive as having partly influenced one or more of those methodological choices.”

A half-hour after receiving Martin’s email, Martellaro sent a note to his boss, Anne Spenner, vice chancellor of marketing and communications: “Bottom line: Martin is taking the Star’s side.”

JPIM officials said earlier this week that they would have no further comment on the academic paper.

UMKC continues to defend the JPIM study and that lack of disclosure. In a news release last week, the university cited the conclusions of Robert D. Hisrich, a professor emeritus of entrepreneurship at the Thunderbird School of Global Management in Glendale, Ariz. He was hired by the Board of Curators to review the PricewaterhouseCoopers audit.

“The journal article that led to the Bloch School’s top ranking in innovation management research,” UMKC said, “was consistent with generally acceptable professional practices.”

Student reaction

On UMKC’s campus Friday, reaction to the lost rankings and the controversial study ranged from disappointment to indifference.

Taylor Johnson, a senior from Liberty who is double-majoring in liberal arts and foreign language, said, “It is really disappointing and embarrassing for the Bloch School.

“They are trying to give off this professional brand; they teach us in class the importance of being truthful and in giving the correct numbers, and then they don’t do it themselves.”

At the Bloch School, entrepreneurship major Jasin Wang, a senior from Indonesia, said, “I’m disappointed, but I do feel that I receive a great education here and I think that is what is important.”

Wang said he came to UMKC for its reputation as a good business school. When he first heard the school’s rankings had been stripped because of the false data, “I was worried,” he said, about how such a blemish would affect his ability to get a job.

But he said other students had convinced him that the lost rankings wouldn’t hurt his job prospects.

David Wallace, a psychology and biology major from St. Louis, said the controversy didn’t affect him. “I am not in the business school and I didn’t come to UMKC because of the business school.”

Alex Mikic, a Kansas City freshman majoring in psychology, said she found it “annoying” that problems in the business school might affect the way the entire university is viewed.

“I don’t think that it is fair,” she said, “because that is not what we are as a university.”

Zach Anderson Pettet, a senior majoring in entrepreneurship at the Bloch School, said he doesn’t think losing rankings or a few people being involved in “back alley” deals will change how business students feel about their school.

“This is an awesome school,” said Pettet, a Kansas City native who was selected student entrepreneur of the year for 2014. “If anything, this has taken the focus off of rankings and now the focus is going to teaching. That is where the value is. The value was never in the rankings.”

This story was originally published February 6, 2015 at 5:57 PM with the headline "Bloch School leader: No. 1 ranking lacked credibility."