The Academy of Management conference in Chicago is one of the world’s largest and most important annual gatherings of business school academics. At their most recent conference last August, one of the most prominent themes, which found its way into multiple presentations, was the difficulty in striking a balance between rigor and relevance in academic research.
Among the hundreds of projects discussed at the conference, several studies stood out as highly impressive, with astounding implications for their application. However, many others were largely rooted in theoretical ideas with little applicable value. Research with real-world practical applications was less evident, and research focusing on addressing the most important issues facing society, such as climate change, poverty, and inequality, was all but absent. In the months since the conference, this has incited substantial debate about the purpose of academic research in this field.
The Debate Over Purpose
This points to a broader debate about the role of business schools, their relationship to the world beyond universities, and the nature of academic governance and incentives around research. To some academics, any attempt to hold them accountable through measurement of their outputs is a threat to their independence in principle and is doomed to fail as a way to drive better outcomes in practice.
But, as George Feiger, a veteran business school dean who also worked as a management consultant and banker, puts it: “The real issue is a truly dramatic lopsidedness in focus in business schools, where academics measure their success — and their institutions evaluate them — almost entirely according to publications in a proliferating array of journals that nobody reads.” Attempts to hold some academics accountable, by measuring their outputs, is doomed to fail The FT, like other organizations seeking to assess academic research, has long used as its yardstick “high impact” peer-reviewed academic articles published in leading journals — even though the approach has limitations.
The Role of Metrics in Value
Publication in journals highly regarded by academics is a reasonable proxy for the best recently released articles considered valuable by other academic experts from the relevant journals’ editorial advisory boards and peer reviewers.
These metrics are closely watched by those seeking ways to assess academic quality. They include librarians considering which journals to stock, as well as external funders and scholars themselves when evaluating recruitment, promotions, benchmarking against other institutions, and reviewing draft papers and grant applications. However, the journal’s ‘impact factor’ is an average measure based on the impact of past articles, not a guarantee that each new paper will hold the same significance.
Original research may be dismissed ahead of a paradigm shift. Any specialist list of high-impact journals may exclude articles and publications in emerging fields or interdisciplinary work that has traditionally sat beyond the core domains of business schools, such as artificial intelligence, sustainability, or neuroscience.
Read the Papers, Not the Metrics
One academic at Chicago’s Booth School of Business declares, “We don’t look at the metrics; we read the papers!” Those with expertise are best able to make such assessments of their peers’ work. However, metrics will inevitably continue to have a role for those under time pressure or without the same level of insight.
Yet, it typically takes several years for a representative volume to emerge. That limits the value of these figures in regularly updated rankings, which seek to capture quality recent academic output. In addition, the level of citations varies between different academic disciplines, even within business.
Resonance considers how far academics’ work has been disseminated more broadly. Of all academic departments, business schools should engage with practice by disseminating the best ideas and engaging with policymakers and decision-makers in the public, private, and non-profit worlds.
This approach is not intended to undermine academic independence or to dismiss the importance of theoretical work distant from any practical application. Nor can it fully demonstrate the ultimate “impact” of research on society. Many ideas once in vogue are flawed and even counter-productive. Valuable concepts that might first have been developed in academia may be significantly changed before adoption. Implementation is often uncertain and slow, and practitioners may be reluctant to credit innovations to their original authors.
