The Evaluation Game: Why the Context of Implementation of Research Evaluation Systems Matters
Today, researchers are publishing more than ever before. New assistant professors have already published twice as much as their peers did in the early 1990s to secure a position in top departments or to achieve tenure. Nobel laureate Peter Higgs believes he wouldn’t be deemed “productive” enough for academia in today’s world. However, merely publishing more papers doesn’t suffice. The number of citations those papers receive is the true currency in science.

Rudolf Weigl, a Polish biologist who invented the first effective vaccine against typhus, described the practice of publishing many papers as ‘duck shit’: just as ducks leave a lot of traces while walking about in a yard, scientists hastily publish articles with partial results that are the product of underdeveloped thought. This is one of the unfortunate outcomes of the evaluation game in today’s science, where researchers attempt to follow various evaluation rules and meet metrics-based expectations.

Counting scholarly publications has been practiced for two centuries. In Russia from the 1830s, professors had to publish yearly to determine their salaries. The Soviet Union and various socialist countries developed national research evaluation systems before the Western world. The effects of those practices are still vital.

In my talk, I will use the concept of the ‘evaluation game’ developed in my recent book (The Evaluation Game: How Scholarly Metrics Shape Scholarly Communication, CUP 2023) to show how this concept can enrich our understanding of how researchers, institutions, and other stakeholders respond to pressures generated by metrics and research evaluation exercises.

I will offer a fresh take on the origins and effects of metrics in academia, as well as suggest ways to improve research evaluation. I will show why the phenomenon of predatory publishing has not only geopolitical aspects but that publishing in so-called predatory journals might be perceived at the (semi)periphery as a justified and rational way to in accordance with institutional loyalty.
Date: 4 June 2024, 14:00 (Tuesday, 7th week, Trinity 2024)
Venue: 15 Norham Gardens, 15 Norham Gardens OX2 6PY
Venue Details: Hybrid (Zoom and in Seminar Room D 15 Norham Gardens)
Speaker: Emanuel Kulczycki (Adam Mickiewicz University in Poznań and West Pomeranian University of Technology in Szczecin (Poland))
Organising department: Department of Education
Organiser: David Mills (University of Oxford)
Organiser contact email address: david.mills@education.ox.ac.uk
Host: David Mills (University of Oxford)
Booking required?: Required
Booking url: https://www.researchcghe.org/events/cghe-seminar/the-evaluation-game-why-the-context-of-implementation-of-research-evaluation-systems-matters/
Cost: FREE
Audience: Public
Editor: Kristina Khoo