Qualitative Judgement of Research Impact: Domain Taxonomy as a Fundamental Framework for Judgement of the Quality of Research

by   Fionn Murtagh, et al.

The appeal of metric evaluation of research impact has attracted considerable interest in recent times. Although the public at large and administrative bodies are much interested in the idea, scientists and other researchers are much more cautious, insisting that metrics are but an auxiliary instrument to the qualitative peer-based judgement. The goal of this article is to propose availing of such a well positioned construct as domain taxonomy as a tool for directly assessing the scope and quality of research. We first show how taxonomies can be used to analyse the scope and perspectives of a set of research projects or papers. Then we proceed to define a research team or researcher's rank by those nodes in the hierarchy that have been created or significantly transformed by the results of the researcher. An experimental test of the approach in the data analysis domain is described. Although the concept of taxonomy seems rather simplistic to describe all the richness of a research domain, its changes and use can be made transparent and subject to open discussions.


page 1

page 2

page 3

page 4


COMPARE: A Taxonomy and Dataset of Comparison Discussions in Peer Reviews

Comparing research papers is a conventional method to demonstrate progre...

A Taxonomy of Knowledge Gaps for Wikimedia Projects (First Draft)

In January 2019, prompted by the Wikimedia Movement's 2030 strategic dir...

OntoCat: Automatically categorizing knowledge in API Documentation

Most application development happens in the context of complex APIs; ref...

Using Full-text Content of Academic Articles to Build a Methodology Taxonomy of Information Science in China

Research on the construction of traditional information science methodol...

RaTE: a Reproducible automatic Taxonomy Evaluation by Filling the Gap

Taxonomies are an essential knowledge representation, yet most studies o...

Novel processes and metrics for a scientific evaluation rooted in the principles of science - Version 1

Scientific evaluation is a determinant of how scientists, institutions a...

Please sign up or login with your details

Forgot password? Click here to reset