A citation in a humanities and social science paper may be more likely to be praising another academic’s work than one in a pure science journal article, new research suggests.
The study, led by an academic at National Taiwan University, analysed almost 27,000 citations across six different humanities and social science disciplines to estimate what proportion actually offered “essential and affirmative support to the citing paper”.
One criticism of using bibliometrics to assess research quality is that it assumes that all citations carry equal weight and are always a positive endorsement by an author of previous studies.
However, as Chi-Shiou Lin, director of National Taiwan University's department of library and information science, points out in his paper, due to be published shortly in the journal Scientometrics, sometimes citations are merely signposts to further information or “occur where authors cite to disapprove or challenge a previous work”.
The proportion of such “perfunctory” or “negational” citations can vary between disciplines, but Professor Lin says that while there is some evidence about the extent of their use in some science subjects, data for humanities and social sciences are relatively limited.
His study looked at citation patterns in 360 articles published in Taiwanese journals in three humanities disciplines (Chinese literature, history and arts) and three social sciences (sociology, economics and psychology).
It found that for the social sciences, only economics carried a relatively high proportion of perfunctory citations (44 per cent of all the citations were perfunctory) while for the humanities subjects, history had the highest share (47 per cent).
In the other subjects, the proportion of perfunctory citations ranged from 22 per cent in Chinese literature to 39 per cent in the arts.
History also had the highest share of negational citations across all the subjects at 9 per cent but the other five subjects all had proportions of such citations below 3 per cent.
Professor Lin says that the results “surprisingly suggest that the problems of the equal contribution and positive endorsement assumptions in [the humanities and social sciences] may not be as serious as in the sciences”.
This was because other studies had suggested that the rate of perfunctory citations in some fields of physics, for example, could be as high as 60 or 70 per cent, while negational citations were at about the 10 per cent mark.
However, he adds that it should not necessarily follow that citations are an “effective impact indicator” for the humanities and social sciences.
“Other problems continue to limit and challenge the validity and applicability of citation count for measuring research impact in [the humanities and social sciences],” he says. These include the limited coverage of the subjects in bibliometric databases owing to the tendency – compared with pure science subjects – for more research in the disciplines to be published in monographs or in languages other than English.
He also says that the “significant” variation in the rate of perfunctory and negational citations in the humanities and social sciences examined in the study “also indicate limitations of using citation for measuring impact”.
For example, Professor Lin says because history and economics researchers appear to use perfunctory citations more, and historians deploy more negational citations, “plain citation counts will likely inflate the contribution of the cited works more in history and economics than in other fields”.
Find out more about THE DataPoints
THE DataPoints is designed with the forward-looking and growth-minded institution in view
