News & Views item - March 2012



Two Comments Note Caveats Regarding Citation Metrics. (March 26, 2012)

A couple of "correspondence" in the March 1, 2012 issue of Nature point out that 1) "Citations: not all measures are equal" and 2) "Citations: results differ by data bases".


Scott L. Hooper
Ohio University, Athens, USA
Nature 483, 36 (01 March 2012) doi:10.1038/483036c
Carles Alcaraz & Sofia Morais
Institute of Agrifood Research and Technology (IRTA), Sant Carles de la Rąpita, Spain
Nature 483, 36 (01 March 2012) doi:10.1038/483036d

Citations: not all measures are equal
The scientific community needs to be aware of the limitations of Google Scholar's personalized citation reports. Clicking on 'My citations' on the site may offer a nice ego boost, but I would not recommend using the reports for decisions that could affect careers.

Google Scholar overestimates the number of citable articles (in comparison with formal citation services such as Scopus and Thomson Reuters) because of the automated way it collects data, including 'grey' literature such as theses. For my own publications, for example, Google Scholar yields 38% more citations and boosts the h-index by 26%.

A citation report for one of my articles revealed that Google Scholar had counted as independent citations four web pages on which authors had posted copies of their articles, plus one listing only an article title; and one to a paper in which my name didn't appear. Personalized searches by my colleagues exposed comparable errors.

These drawbacks might also allow unscrupulous individuals to use such tactics to inflate their citation reports, particularly as independent vetting is blocked by password access.

Citations: results differ by database
Databases such as Thomson Reuters' ISI Web of Science, Scopus, Google Scholar and Microsoft's Academic Search allow authors to compute their own citation statistics, but they yield inconsistent results.

The discrepancies come from differences in information sources and in temporal citation coverage. Web of Science and Scopus, for example, provide citation data only for their indexed journals, giving different coverage for the number of journals, precursor articles and fields of academic research — often with regional biases (such as European versus US sources). Google Scholar includes all journals (indexed, free access and popular science), conference proceedings, books, theses, reports, local press and electronic sources — all subject to variable degrees of control and scrutiny.

A debate is crucial on how these tracking tools compare and should be used, given that their indiscriminate usage has potentially negative implications for academic careers.