Bibliometric methods are widely used to assess research based on quantitative measures. Bibliometrics is one way to demonstrate influence and impact of research based on measures such as citation counts and h-index. Thing 1 introduces you to basic bibliometric methods, and the strengths and limitations of bibliometrics.
Getting Started
What is bibliometrics?
The Oxford English Dictionary defines bibliometrics as, “The branch of library science concerned with the application of mathematical and statistical analysis to bibliography; the statistical analysis of books, articles, or other publications.”
In the past, bibliometric methods were commonly used to assist with collection management. There was a range of bibliometric models for journal productivity which researchers stated could be used to rationally and economically plan and organise untidy information systems and library services. If you are interested and you have access to the JSTOR database, you can read more about this use of bibliometrics in “Bibliometric models for journal productivity”.
Today, bibliometrics or bibliometric methods are predominantly used in quantitative research assessment exercises, in which the impact of a research field, a researcher or group of researchers, a particular journal article or journal are explored and sometimes compared.
Author-level metrics quantify a researcher’s impact by analysing citations to their publications. They are used in grant applications, as well as in tenure, promotion, and performance reviews.
Author-level metrics include a researcher’s total number of publications, the number of times those publications have been cited, and a researcher’s h-index.
Read the brief introduction to author-level metrics from Hunt Library at the Embry-Riddle Aeronautical University in Florida.
Watch the video “H-index explained” by Curtin University Library or read the h-index page about this metric on the Metrics Toolkit website to learn how to calculate an h-index.
Exercise: Go to Professor Peter Doherty’s Google Scholar profile page. What is his h-index based on Google Scholar data? How does the h-index disadvantage early career researchers?
Learn More
Watch the video “Bibliometrics in under 2 minutes” by Leeds University Library for a quick introduction to bibliometrics.
Author-level metrics are covered in “Getting Started” of this Thing. In “Learn More” we will focus on article-level and journal-level metrics.
Article-level metrics measure the usage and impact of individual published articles, such as the number of times an article has been cited.
Exercise: Explore the publication metrics of the article “Evolution of facial muscle anatomy in dogs”. How many times has this article been cited? Under “Publication metrics”, you will find the “Recent citations” metric. What is the advantage of knowing how many citations are recent?
Journal-level metrics: The Journal Impact Factor (JIF) is a metric commonly used to demonstrate the importance and prestige of a journal. It is a Clarivate Web of Science metric and can be looked up using Journal Citation Reports (JCR), a subscription-based database. A journal’s impact factor is calculated by dividing the number of times articles were cited by the number of citable articles in a journal for a two-year period.
Scimago Journal Rank (SJR) is another journal-level metric and expresses the average number of weighted citations received in a selected year by the documents published in that journal in the previous three years.
Exercise: Go to the Scimago Journal Rank website. Filter the results by subject area and region to find out the top ranked journal titles.
Some common sources of bibliometric data are Web of Science, Scopus, and Google Scholar. “Thing 3: Traditional Metrics” will explore these tools further.
Challenge Me
Some of the well-known shortfalls of bibliometrics are the scope of the tools, discipline variation, publication languages, unpublished works, and lack of distinction between positive and negative impact.
Learn about some of these shortfalls in more depth by reading “Limitations of bibliometrics” from York University Libraries, and “Bibliometric indicators: opportunities and limits“.
Consider: Google Scholar often records a higher number of citations than other tools such as Web of Science. Why is that the case? Why is it unfair to compare the h-index of researchers in different disciplines? How can early career researchers demonstrate their impact if their h-index is low?
Scite.ai is a tool that helps users to evaluate if the citing articles are supporting the cited paper, just mentioning the cited paper or contradicting the cited paper.
Exercise: Go to https://scite.ai/ and search using the DOI 10.1371/journal.pone.0111913. Explore the supporting and contradicting citations.
Consider: How can researchers use tools such as Scite.ai to demonstrate their impact?
