Traditional measures of research outputs, covered in Thing 3, focus on citations and don’t provide the full picture of how research is distributed and discussed. New ways of measuring attention and engagement with research have been developed. Completing Thing 4 will give you an idea about the advantages and limitations of emerging metrics such as altmetrics (alternative metrics), data citation, and patent metrics.
Getting Started
What are altmetrics?
Watch the short video “Altmetrics explained in under 2 minutes” by Leeds University Library, which introduces you to altmetrics and what they can measure.
Read through the University of Melbourne altmetrics guide to better understand the strengths and weaknesses of altmetrics.
Altmetric.com and PlumX Metrics (Scopus) are platforms that capture mentions (news, blogs etc.) and social media attention alongside citations.
Now settle in for a slightly longer video (around 15 minutes) and watch Andy Tattersall from the University of Sheffield give his talk “Altmetrics – what are they good for?“. If you don’t have much time, the majority of Andy’s discussion points are between 1:15min and 11:05mins.
Consider: Does your institution provide access to tools or guides in support of altmetrics? What are these? What impact claims can be supported by altmetrics?
Learn More
Let’s learn a bit about data citations and patent metrics.
Data citations
Data citation is the practice of giving a reference to data in the same way that researchers provide a bibliographic reference to more traditional research outputs such as journal articles.
- The Metrics Toolkit website gives a clear definition of data citations.
- Read the Australian National Data Service’s (ANDS) pages on Data Citation and Software Citation to find out how data and software citation works.
- Clarivate’s Data Citation Index is a proprietary tool that tracks data and software citations.
Exercise: Check out the dataset record “Data from: vocalisations of killer whales (Orcinus orca) in the Bremer Canyon, Western Australia” in Dryad (an international repository of data underlying scientific and medical literature). How many times has it been viewed, downloaded, and cited?
Patent metrics
Patents are another source that can be used to demonstrate research impact. Patents protect technical inventions and include references to work already done in the field. A researcher can therefore find out how their work is influencing inventions, or they might even have a patent or patents themselves.
Read the University of Queensland Library’s page on patent citations for an introduction.
Exercise: Altmetric.com provides patent citation data.
- Go to the article “Preliminary investigations into Tris(2,2’-bipyridyl) Ruthenium (III) as a chemiluminescent reagent for the detection of 3,6‐Diacetylmorphine (Heroin) on surfaces”.
- On the right-hand side under “Metrics”, click on the “Am score” icon. This will take you to the detail page for this article.
- Click on the “Patents” tab to view the patents that have cited this paper.
Another free tool that allows you to explore citations of scholarly work in patent literature is PatCite by Lens.org.
Exercise: Have a look at Feng Zhang’s search query in PatCite. In the left-hand menu select “Document type” under “Citing Patents”. How many patents cite Zhang’s work? How many of the patents have been granted?
Challenge Me
Discussions around the advantages of alternative metrics include that they may appear before citations for a published article, and therefore can be early indicators of public engagement with a researcher’s work. However, one limitation is their susceptibility to gaming.
Read the 2020 report “The pros and cons of the use of altmetrics in research assessment” for an in-depth discussion of altmetrics and their place in research evaluation.
Consider: Based on the above report, what are some of the strategies that have been adopted to evaluate altmetrics? How well do the following alternative metrics correlate with traditional citation counts?
- Citations from blogs
- Mentions in news stories from around the world
- Mendeley reader counts
- Tweets
- Grey literature citations
- Wikipedia citations
