Thing 4: Emerging Metrics

Traditional measures of research outputs, covered in Thing 3, focus on citations and don’t provide the full picture of how research is distributed and discussed. New ways of measuring attention and engagement with research have been developed. Completing Thing 4 will give you an idea about the advantages and limitations of emerging metrics such as altmetrics (alternative metrics), data citation, and patent metrics. 

Getting Started

What are altmetrics? 

“Altmetrics explained in under 2 minutes” by Leeds University Library

Watch the short video “Altmetrics explained in under 2 minutes” by Leeds University Library, which introduces you to altmetrics and what they can measure. 

Read through the University of Melbourne altmetrics guide to better understand the strengths and weaknesses of altmetrics. and PlumX Metrics (Scopus) are platforms that capture mentions (news, blogs etc.) and social media attention alongside citations.

“Altmetrics – what are they good for?” by Andy Tattersall

Now settle in for a slightly longer video (around 15 minutes) and watch Andy Tattersall from the University of Sheffield give his talk “Altmetrics – what are they good for?“. If you don’t have much time, the majority of Andy’s discussion points are between 1:15min and 11:05mins

Consider: Does your institution provide access to tools or guides in support of altmetrics? What are these? What impact claims can be supported by altmetrics?  

Learn More

Let’s learn a bit about data citations and patent metrics.  

Data citations 

Data citation is the practice of giving a reference to data in the same way that researchers provide a bibliographic reference to more traditional research outputs such as journal articles. 

Exercise: Check out the dataset record “Data from: vocalisations of killer whales (Orcinus orca) in the Bremer Canyon, Western Australia” in Dryad (an international repository of data underlying scientific and medical literature). How many times has it been viewed, downloaded, and cited?  

Patent metrics 

Patents are another source that can be used to demonstrate research impact. Patents protect technical inventions and include references to work already done in the field. A researcher can therefore find out how their work is influencing inventions, or they might even have a patent or patents themselves.  

Read the University of Queensland Library’s page on patent citations for an introduction.  

Exercise: provides patent citation data.

Another free tool that allows you to explore citations of scholarly work in patent literature is PatCite by  

Exercise: Have a look at Feng Zhang’s search query in PatCite. In the left-hand menu select “Document type” under “Citing Patents”. How many patents cite Zhang’s work? How many of the patents have been granted?   

Challenge Me

Discussions around the advantages of alternative metrics include that they may appear before citations for a published article, and therefore can be early indicators of public engagement with a researcher’s work. However, one limitation is their susceptibility to gaming.  

Read the 2020 report “The pros and cons of the use of altmetrics in research assessment” for an in-depth discussion of altmetrics and their place in research evaluation. 

Consider: Based on the above report, what are some of the strategies that have been adopted to evaluate altmetrics? How well do the following alternative metrics correlate with traditional citation counts?  

  • Citations from blogs 
  • Mentions in news stories from around the world 
  • Mendeley reader counts 
  • Tweets  
  • Grey literature citations 
  • Wikipedia citations