Skip to Main Content

Citations, Altmetrics and Researcher Profiles: Research output metrics

A guide to the key tools for measuring and describing your research performance, as well as setting up and managing your research profiles

Measure the performance of your research outputs

There are a number of ways of measuring and describing the performance of individual research outputs. These can be broadly categorised as citation metrics, altmetrics and indicators of esteem. Determining where to focus your attention will depend on your discipline, the type of research output and an understanding of the likely performance strengths of an individual output. 

Find suggested performance measures for a :

Journal article

The number of citations received by an article is an indicator of the level of engagement it has achieved. To find who is citing a journal article, search citation databases such as Web of Science, Scopus, Dimensions, or Google Scholar.

  1. Do a title search for your article
  2. Look for Times Cited or Cited by
  3. Link to the list of citing sources

This is how the citation counts will appear in each database:

  • Web of Science

 

  • Scopus

  • Google Scholar

 

  • Dimensions 

Track citations​​​​​​​

Citation databases allow you to set an alert so you can be notified as soon as a new citation is added to the database. Alerts enable researchers to track where, by whom, and how often an article has been cited. Create an author alert in citation databases such as Scopus or Web of Science.

The FWCI (Field-Weighted Citation Impact) and CNCI (Category Normalised Citation Impact) both calculate the ratio of citations received relative to the expected world average, as normalised for the subject category, publication type and publication year. These metrics are considered a valuable and unbiased indicator of impact irrespective of age, subject focus or document type.

In both cases, an FWCI or CNCI of 1 indicates that the publication has been cited at the world average for similar publications:  

  • If greater than 1, then the publication has been cited more than would be expected for the world average for similar publications. For example, an FWCI or CNCI of 2, indicates that the article has received double the citations than the world average for similar publications.
  • If less than 1, this indicates that the publication has been cited less than would be expected based on the world average for similar publications.
  • If the FWCI or CNCI is absent, this indicates zero citations.

Many researchers find that the FWCI and CNCI metrics for the same article are different – sometimes significantly so. This is due to differences in database coverage between Web of Science and Scopus, affecting the number of citations that are indexed in each system.

Field Weighted Citation Index (FWCI)

The FCWI is can be found in SciVal, where it can be seen at a publication level as at a researcher level. A list of publications can generated from a profile including the FWCI. Reports can also be created displaying the FWCI for groups of publications sorted by subject.

The FWCI can also be found in Scopus. where it can only be seen in individual publication records.

UWA authors can easily also access the FWCI for their individual publications, if they are indexed by Scopus, when they log into their UWA Research Repository profile:

  • Click on the link to the relevant publication
  • choose the Metrics tab.

Category Normalised Citation Impact (CNCI)

The CNCI is calculated on Web of Science data, and is accessible using InCites (benchmarking & analytics). As well as for individual publications, Incites can be used to find the  overall CNCI for all publications by an individual researcher, which are indexed in Web of Science.

Journal Normalized Citation Impact  (JNCI) 

The JNCI is similar to the CNCI but normalizes the citation rate for the journal in which the document is published. The JNCI of a single publication is the ratio of the actual number of citing items to the average citation rate of publications in the same journal, in the same year and with the same document type. The JNCI for a set of publications is the average JNCI for each publication.

Percentile (Web of Science/ Incites (benchmarking and Analytics))

A percentile indicates how a paper has performed relative to other Web of Science-indexed publications in its field, year, and document type, and is therefore a normalized indicator. The percentile for a publication is determined by creating a citation frequency distribution for all the publications in the same year, subject category and of the same document type. The percentile is the percent of items cited less often than the item of interest, and therefore a higher percentile indicates better relative performance. 

Average percentile

Average (mean) of the percentiles for all of your Web of Science-indexed publications.

Find your article percentile

You can find the percentile for each of your Web of Science - indexed publications in Web of Science

  • Use the Researchers tab to perform a Name Search for yourself.
  • Select your name from the results list.
  • The Author Impact Beamplot Summary shows your overall citation citation percentile median in Web of Science.
  • Click on the View full beamplot link in your Web of Science record to see the percentiles for your individual articles.
  • You can also run a comprehensive analysis of your Web of Science - indexed publications in Incites (Benchmarking and Analytics)

To find highly cited articles on a particular topic, run a keyword search in citation databases such as Scopus or Web of Science.  Sort on the search result by "Cited by (highest)" citations (Scopus) or  "Sort by: Times cited: Highest to lowest" (Web of Science), so the most highly cited work is at the top of your search results.

This Scopus tutorial demonstrates the use of Scopus article metrics.

Altmetrics measure how many times a research output has been shared, mentioned or downloaded from online sources such as social media sites, blogs, mainstream media and reference managers. Find out more about altmetrics and locating yours here.

The Altmetrics tools covered in this guide collect social and traditional media mentions, but there are some other sources you might like to check as well. 

  • The Conversation - If you have written for The Conversation you can log in to your profile to see a dashboard detailing metrics and engagement with your article(s) that may be used to demonstrate impact of your research.
  • Google News  - Try searching for your name or research output in Google News to see if it is appearing in any media.
  • Capital Monitor - Search for your publication or your name in Capital Monitor to locate mentions in government documents and proceedings, for example references to your report in the Hansard. These would be excellent evidence of the impact of your work.

Policy citations are the number of times your research outputs have been mentioned in policy documents from government bodies, professional organizations or bodies, or non-government organizations (NGO). They are a good way of demonstrating how your research has influenced policy or practice in a particular field and support the development of your impact narrative beyond citations. 

There are a few tools you can use to track policy citations, all of which use data from Overton, the world’s largest searchable index of policy documents, guidelines, think tank publications and working papers. Overton collects data globally from over 180 countries and in many different languages. 

  • Scival identifies and counts citations that research papers have received from policy documents. To access a policy report on yourself, follow these steps:
    1. Access SciVal through OneSearch. SciVal requires users to create a personal account. Please do so using your UWA email address to ensure full access to subscribed content.
    2. Go to the Impact Module.
    3. Click on Researchers and Groups in the Entity Panel (left hand side of screen) and search for your name. Note that if your name does not appear, you will need to Import a new Researcher
    4. Select your date range.

  • PlumX reports on policy citations. You’ll be able to see policy citations wherever you find PlumX Metrics, including Scopus, Science Direct, Digital Commons, SSRN, Engineering Village and Elsevier journal websites. The PlumPrint is also embedded at the research output level in the UWA Profiles and Research Repository. Search for a particular publication or browse your 'Research Output' list in the Repository and if available, hover over the PlumPrint to see whether any policy citations have been picked up. Click the  'see details' link for a full report and information on the source of the citation. 

  • Sage Policy Profile - A free tool for individual researchers and academics. Set up a profile to track and visualise how your research has been cited in policy from governments, think tanks, and policymakers across the world. Note, this tool uses your ORCID ID to identify your publications so make sure your ORCID profile is up-to-date. If you haven't already, connect your ORCID iD to your UWA profile for automatic updates to your ORCID profile when new outputs are added to the Repository.

Patents often cite research papers. If your research has been cited in a patent, this shows the connection between your work and industry or commercial activity.

Several tools collect Patent citations and information: 

  • SciVal identifies and counts citations that research papers have received from patents, and links to both the citing patents and cited articles
    1. Access SciVal through OneSearch. SciVal requires users to create a personal account. Please do so using your UWA email address to ensure full access to subscribed content.
    2. Go to the Impact Module.
    3. Click on Researchers and Groups in the Entity Panel (left hand side of screen) and search for your name. Note that if your name does not appear, you will need to Import a new Researcher
    4. SciVal will default to a 'Policy' report. Select 'Patent' instead from the top menu
    5. Select your date range.

  • Lens is a database for articles and patents. Search by patents, scholarly works, profiles and more.
  • Both PlumX and Altmetric track patent citations. Access via the PlumPrint and Altermetrics doughnut embedded at the research output level in the UWA Profiles and Research Repository. Search for a particular publication or browse your 'Research Output' list in the Repository and if available, hover over the icons to see whether any patent citations have been picked up. Click the  'see details' link for a full report and information on the source of the citation. 

  • Google Patents includes more than 120 million patent publications. A list of citing patents is included in the full text of the patent.

Book/book chapter

There is no one measure that captures the impact of books/book chapters across the board, and which measures are effective may depend on the type, discipline, content, format, publisher etc. You may find you need to use a combination of traditional metrics, altmetrics and other measures of esteem to fully reflect the value of your book in research, teaching and other professional activities. Various recommended strategies are detailed in the subsequent tabs.

Books and book chapters are increasingly indexed in databases, so it is worth checking to see if you can find citations metrics. Resources to check:

See the Clarivate Master Book List for publishers that are indexed.

Photo of a Google Scholar book record with the cited by list highlighted

 

If your book/chapter only has a few citations, consider looking more closely at who has cited your work. Are they a high profile researcher in your area? Have they reflected favourably on your findings? 

 

Book reviews can be a good source of examples of impact in a field, especially if they’re from notable people in the field. To find more formal, ‘reputable’ reviews (meaning published reviews, rather than those written by members of the public), search in the following locations:

1. OneSearch indexes a vast array of content, including reviews. Search by book title and use the ‘Review’ filter to limit your results:

 

 

2. Established cultural magazines and publications are also good sources of non-scholarly but still reputable book reviews:

3. Quantity can also be a useful measure though (i.e. my book received over 200 reviews on Amazon). Check book review sites like Goodreads, Amazon and Google Books

There are many other ways to measure the quality and impact of your books and book chapters. 

Libraries holding your book or the book you contributed to in their collections is another sign of impact. Useful metrics may be:

  • The total number of libraries that hold your book
  • Whether prestigious libraries hold the book, i.e., Bodleian Libraries, The Library of Congress, The National Library of Australia etc. Look out for significant libraries for your particular area/topic. 

For holdings in international libraries, search WorldCat. Look out for different editions, which may have separate records!

Image shows an example record in WorldCat. A line under a section titled Availability reads Libraries worldwide that own item is highlighte and provide a total count and links to a list of the libraries that hold the book.

For holdings in Australian Libraries, search the National Library tool, Trove, using their Books and Libraries filter. Look out for different editions, which may have separate records!

Here are other ways books can be received and used that can indicate impact and quality:

  • Altmetrics (see the Alternative metrics tab), though note that altmetrics tools use ISBNs to track and collect information on books, and when an ISBN cannot be found, may search for different identifiers which can be for different editions and may result in metrics that aren't directly measuring the specific edition of the book
  • If a book was published by a prestigious/reputable publisher – for example, it is an academic/university publisher? Is it on the WASS-SENSE book publisher ranking list?
  • If the book resulted in publicity and events- Were you interviewed during the book release or invited to a signing or to do a talk?
  • If the book has received prizes/awards
  • Ongoing demand - If the book was republished, published in electronic format, or translated into other languages  
  • If the book has been used as a textbook by a school or uni
  • Is it an Open Educational resource?

Report

Reports are rarely indexed in the large citation databases, Scopus and Web of Science. The UWA Profiles and Research Repository is indexed by Google Scholar which not only allows UWA authors to track citations within Google Scholar but increases the discoverability of UWA  publications and the opportunity for citation. We recommend that UWA authors:

1. Add their reports to the UWA Repository, including a PDF copy if their publisher agreement allows, or the link to its online location. 

Report index in Google Scholar with 20 citations

Google Scholar will only index publication records which include an abstract, and may preferentially index papers with full text available (Google Scholar Help, 2022).

If the full text of your report can be made available through the UWA Repository, the number of times the report has been downloaded will be displayed in the Repository which can be a useful measure of engagement.

2. Request a Digital Object Identifier (DOI) from the Library as soon as possible so that "altmetrics" such as mentions of the report on social media can be tracked accurately using the DOI. These will appear next to your report in your Repository profile as colourful visualisations from PlumX and Altmetric.com. Email staffsupport-lib@uwa.edu.au to request a DOI.

Search for your report in Capital Monitor to locate mentions in government documents and proceedings, for example references to your report in the Hansard. These would be excellent evidence of the impact of your work.

Non-traditional Research Outputs (NTROs)

Non-traditional research outputs (NTROs) are scholarly works or creative endeavours that go beyond traditional academic publications, such as journal articles and books. NTROs are typically produced by researchers in disciplines where non-textual forms of expression and dissemination are valued, such as the arts, design, music, film, and digital media. 

Measuring NTROs can be challenging due to their diverse nature and formats and that they are typically not indexed by academic databases and therefore do not receive citation counts. Altmetrics tools have made tracking easier for some NTRO outputs, but rely on the work having a persistent identifier (PID) which is used in all communcations around that output. Read more about PIDs for publications and how to apply for them in our How to Publish and Disseminate Research guide.

Other possible measures of quality and esteem that can help when building a narrative around engagement and impact are listed below by output type.

Measures of quality, esteem and engagement include:

  • Awards and prizes
  • Reviews
  • Commissions
  • Collaborations
  • Exhibitions
  • Impacts such as sustainability, occupant health, liveable cities

Measures of quality, esteem and engagement include:

  • Reviews
  • Editions and translations
  • Prestige of the publisher or publication
  • Awards, prizes and  nominations
  • Best seller listings
  • Sales or download figures from the publisher
  • World-wide and Australian library holdings
  • Esteem of the publisher and/or editor
  • Media mentions, including traditional and social media

Measures of esteem, quality and engagement include:

  • Invitations to curate
  • Invitations to exhibit
  • Commissions
  • Prestige of the venue
  • Reviews
  • Visitor numbers
  • Website visits
  • Sales data
  • Catalogue sales and downloads
  • Media and news mentions, including traditional and social media

Measures of esteem, quality and engagement include:

  • Invitations to perform
  • Commissions
  • Audience numbers
  • Prestige of the venue
  • Reviews
  • Media and news mentions, including traditional and social media

Measures of quality, esteem and engagement include:

  • Prestige of publisher or publication
  • Performance of the work and prestige of the performer
  • Reviews
  • Awards, prizes and nominations
  • Sales or download figures 
  • Commissions and grants
  • Library holdings
  • Information about accompanying work (e.g. performance; production of film soundtrack)
  • Media mentions, including traditional and social media

Measures of quality, esteem and engagement include:

  • Reviews
  • Sales, downloads, streaming, playlist data
  • Prestige of the record label
  • Prestige of the record producer
  • Media mentions, including traditional and social media
  • Inclusion in compilations

Measures of quality, esteem and engagement include:

  • Commissions and grants
  • Invitations to exhibit
  • Inclusion in exhibition catalogues
  • Collaborations
  • Awards and prizes
  • Sales data
  • Media mentions, including traditional and social media
  • Artist-in-residence programs (public and community engagement)

Data

Just as for other research outputs, use of your data by other researchers can be described when discussing your research performance.

Data citation is an emerging practice. "Data citation refers to the practice of providing a reference to data in the same way as researchers routinely provide a bibliographic reference to outputs such as journal articles, reports and conference papers. Citing data is increasingly being recognised as one of the key practices leading to recognition of data as a primary research output."

Australian National Data Service (ANDS) data citation support material

Resources

Halevi, G., Nicolas, B. & Bar-Ilan, J. (2016). The Complexity of Measuring the Impact of Books. Publishing Research Quarterly 32, 187–200. https://doi.org/10.1007/s12109-016-9464-5 

Mills, K., & Croker, K. (2020). Measuring the research quality of Humanities and Social Sciences publications: An analysis of the effectiveness of traditional measures and altmetricshttps://doi.org/10.26182/v063-r898

CONTENT LICENCE

 Except for logos, Canva designs or where otherwise indicated, content in this guide is licensed under a Creative Commons Attribution-ShareAlike 4.0 International Licence.