Whilst Gen AI can be a useful tool to support your studies, it is not without limitations. There are many types of Gen AI and each comes with it's own advantages and disadvantages. Gen AI is a tool like any other, it is important to select the right tool for the job. By evaluating the outputs produced you can identify inaccuracies, misinformation, disinformation, and bias.
Before you begin:
This page of the guide aims to help you:
Applying similar principles that you would when assessing any other source of information or tool is a good place to start. There are a number of tests that can help you evaluate Gen AI.
Tests to evaluate Gen AI:
Tests to evaluate outputs:
The ROBOT test is comprehensive, and was designed to help you think critically about AI. It was created by Amanda Wheatley and Sandy Hervieux from McGill University. You can find more resources on the LibrAIry blog.
Choose a Gen AI tool and try to answer the questions in each section of the test.
Reliability
Tips: Try to find information about the generative AI tool on the website of the company/organisation/individual who made it. Is the information about the tool easy to locate and is it clear? Additionally consider if the generative AI tool has been designed to produce reliable information or content, try to fact check the output.
Objective
Tip: Consider the purpose the AI tool was made for, does this align with how you are using it? ChatGPT was made by a company called OpenAI, what is the purpose of this company, what is their mission? What did they intend their users to do with the tool they created?
Bias
Tips: Read the Ethical Considerations page of this guide for more information on issues and ethical considerations.
Search the library's catalogue OneSearch with the follow search string, and use filters to limit the results:
Ownership
Tips: Consider the motivations of the owner, how might this impact the the technology?
Type
Tips: These questions may be applied to AI in the broader sense, as well as more specifically to generative AI. For example, if you want to use AI to search for information, it might be best to find a type of generative AI that has uses a search engine and can give you links to the resources used in the content it generates.
Attribution:
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
To cite in APA: Hervieux, S. & Wheatley, A. (2020). The ROBOT test [Evaluation tool]. The LibrAIry. https://thelibrairy.wordpress.com/2020/03/11/the-robot-test
The EVERY tool provides guidance on how to use AI responsibly with 5 steps to follow.
Is it CRAAP? can be used to evaluate information resources. Similar principles may be applied to Gen AI outputs. Remember always fact check and follow up on topics by finding academic sources for your assignments.
Currency
Relevance
Accuracy
Authority
Purpose
Except for logos, Canva designs, AI generated images or where otherwise indicated, content in this guide is licensed under a Creative Commons Attribution-ShareAlike 4.0 International Licence.