Use of Generative Artificial Intelligence Tools for Grading

Print | PDF

Laurier full-time and contract teaching faculty currently have the option to use tools for automated grading for assessments (e.g. MyLS quizzes, scantron) where the answer is an objectively right or wrong answer, such as multiple choice, true or false, numerical, fill in the blank, or short answer. Tools are now available to automate the grading of subjective assessments through the use of generative artifical intelligence (GenAI). Subjective assessments are those where students analyze, evaluate, and create knowledge. The Academic Subcommittee of the Generative AI committee has undertaken analysis of the potential use of GenAI for the grading of subjective assessments and has developed the following recommendation.

The Academic Subcommittee of the Generative AI Committee recommends against the use of GenAI tools for subjective grading and formative feedback or assessment.

Concerns

The subcommittee has articulated the following concerns with the use of generative AI for subjective grading and formative feedback or assessment:

• Many instructors are developing new assessments to minimize the use of GenAI by students and to ensure that students are engaging in creative thought and critical analysis beyond the capacity of a GenAI tool. These types of novel, analytical, creative, and/or personal assessments cannot be adequately evaluated by a GenAI tool.
• GenAI tools are focused on content generation - they generate text, images, and other media using predictive modelling based on the content that was used to train the model. Using GenAI tools for grading moves into the realm of decision-making. “Current GenAI tools are not built for decision making; their output is unreliable, they lack explainability, and they are not able to model decisions in an explicit way to achieve outcomes.” ("When Not to Use Generative AI").
• The use of GenAI tools for subjective assessment may negatively influence course dynamics and learning in the following ways:
  • If students know an instructor is using GenAI for grading, it may decrease their motivation to do their own work without the use of GenAI;
  • An instructor’s feedback on subjective work is an integral part of the learning process – for the instructor to understand the progress that their students are making, and for the students to understand that their instructor is interested in, and engaging with, the work they are producing.

Considerations

For an individual instructor to use GenAI for subjective assessment, the grading tool must be approved through the Privacy and Security Impact Assessment process and the Provost’s Office, and the following must be taken into careful consideration: 

• A student’s work is their intellectual property. Any GenAI grading tool must not retain the student’s work for use in training the model.
• Transparency in the use of GenAI will require that students have access to a timely process for questioning or challenging any grade assigned by a GenAI tool. 
• Laurier will not have had input into, nor explicit knowledge of, the dataset that was used to train the GenAI model. 
  • Did the dataset align with Laurier’s expectations for academic rigour?
  • Did the dataset include a broad range of information from varying perspectives, frameworks, ideologies, and geographic areas?
  • Will the model amplify the inherent bias that was represented in the dataset?
    • For example, will it grade as correct those submissions that align with mainstream data and incorrect those submissions that include a critique of commonly held beliefs. Will it grade as correct submissions where a CEO is described as a light-skinned man and incorrect submissions where a CEO is described as a woman or as a dark-skinned man? Will it privilege dominant cultural norms in research and writing, but disadvantage research and writing that accounts for alternate forms of cultural capital, such as Indigenous knowledge frameworks?
  • Will the model be capable of assessing varying degrees of competency with English? Will it grade as correct “standard English,” but grade as incorrect code-meshing in academic writing?
  • Will the GenAI tool grade consistently and accurately across all assessments?
  • Will the GenAI tool be able to account for different expectations across individual faculty members, course levels, and disciplines?