Adaptive Peer Grading during Formative Assessment

Intent

Peer grading – the process of students grading some of their peer’s assignments – is an effective way to improve student learning, critical thinking, and metacognition while reducing potential faculty bias of students. However, relying on student evaluation in assessment can be difficult, as students may not understand the material well wenough to assess the skills of their peers. This play seeks to mitigate this issue by iteratively weighting the assessment of students by their own performance in the course, or the difference between their own evaluations and those of an instructor, with the idea that those who have a strong understanding of the material are able to give the most accurate peer feedback, and their feedback should have increased weight.

Problem

1. Providing timely feedback in large classes can be challenging. Faculty want to give meaningful feedback in a timely manner so that students can react and improve. But such feedback can be time consuming to provide.
2. Students often get locked into seeing one solution for a problem when multiple solutions are possible. Seeing, understanding, and “weighing” multiple solutions can be an important part of the learning process.
3. Students may not understand how to generate a solution at all. In these situations, viewing and evaluating a working solution is of benefit.
4. Developing critical analysis skills is often as important as learning the underlying content being taught in a course.
5. Students just learning material often do not have the capacity to accurately evaluate solutions produced by their peers.

Solution

Each student is provided a small number of their peers’ assignments to evaluate and provide feedback. These grades are then combined to form the final assignment grade.

This benefits the problems above by

#1 Grading and feedback is “crowd-sourced” by having lots of people grade a few assignments rather than one or two people grading all assignments.

#2 and #3 Students are, ideally, exposed to more than one possible technique for solving a problem.

#4 When a student considers a solution significantly different from the one they created, they have to apply a critical thinking lens to review how the solution works, why it works, and if the solution is better, worse, or comparable to their own solution. This allows them to develop comparison skills.

In order to address problem #5, students’ evaluations are weighted iteratively over time to attempt to give weight to those who are performing well in the class, or who have given strong feedback in the past.

Applicability

The main challenges in peer grading are the reliability of grades assigned by students which can lead to inaccurate and inconsistent feedback. This play can be used in any course where students are asked to give peer feedback on multiple instances.

How to Implement

Albano et al., 2017 discusses a technology-supported peer grading system that automates this process. But this can be implemented without such a system.

1. Randomly assign each student some number (n) of peer deliverables from the overall class (m) where n<m.
2. If desired, these deliverables can be anonymized prior to distribution to reduce bias in peer evaluations.
3. Each student performs a peer-assessment of each of the n deliverables they were assigned using an instructor provided rubric. If desired, each student will also provide feedback or comments (either free-form, or based on specific prompt questions).
4. If m students each conduct n peer-assessments than each of the m deliverables will receive n peer grades. These n grades are combined to determine the student’s grade on the assignment. Each student is provided access to the n individual grades (and any free-form feedback provided in them) as well as this final, combined grade.

While the straight forward way to combine grades in step #4 above is to average them, Albano et al., 2017 includes several mathematical formulas for weighting various peer-assessments differently. These techniques include a. Asking students to peer-assess a set of deliverables already graded by the instructor. A weighting factor is calculated by considering the degree to which the student’s assessments match the instructor’s assessments. Those who match closely are given higher weight during future assessments than those who deviate significantly. b. As time progresses, students whose assignment deliverables consistently score highly are given more weight than those whose deliverables consistently score low. The concept is that students who are performing well understand the material and the assignments and, thus, provide more accurate peer-feedback. Having said that, this score should be an iterative process which allows students to increase their scoring weight by improving their work as the semester progresses.

See Also

Source

Source: Albano, G., Capuano, N., & Pierri, A. (2017). Adaptive Peer Grading and Formative Assessment. Journal of E-Learning and Knowledge Society, 13(1). https://doi.org/10.20368/1971-8829/159

Described by: Ben Schafer (ben.schafer@uni.edu)

References

Community Discussion

Community members are free to comment on, ask questions about, share experiences, or otherwise contribute to knowledge about this play by posting comments below. See Chapter 33. Join Our Discussions for details.

• Insert a comment here.