Students’ engagement in online peer review processes is affected if students can see how their performance is ranked compared to other students’. This is the conclusion of a recent study by Assistant Professor Pantelis Papadopoulos.
Pantelis Papadopoulos at the Centre for Teaching Development and Digital Media does research in peer review conducted via a web-based learning environment. He recently carried out a study with a fellow researcher to investigate whether information related to student activity could affect performance, motivation, and engagement in an online peer review process.
The study showed that students’ motivation to participate in peer review was likely to increase, if the students could see how their level of activity ranked compared to other students. However, for some students, ranking information had the opposite effect and was used as an excuse to disengage. These students believed that a high ranking meant that they did not have to put in any additional effort. Thus, ranking information could be a double-edged sword.
No effect from usage data
Pantelis Papadopoulos and his colleague also looked into whether students’ access to usage data would affect their engagement in the process. It did not. Usage data refers to students’ actions in the learning online environment: clicks, submissions, visits, etc.
The study was carried out among Software Engineering students, grouped into three study conditions: a group with access only to usage data, a group with access to both usage data and ranking information, and a control group.
A web-based learning environment developed by Dr. Papadopoulos called eCASE was used to carry out the peer review activities. eCASE monitored and recorded every student action within the system, and the study’s findings are based on data analysis from the eCASE database, written tests, and a questionnaire recording students’ opinions on the activity.
The study was the last in a series of five on online peer review by Pantelis Papadopoulos and several different co-authors. Two of them are focusing on general issues around collaborative learning and peer review, while the rest three form a series of studies on “free-selection” peer review protocol. “Free selection” implies that students are free to select which peer work they want to review.
Read more about the studies:
- Papadopoulos, P. M. & Lagkas, T. D. (2014). Usage Data and Group Rankings In Peer Review Settings: A Case Study on Students’ Behavior and Performance. In Proceedings of the 14th International Conference on Advanced Learning Technologies – ICALT 2014, Athens, Greece (accepted). Slides from a conference presentation of the study.
More on the “free-selection” review process:
- Papadopoulos, P. M., Lagkas, T.D., & Demetriadis, S. N. (2012). How to Improve the Peer Review Method: Free-Selection vs Assigned-Pair Protocol Evaluated in a Computer Networking Course. Computers & Education, 59, 182-195.
- Papadopoulos, P. M., & Cerone, A. (2012). The Role of Peer Review in Supporting the Sustainability of Technology-Enhanced Learning Environments. Information Technology and Open Source: Applications for Education, Innovation, and Sustainability, Lecture Notes in Computer Science 2014, 89-103.
More on other issues of peer review:
- Papadopoulos, P. M., Demetriadis, S. N., & Weinberger, A. (2013). “Make It Explicit!”: Improving Collaboration through Increase of Script Coercion. Journal of Computer Assisted Learning, 29(4), 383-398.
- Papadopoulos, P. M., Demetriadis, S. N., & Stamelos, I. G. (2009). Analyzing the Role of Students’ Self-Organization in Scripted Collaboration: A Case Study. Proceedings of the 8th International Conference on Computer Supported Collaborative Learning – CSCL 2009 (pp. 487-496), Rhodes, Greece.