The Report by Class Central

Disclosure: Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

News

The Problems of Peer Assessments at Scale: Cheating, Delays, Inconsistencies

While peer-assessed assignments can be a great learning tool, the credibility of online courses will not endure unless providers take strong action against cheating.

A common problem of peer grading: the lack of reviewers
A common problem of peer grading: the lack of reviewers

Recently, I took Coursera’s Social Psychology with the Class Central Study Group. The course includes four peer-assessed assignments that are scored by other students taking the course online.

During the Study Group’s bi-weekly meeting, peer assessments were brought up, and we shared some of the experiences we’d had with them, in this course and others. They can be quite frustrating. Here are some of the problems we’ve encountered.

Blank Submissions

While taking a course on academic note-taking, @manoel was asked to score several assignments that, surprisingly, were blank. Why would anyone submit a blank assignment?

One reason is that some learners will score submissions without even looking at them. For your own score to be released, you must first score other learners’ submissions. So some learners go through the motions just for the sake of getting their own score. As a result, even a blank copy might receive a passing grade.

Another reason is that once a learner has submitted his assignment, he has access to other students’ assignments. He can then copy one of these and resubmit it as his own. Providers like Coursera allow learners to revise and resubmit assignments, which opens the door to widespread plagiarism.

Class Central comment screenshot
Comment on The Second Year of The MOOC: A Review of MOOC Stats and Trends in 2020

Plagiarism

Sometimes, submitting a blank assignment isn’t even required to access other students’ submissions. A simple Google search does the trick. For instance, I found a website with solutions for all the peer-graded assignments of Coursera’s Social Psychology course we’d just taken. The website also has solutions for other courses, including courses from other platforms.

A few keywords, often taken from the assignment prompt itself, is usually all that’s needed to find solutions online. Some popular courses even have complete walkthroughs on YouTube.

While providers usually ask students to uphold an Honor Code, sometimes by checking a box before submitting an assignment, it doesn’t seem to do much to deter motivated cheaters.

Coursera Screenshot

Many learners who spend considerable time and effort on their assignments feel aggrieved that cheaters appear to go unpunished.

Some instructors mention the risk of plagiarism and ask reviewers to actively help catch cheaters by flagging suspect assignments. One way to find if an assignment is plagiarized is to Google a few sentences from an assignment and see if it returns some matches.

Organized Cheating

According to a comment left on Class Central by Coursera mentor Kristy Read, students created WhatsApp groups for some courses and gave each other good grades. Kristy and others feel that the issue mushroomed with Coursera’s free certificates offer during the COVID-19 pandemic.

On YouTube, I also found videos that show how to cheat on popular online courses. These invite learners to join groups on various messaging platforms, such as WhatsApp and Telegram, to progress through courses together, share answers, and give each other passing grades.

Recently, I came across a course discussion in which a mentor said a learner’s assignment was plagiarised. The learner protested, claiming that it was their own work.

Cheating on assignments is not new. The issue was already brought up in 2012 during The Year of the MOOC, when Coursera and other large MOOC providers began to operate. While some measures have been taken, such as disabling the ability to copy/paste other student’s assignments, it appears that over the last nine years, online course providers have made little progress in preventing at-scale cheating on their platforms.

Delays

Delays occur when no one is available to review your assignment. Some course forums are full of pleas to “Please assess my assignment”. In fact, requesting reviews on the forum is one of Coursera’s own recommendations, but not a very useful one considering many course forums are deserted.

In 2020, this issue eased somewhat because of increased learner numbers, but as the surge of interest in online education abates, the requests are reappearing.

In a course I took in 2017, several of my assignments were assessed by the same learners, and I assessed theirs. When few learners are in a particular session of a course, it can sometimes take weeks for results to come back. @ruima waited more than a month for her final assessment in Social Psychology to be reviewed. My wait was shorter — 10 days.

A similar issue can occur when there are no assignments available to review. I have sometimes had to check the course each day for several days before I could review enough assignments to receive my own course grade.

Inconsistent Reviews

Most courses with peer reviews ask learners to review 3 to 5 submissions, then results at the extreme ends are ignored. This helps reduce bias, but it can be confusing for the learner.

Some people will give full points just because the assignment was attempted. Alternatively, some people will give 0 points if they disagree with the assignment’s opinions, even if the rubric asks peers to assess the writing, not the point of view. As a result, some learners who put time and effort into their assignment might get a low grade, while some who merely attempted the assignment might get a passing one.

With participants worldwide, learners have varying command of the language (usually English), which can affect writing assignments as well as reading and commenting on others. Feedback may be minimal, which can be particularly frustrating if points have been deducted without explanation.

Coursera Screenshot
Assessors who did not read the instructions.

Sometimes, such as when there’s a lack of reviewers, your assignment score might end up being determined by a single person. In those cases, whether you get a fair score will entirely depend on that person, which may lead to a lot of frustration.

If you are unhappy with your results, some providers allow you to resubmit your assignment.

Coursera Screenshot

Advantages of peer assessments

With so many problems with peer assessments, you might wonder if there is anything good about them. Happily, there are some advantages. By researching, planning, and writing your assignment, you will remember more about the subject. Sometimes, you will find some interesting insights as you assess your peers’ submissions. But don’t copy other assignments and resubmit them as your own!

Your peers might point out issues with your writing that you were unaware of, such as a poorly written essay or arguments that are hard to understand. Use this feedback to learn and improve.

While peer-assessed assignments can be a great learning tool, the credibility of online courses will not endure unless providers take strong action against cheating.

Pat Bowden Profile Image

Pat Bowden

Online learning specialist, still learning after 100+ MOOCs completed since 2012. Class Central customer support and help since 2018. I am keen to help others make the most of online learning, so I set up a website:  www.onlinelearningsuccess.org

Comments 6

  1. Avatar

    Ronny De Winter

    “Motivated cheaters” = contradictio in terminis 🙂
    It went worse over the years. In the early days of moocs the emphasis was more on learning and less on getting certificates.

    Reply
    • Avatar

      Steve

      Sometimes the motivation is to pass a F/T course (where fees are charged) that use MOOCs as partial credit. The only test appears to be producing a certificate, not demonstrating knowledge of the subject.

      Reply
  2. Avatar

    Ronny De Winter

    Let’s hope this article triggers more the course platforms & providers than the potential cheaters!

    Reply
  3. Avatar

    Robert Urbaniak

    Hi Pat

    We sincerely appreciate your engagement with the platform and efforts to make online learning better for everyone. At a time when the world is mostly learning online, we all have the responsibility to make the experience better. We are inspired by our learner community’s passion and commitment to developing new skills by applying themselves across assessments and assignments. However, there are some instances that keep learners from reaching their full potential. Such behavior must be curbed.

    We’ve taken note of your observations and excited to share with you and the community that we’re already working on solutions to help address them. Stay tuned.

    Thanks
    Robert Urbaniak
    Sr. Product Manager, Assessments
    Coursera

    Reply
    • Avatar

      Pat Bowden

      Thanks for the feedback, Robert! I’m looking forward to seeing improvements in the peer assessment process.

      Reply
  4. Avatar

    Dennis

    The issue with delays are too few people take a specialization or course. And peer grading requires three people to grade! I took one on job hunting and only three of students taking the space of one month to grade each other. I am still short of one grader for capstone project. Longer wait entails higher costs to be paid out.

    Coursera need to market low enrolled courses aggressively, or change to one person grading or lower the price tag or make some free with certs for more students to take.

    Thanks.

    Reply

Leave a reply

Your email address will not be published. All comments go through moderation, so your comment won't display immediately.

This site uses Akismet to reduce spam. Learn how your comment data is processed.