The Report by Class Central

Disclosure: Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

News

The Problems of Peer Assessments at Scale: Cheating, Delays, Inconsistencies

While peer-assessed assignments can be a great learning tool, the credibility of online courses will not endure unless providers take strong action against cheating.

A common problem of peer grading: the lack of reviewers
A common problem of peer grading: the lack of reviewers

Recently, I took Coursera’s Social Psychology with the Class Central Study Group. The course includes four peer-assessed assignments that are scored by other students taking the course online.

During the Study Group’s bi-weekly meeting, peer assessments were brought up, and we shared some of the experiences we’d had with them, in this course and others. They can be quite frustrating. Here are some of the problems we’ve encountered.

Blank Submissions

While taking a course on academic note-taking, @manoel was asked to score several assignments that, surprisingly, were blank. Why would anyone submit a blank assignment?

One reason is that some learners will score submissions without even looking at them. For your own score to be released, you must first score other learners’ submissions. So some learners go through the motions just for the sake of getting their own score. As a result, even a blank copy might receive a passing grade.

Another reason is that once a learner has submitted his assignment, he has access to other students’ assignments. He can then copy one of these and resubmit it as his own. Providers like Coursera allow learners to revise and resubmit assignments, which opens the door to widespread plagiarism.

Class Central comment screenshot
Comment on The Second Year of The MOOC: A Review of MOOC Stats and Trends in 2020

Plagiarism

Sometimes, submitting a blank assignment isn’t even required to access other students’ submissions. A simple Google search does the trick. For instance, I found a website with solutions for all the peer-graded assignments of Coursera’s Social Psychology course we’d just taken. The website also has solutions for other courses, including courses from other platforms.

A few keywords, often taken from the assignment prompt itself, is usually all that’s needed to find solutions online. Some popular courses even have complete walkthroughs on YouTube.

While providers usually ask students to uphold an Honor Code, sometimes by checking a box before submitting an assignment, it doesn’t seem to do much to deter motivated cheaters.

Coursera Screenshot

Many learners who spend considerable time and effort on their assignments feel aggrieved that cheaters appear to go unpunished.

Some instructors mention the risk of plagiarism and ask reviewers to actively help catch cheaters by flagging suspect assignments. One way to find if an assignment is plagiarized is to Google a few sentences from an assignment and see if it returns some matches.

Organized Cheating

According to a comment left on Class Central by Coursera mentor Kristy Read, students created WhatsApp groups for some courses and gave each other good grades. Kristy and others feel that the issue mushroomed with Coursera’s free certificates offer during the COVID-19 pandemic.

On YouTube, I also found videos that show how to cheat on popular online courses. These invite learners to join groups on various messaging platforms, such as WhatsApp and Telegram, to progress through courses together, share answers, and give each other passing grades.

Recently, I came across a course discussion in which a mentor said a learner’s assignment was plagiarised. The learner protested, claiming that it was their own work.

Cheating on assignments is not new. The issue was already brought up in 2012 during The Year of the MOOC, when Coursera and other large MOOC providers began to operate. While some measures have been taken, such as disabling the ability to copy/paste other students’ assignments, it appears that over the last nine years, online course providers have made little progress in preventing at-scale cheating on their platforms.

Delays

Delays occur when no one is available to review your assignment. Some course forums are full of pleas to “Please assess my assignment”. In fact, requesting reviews on the forum is one of Coursera’s own recommendations, but not a very useful one considering many course forums are deserted.

In 2020, this issue eased somewhat because of increased learner numbers, but as the surge of interest in online education abates, the requests are reappearing.

In a course I took in 2017, several of my assignments were assessed by the same learners, and I assessed theirs. When few learners are in a particular session of a course, it can sometimes take weeks for results to come back. @ruima waited more than a month for her final assessment in Social Psychology to be reviewed. My wait was shorter — 10 days.

A similar issue can occur when there are no assignments available to review. I have sometimes had to check the course each day for several days before I could review enough assignments to receive my own course grade.

Inconsistent Reviews

Most courses with peer reviews ask learners to review 3 to 5 submissions, then results at the extreme ends are ignored. This helps reduce bias, but it can be confusing for the learner.

Some people will give full points just because the assignment was attempted. Alternatively, some people will give 0 points if they disagree with the assignment’s opinions, even if the rubric asks peers to assess the writing, not the point of view. As a result, some learners who put time and effort into their assignment might get a low grade, while some who merely attempted the assignment might get a passing one.

With participants worldwide, learners have varying command of the language (usually English), which can affect writing assignments as well as reading and commenting on others. Feedback may be minimal, which can be particularly frustrating if points have been deducted without explanation.

Coursera Screenshot
Assessors who did not read the instructions.

Sometimes, such as when there’s a lack of reviewers, your assignment score might end up being determined by a single person. In those cases, whether you get a fair score will entirely depend on that person, which may lead to a lot of frustration.

If you are unhappy with your results, some providers allow you to resubmit your assignment.

Coursera Screenshot

Advantages of peer assessments

With so many problems with peer assessments, you might wonder if there is anything good about them. Happily, there are some advantages. By researching, planning, and writing your assignment, you will remember more about the subject. Sometimes, you will find some interesting insights as you assess your peers’ submissions. But don’t copy other assignments and resubmit them as your own!

Your peers might point out issues with your writing that you were unaware of, such as a poorly written essay or arguments that are hard to understand. Use this feedback to learn and improve.

While peer-assessed assignments can be a great learning tool, the credibility of online courses will not endure unless providers take strong action against cheating.

Pat Bowden Profile Image

Pat Bowden

Online learning specialist, still learning after 100+ MOOCs completed since 2012. Class Central customer support and help since 2018. I am keen to help others make the most of online learning, so I set up a website:  www.onlinelearningsuccess.org

Comments 13

  1. Ronny De Winter

    “Motivated cheaters” = contradictio in terminis 🙂
    It went worse over the years. In the early days of moocs the emphasis was more on learning and less on getting certificates.

    Reply
    • Steve

      Sometimes the motivation is to pass a F/T course (where fees are charged) that use MOOCs as partial credit. The only test appears to be producing a certificate, not demonstrating knowledge of the subject.

      Reply
  2. Ronny De Winter

    Let’s hope this article triggers more the course platforms & providers than the potential cheaters!

    Reply
  3. Robert Urbaniak

    Hi Pat

    We sincerely appreciate your engagement with the platform and efforts to make online learning better for everyone. At a time when the world is mostly learning online, we all have the responsibility to make the experience better. We are inspired by our learner community’s passion and commitment to developing new skills by applying themselves across assessments and assignments. However, there are some instances that keep learners from reaching their full potential. Such behavior must be curbed.

    We’ve taken note of your observations and excited to share with you and the community that we’re already working on solutions to help address them. Stay tuned.

    Thanks
    Robert Urbaniak
    Sr. Product Manager, Assessments
    Coursera

    Reply
    • Pat Bowden

      Thanks for the feedback, Robert! I’m looking forward to seeing improvements in the peer assessment process.

      Reply
    • Mark McPartland

      I have similar thoughts regarding Peer Assessments after also taking the Social Physcology Coursera course (great course by the way @Coursera!)

      Firstly Peer Assesment is a great part of any MOOC. It just needs to catch up at least to those trying to game it.

      You can probably break the students into two major camps
      – Those that just want to learn a new skill
      – Those that want to learn a new skill, to improve their career
      – Those that just want a certificate to use for other purposes (to say they have a skill, compete in the market places for jobs etc)

      The first 2 groups are frustrated the 3rd group is able to game the system. Especially the second group which sees their hard work dimished due to the platform integrity being questioned.

      Obviously technology exists to check assignments for plagarism, so if not already employed in Coursera courses, should be.
      Advising students to seek assignment reviewers on the forums is a poor solution to anything and just faciliates the reviews-for-mates issue noted in the article

      One thing that is lacking in online education is sense of community that you get from in-attendance physical learning (and socialising).
      It is certainly an area that MOOCs may want to exploit more.
      Punishment of cheting on assignments is one thing missing, but so too is reward for being the ‘good student’. For those that participate in forums, grade more than their share of assignments, you can certainly build a community starting around that, where reward may only need to be recognition and being able to contrinute to improved systems.
      (Take Google Local Guides as one type of community of the masses. OK a lot more contriutions to work with for Goolge here, but the thing is they pay virtually nothing for this community (give them virtual badges and a chance to attend a conference for free) peanuts, but without it the contributions would likely be a fraction.

      Forums in coursera lost a lot of their use when courses when from scheduled weekly releases to on-demand. Can’t have your cake and eat it too, so do need a rethink on how to build community.

      Delays in marking do affect motiviation and take up of courses. But the platform can do many things to counter this. Either have SLAs for assignment markings that once breached then a real person from Coursera takes over.
      Or utilise your growing community (and advocates) with things like Ubers surge pricing, where greater rewards may be given when there is a demand for marking outstrips supply.

      One doesnt want to have a business model that is built on extrinsic motivation, but a combination of intrinsic (the way the platofrm works helps here or how you can contribute) and extrinsic ( Points, Badges, Leaderboards – OK I also took a coursera Gamifcation course about 10 years ago :)) together can help. Nothing is ever cut and dry.

      I think the main thing for Coursera and others is to start showing they are doing something in this area , as nothing has been seen happening for a long time

      Reply
      • Pat Bowden

        Well said, @Mark McPartland!

        Reply
  4. Dennis

    The issue with delays are too few people take a specialization or course. And peer grading requires three people to grade! I took one on job hunting and only three of students taking the space of one month to grade each other. I am still short of one grader for capstone project. Longer wait entails higher costs to be paid out.

    Coursera need to market low enrolled courses aggressively, or change to one person grading or lower the price tag or make some free with certs for more students to take.

    Thanks.

    Reply
    • Bret

      I suspect that delays are deliberate so that you have to pay more – simply to wait for your results…

      Reply
  5. Bret

    I found on Coursera that if you flagged an assignment as plagiarism then they would remove the assignment and you would have to find another assignment to count for your peer reviews.. so instead from then on I mark them as a 0 and comment that it was plagiarism … I found that almost all assignments from the course I was doing were copies – but it didn’t help that the course itself had an “example” that was the answer.. some people just submitted the course’s example.

    I also truly hate the “mark two peers assignments for your one submission – it drives most of these behaviours.

    Reply
  6. ME

    Coursera is a cyber diploma mill.

    People pay money (or receive financial aid), submit blank, bogus, or copied assignments, and then receive their certificate. They get their sham credential and Coursera gets money.

    Coursera revenues and student cheating increased when courses went to continuous sessions with unlimited re-submissions because that drove away instructor involvement and oversight in the courses they created!

    Even if Coursera does something about the cheating, that will not make the certificates more credible because instructor involvement does not go beyond the sage on the screen and it’s easy to pass a course without ever watching a video or doing any of the readings, whether you cheat or not.

    A month ago I enrolled in a business course and passed without watching any videos or doing any of the readings. I just went straight to the quizzes. With unlimited submissions of auto-graded multiple-choice quizzes and peer-graded assignments, it’s easy to pass and receive a certificate without learning anything, and you don’t even have to cheat to do it.

    Another reason for the increase in cheating is the increase in universities accepting Coursera certificates for some sort of credit. The higher the stakes, the more likely students are to cheat. That’s a truth universally known in education. The answer to the problem is human involvement and interaction with students and review of their assignments by qualified instructors, not technology or AI.

    It’s a sham at scale.

    Reply
  7. José Figueiredo

    The “education system” based on this model is merely another business where the so called educator outsources their responsibility to assess and mark the assignments!
    It is populated mainly by individuals who have understood the trend in the politically correct world of equity in education, where everyone has a right to be equally “qualified” by the so called education system!
    Everyone in this “education game” is fully aware of what’s going on and still persist in fooling each other.
    Anyone who really wants an education must follow the fundamental methodology of the previously well established education system now made much easier by the access over the internet to gather educational sources.
    Real knowledge/education is power and you must work hard to acquire the real education!

    Reply
  8. selma

    I think allowing resubmissions after grading other’s work is causing some of these issues.

    In addition, as @Mark McPartland stated, to prevent plagiarism, a program checking similarity of work to previously submitted work during submission could work. Since each course is different, each instructor could determine that similarity percentage so that a submission is not possible if it is a certain percent similar to another work.

    Reply

Leave a reply

Your email address will not be published. All comments go through moderation, so your comment won't display immediately.

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Browse our catalog

Discover thousands of free online courses from top universities around the world like MIT, Stanford, and Harvard.

Browse all subjects