The Report by Class Central

Disclosure: Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

Opinion

Learning Music Online 3: Curriculum, Robot Graders and Teachers

In this final installment we’ll dig deeper into computer-based techniques for teaching and evaluating student work.

Editor’s note: This is a guest post from Perry Cook. Dr. Cook is the co-founder of Kadenze, a Professor Emeritus at Princeton University, and a Research Coordinator and IP Strategist at Smule, Inc.

In Parts 1 and 2 of this multi-part series, we looked at the history of studying music and art online. We also looked at the role that social networks and mobile apps can now play in learning and practicing the arts. We also touched on how Digital Signal Processing (DSP) and Machine Learning (or AI) are beginning to change online education. In this final installment we’ll dig deeper into computer-based techniques for teaching and evaluating student work. First we’ll talk some about curriculum design, less specifically about music or even art, but I promise that we’ll loop back to grading and music by the end.

Intro Video for Dr. Cook’s Kadenze MOOC

How We Teach: Curriculum Design

Online education must be much more than just putting a camera in the back of a classroom, taping lectures, maybe adding graphics, and posting videos. We can teach better, tell engaging stories, embed demos, links and other resources, use in-video quizzes, and grab/retain more students. Good professional videos are important, and tech-wise, a good video player interface is essential. This should include rewind, ½ speed and 2x (or faster) video playback, along with closed captions. Couple this with in-video quiz questions, and we’ve gone a long way toward helping different types of students learn at their own pace, while still guaranteeing that they’re watching AND getting the important concepts.

Every “lecture” doesn’t need to be an hour long! And every “week” doesn’t need to have exactly three lectures.  

After over 30 years of conventional teaching, I had a wonderful realization when creating my own Kadenze class. Every “lecture” doesn’t need to be an hour long! And every “week” doesn’t need to have exactly three lectures. Some weekly sessions need over 2 hours total video time, while others are more like an hour. It’s totally freeing to cover what needs to be covered, and send the students off to work on their assignments and projects. Breaking videos up into small 2-10 minute chunks is really important. We live in a world where the most popular online articles take 3-6 minutes to read, so allowing our learners to drop in and move forward in a course at their own time and pace is essential.

And we have to make our courses interesting and fun. In the creative coding world, Dan Shiffman of NYU and the Processing Foundation is a master of this. Dan has made and offered courses and modules to teach Processing since 2012, and he’s always churning out new videos and initiatives. When I visited him at NYU, I was amazed that his faculty office IS a green-screen video production studio.

Intro video for Dan Shiffman’s Coding Rainbow YouTube channel

Speaking of green screens, one of the ways that online video teaching can differ from the classroom is in the use of motion graphics, special effects, and artful editing. Like it or not, since online courses live in the internet world, we must compete with kitty videos on YouTube, or whatever Kanye is doing. A teacher, seated at their office desk, with a webcam pointed up their nose, with annoying background noise and bad lighting, might have been OK in the early days of MOOCs, but no more. We must make our videos and coursework as interesting, artful, and professional as possible.

How We Evaluate: Testing, Grades, etc.

Education has always relied on one or more forms of testing. Pop quizzes with a midterm and final. Weekly homeworks and a big final project. Just these very words induce strong post-traumatic anxiety in many. Fortunately, online education gives us a variety of different models for approaching assessment, verification, and certification.

The “Massive” in MOOC means that we may have to deal with education and assessments at scale. For open online courses, 10,000 students enrolling and submitting the first assignment can mean disaster if humans are expected to do the grading. Peer assessment or grading is an option, but students don’t necessarily trust each other, or themselves, to do a task that teachers and teaching assistants have traditionally done.

I often try to provide one funny incorrect answer as part of multiple choice, or make the questions amusing 

I already talked about in-video quizzes, which are a cool way to keep students engaged and ensure that they’re learning the important concepts. To make it interesting, I often try to provide one funny incorrect answer as part of multiple choice, or make the questions amusing. I also try to put some extra instruction and useful/ interesting facts into the questions, so the student is always learning and engaged. Figure 1 shows a quiz question from my course.

QuizQuestionAshtonKutcherFigure 1 A well designed quiz question can assess, teach, and amuse.

With some thought up front, and computer assistance, large parts of many courses can be graded at scale. We’ve found that for many courses, starting with machine-graded assignments, then moving toward some peer grading/critique, is a good mix. By the mid+ point of the course, the pool of serious students has usually stabilized, and they know and trust each other more from discussion forums. We’ve also found that a gentler “Gallery Critique” rather than strict “Peer Grading” is another useful term and mechanism for assessment. Figure 2 shows student gallery critiques from an illustration course.

GalleryCritiqueImageFigure 2  Required “Gallery Critiques” can be an effective means of both grading and engaging.

But Is Machine Grading Fair?

Proper grading algorithms don’t carry any inherent bias. No teacher’s pets. No “that kid bugs me.” 

Short answer, Yep! Longer answer: Well designed computer algorithms, combined with well- structured assignments (ask the right questions, clearly), are certainly more consistent, and thus fairer, than a large pool of Instructor(s) and student Teaching Assistants. Proper grading algorithms don’t carry any inherent bias. No teacher’s pets. No “that kid bugs me.” Algorithms don’t wake up hung over, giving themselves 1 hour to grade 50 assignments because they want to meet friends for lunch. I could go on, but your next likely (and valid) question might be might be “what about creativity, and human feedback?” We’ll get to that shortly.

Obviously there are things that computers do much better than humans — counting words, computing the exact length of a submitted music composition sound file, or detecting similarities and minor differences in huge batches of similar files. Plagiarism detection via Grammarly.com, PlagScan.com, and other services is now commonly used for grading traditional university coursework, and for good reason, because sources for buying term papers online have been around for a long time now. Grammarly, and other software like the Hemingway App can help us write better, and target a specific reading level. Similar lexicon and grammar analysis tools can be used to grade essays and other long-form writing, and services like turnitin.com provide “automated assessment of writing at scale.”

There is also a split workload model, where a computer algorithm groups all similar responses for a given question or task, then humans can then grade them in bulk. A new company called Gradescope provides exactly that service.

Back to Art and Music

But what about grading creative arts coursework? As mentioned, gallery critiques and peer assessments are good tools, but there is also quite a lot that computer algorithms can do. There’s a research community called the International Society for Music Information Retrieval that has been looking at “machine listening” topics for a long time now. In Part 2 of this series, we already looked at some music teaching/assessment tools such as SmartMusic. Many years ago, Ajay Kapur, myself, Jordan Hochenbaum, Owen Vallis, and some others started thinking about this topic as applied to the education space. The result was Kadenze, and the graders we’ve constructed extend beyond music and sound, to photos, arbitrary images such as hand drawn and lettered pages (comics!), and videos.

Figure 3 shows the results of machine listening applied to a student composition (a submitted sound file). The “robot” has automatically found and marked the four main sections of the piece, and determined that part of the intro (first) section sounds most similar to the ending section. If the submitted composition had only 1 section, or 100, it would not pass the required submission criteria. This “song” clearly has the requisite form, and variety. Whether it’s a hit or not is up to humans and the marketplace, but the robot likes it a lot.

figure3-imageFigure 3: Robot grader automatically finds and mark sections of student composition.

I think the future is super bright for tools such as these, and as computers get faster and cloud computing gets cheaper, there are huge possibilities for real-time interaction, tutoring, and grading. So it’s possible to start thinking about “robot teachers” or at least something like automated TA office hours, or critique sessions composed of machine learning algorithms trained from the works of the masters, or from past submissions for a given course.

Of course, humans have to select and design the curriculum, create the assignments, write and train the graders, etc. And I hope the final arbiter of great art works will always be humans. But the ability to offer arts courses to millions of students around the world is well worth embracing some new and potentially scary technologies. Personally, I’m in, because I don’t want to grade ten thousand homeworks per week 🙂

Perry R. Cook, PhD
Professor Emeritus, CS and Music, Princeton University
Co-Founder and Executive Vice President, Kadenze Inc.
also, Research Coordinator and IP Strategist, Smule, Inc.

Comments 5

  1. Avatar

    Jose Antonio Pereira

    Congratulations on the wonderful work. I am Jose Antonio Pereira, music teacher for children and youth in Brazil since 1976. From 2008, became a present tutor of undergraduate musical-distance education at a university. I elaborated a method called Music in Schools with activities on the musical sound, instrumental universe. We will explore the history and musical routine of students and teachers, highlight the lyrics and known and practiced performances. I created scripts that material to create games, audio-visuals, animations, etc. Contact mt-pereira@hotmail.com pereira.uab@gmail.com

    Reply
  2. Avatar

    Polly Foster

    Dear Perry, it was a real pleasure to read your article! I’ve never heard before about programs that could evaluate songs written by students. Our world is changing so fast!

    What is my opinion about machine grading? Not a long time ago I designed online course for my students. To make the process faster, I added tests after each unit. Besides, we have a few creative assignments in our course, where students wrote their own argumentive essays. That essays were graded with external tool called Unplag https://unplag.com integrated into our Canvas LMS https://www.canvaslms.com via LTI.

    I was pretty sceptical about the quality of such grading, because we have never tried integrating something like that into our own designed course. But after that I was surprised by how much time was saved due this integration! So, now I believe in machine grading and its prospect in simplifying the job of teaching throughout the world.

    Reply
  3. Avatar

    Monica Seims

    Perry, you made a good job! Your article deserves a proper recognition! I’m a teacher so I’m interested in making an effective online education. The studying process shouldn’t be boring so that we have to use different tools to improve the presentation of material. Besides, there are many tools that may be interesting not only for pupils but for teachers as well. Recently, my colleague recommended me a tool that’s called PlagiarismSearch. This service checks the pupils’ works on plagiarism. What I like the most is that this tool makes accurate searching and detects all the plagiarized phrases. Consequently, I can say for sure that everybody who are involved in studying process should use such tools.

    Reply
  4. Avatar

    Nora Lewis

    “Robot teachers” are something that will be obviously actual within a few years! As far as we are living in the ear of new technologies, online tools are indispensable for both students and teachers. From my own experience, I can say that I’ve refused to work with paper sources and I’m only using electronic tools. Recently, I was forced to buy college term paper from one of custom writing service due to lacking my free time to do it by myself. I have no idea what I would do if there weren’t such online companies that may assist you in solving your studying troubles.

    Reply
  5. Avatar

    Mariana

    Dear Perry, thank you for a great article! I’m amazed by the way you design your courses and wish many colleagues worldwide to learn it from you.

    A couple of years ago our pedagogical team started a new project for connecting learners with objects of their study worldwide: we developed the BeED platform where educators were able to design learning experiences attached to the places of interests in their own city – you could have music classes in Opera house, classes of history in the museums or even maths in the shopping mall – we adapted the content to a new learning environment and made it available for learners on the mobile devices offline.

    Each learning experience can be provided with guidelines, videos, audios and quizzes, but, what was the most important – we added the Reflection stage to stay assured that all the class gained a deep understanding of the topic and they can express their thought in any way more convenient for them.

    We would be happy to share your courses through BeED Nexus https://www.beed.world/nexus and to promote your amazing Music curriculum in South East Asia since our platform is also compatible with rubric making for the assessment of the knowledge, skills and traits.

    Reply

Leave a reply

Your email address will not be published. All comments go through moderation, so your comment won't display immediately.

This site uses Akismet to reduce spam. Learn how your comment data is processed.