Socially Sourced Feedback: The Experiment

Abstract: In this exercise, students are connected to experts in their field for feedback coming from another “expert” beyond the instructor. Using a combination of Jing + Screecast applications, the external reviewers comment on students’ online products (in the original case, modules for an online class).

Description: Ongoing thoughtful instructor feedback without question plays a large role in student learning. However, students tend to appreciate getting feedback from additional people in the field. If available, instructors can tap into their diverse Personal Learning Network (PLN) to match up students with a broader set of reviewers of their work. For example, in teaching “CEP 820: Teaching K12 Students Online” with the culminating project of creating a complete online course module, instructor Leigh Wolf was “lucky to have a large and diverse Personal Learning Network” consisting of “professors, MAET alumni, PhD students, practicing teachers, online learning experts,” and so on. By asking friends and colleagues to access a student course and use Jing + Screencast.com (both free tools) to record their reactions, feedback and suggestions for improvement, an instructor can provide students with an “extra set of eyes” on their work that might be somewhat different from his or her views and preferences. Ideally, one would specifically target people in one’s PLN who would match well with the students’ background or project at hand.

In the case of the student-created online-course modules of CEP 820,online modules were sent to reviewers who’s professional experience and interest matched the content at hand (see examples of such matches below).

Challenges regarding this exercise:

There is a risk in relying too heavily on one’s PLN to provide students with reviewers. At the end of the semester, something like this is the icing on the stress cake. It helps to time the submission of the final project 4 weeks before the official end of the semester to try and alleviate some of this stress.
The process is tedious to manage. Until someone develops a way of making the revier/reviewee process a little more automatic, it has to be done manually. This requires minute attention to detail, making sure the right hyperlinks were going to the right people and making sure no one fell through the cracks. With everyone using different course management systems, keeping track of all the access points can be a bit of a challenge.
Not everyone may turn their assignment in on time. Even understanding instructors who realize that things happen, especially with courses such as CEP 820 where students are working adults, many carrying 2 or more courses plus a full time job will realize that it is not fair to the external reviewers to send them late reviews (as they too have full plates.) In those cases, teaching assistants or gracious volunteers and the instructor may have to fill in for the late submission reviews.
It is unlikely to be able to match everyone up with the “perfect” person — in all likelihood one will have to stretch things a bit when it comes to content experts. This is fine as long as some other relevant expertise can be provided, as in CEP 820 where all reviewers were experts in online/hybrid course creation and pedagogy.

The upsides regarding this exercise:

Students may potentially get connected to AMAZING people they would not otherwise “meet.”
The PLN reviewers are able to experience an alternative form of assessment (using Jing & Screencasts) in a “low-risk” setting, i.e., with students in a course such as CEP 820. In the past, external reviewers expressed appreciation in learning how to use screencasting as a form of assessment/evaluation.

Additional considerations/lessons learned:

It may pay off to be relatively explicit with reviewers regarding the technical side of the reviewing task and to give them some tips on microphone and recording techniques.
Only ask external reviewers to perform one review
Adjust the final project due date even earlier – 5 weeks before the end of the semester? - to avoid the end of semester crunch.

Examples of student work:

Troy’s review of Erin’s “6 Traits of Writing” on Weebly Unit
Jessica’s review of Emily’s “American History” Moodle Unit
Sean’s review of Marc’s “Jared Diamond unit for World History” in Blackboard

History of the assignment: 16 brave souls were willing to participate in the experiment of socially sourcing feedback for students in CEP 820 and graciously offered their precious time to assist in this experiment: