In the 3 months since the previous post about my first two experiences of the Massive Open Online Course world, I have been involved in four more courses, all with different and interesting approaches to assessment.
Social Network Analysis
The Social Network Analysis MOOC was the only that had all materials (except the final exam) available from day one, but still had a weekly schedule of assessment deadlines. Like other Coursera courses, it was possible to distribute an overall ‘voucher’ of days across assignments, to turn them in later without penalty.
This course provided two tracks of assessment. The base track consisted of 8 weekly quizzes (called homework in this MOOC), except for the final 9th week, and a timed exam: 2 hours from the moment you clicked ‘start’. The second, optional track was for those who wished to get a distinction grade: 3 additional assignments and a project were due. The project was peer-assessed, all other assessment was automatically marked multiple-choice questions.
A good part of the automated assessment consisted of applied questions, e.g. to compute a metric for a given real-world network using some software and then select the correct answer form the provided choices. Some other questions required reading (or at least closely skimming) given research papers. The distinction track assignments involved filling out the blanks in given R scripts to analyse a network, but R was not taught in the course! Fine by me: if you want distinction, do some extra learning. Help was available in the forums.
The projects were the most interesting assessment. There were 3 or 4 kinds of project to choose from, e.g. analyse a network of your choice in depth, or come up with a model of network growth and simulate it in NetLogo. Students were encouraged to discuss and even present their projects in the forums before submission. Unfortunately, although I did the 3 required assignments, I didn’t have the time for the project and had to drop out of the distinction track.
Given that the course was about social networks, the instructor provided a rather interesting (and optional) way of getting extra points. Each student could submit the names of two other course students she/he wishes to be ‘friends’ of. The student having the most friends by the deadline would get the extra points and could share them with other students. There were also extra points for the student with the most creative approach in getting ‘friend requests’. As you can imagine, the forum thread dedicated to this ‘assignment’ was very active.
We were told in advance the exam would not require using any software. Still, revising for it was not easy. Going again through all the video lectures was out of the question, so I skimmed the slides. Before starting the exam, I opened all PDF slide files on my computer to quickly search them during the exam: the modern version of an open-book exam.
The stated workload for this course is 5-7h/week for the base track and 8-10 for the distinction track. Can’t comment on the latter, but the former is about right. It took me maybe 4 hours per week because I skipped the forums, the Google hangouts with guest researchers, and the optional social assignment, so that I had time for the following two MOOCs. On hindsight, this is the kind of course that deserves a higher time commitment as there is much you can get out of it. If you fancy this topic, I recommend this course.
A final interesting tidbit: after the course, you may provide your e-mail and website to the Univ. of Michigan (the provider of this course), so that companies looking for people with the skills taught in this course can contact you. Is this a way of funding the MOOC?
Data Analysis
The Data Analysis instructor has made his lectures and slides publicly available. This was a traditional Coursera MOOC, with one track, no exam, weekly automatically graded multiple-choice quizzes and two peer assessed projects, both of them consisting of data analysis assignments. What made this MOOC stand out for me was the quality of the teaching and the excellent alignment between assessment and lectures. The lecturer also provided in advance a complete written up data analysis, including the R scripts used to process the data, and a detailed rubric for the peer assessment grading. Everyone thus had a very good idea of what was expected.
The instructor warned that the week before the first project’s deadline was the most intensive, as we also needed to watch the following week’s lectures beforehand in order to do the project. This is of course a very unfortunate scheduling. I guess it’s to make the two peer assessment periods fit within the overall course duration, but surely it would be better to extend the course by one week without lectures than having some very critical weeks. I wonder how it affected retention. I, for one, had to drop out at the end of week 3, before doing the first project, but I will enrol again if this course is offered in the future. On hindsight, doing three MOOCs in parallel wasn’t one of my brightest ideas.
Introduction to Complexity
The Introduction to Complexity MOOC from the Santa Fe Institute (SFI) uses their own platform, which is similar in functionality to Coursera’s (from the student perspective). Videos are delivered from YouTube (to save SFI’s bandwidth?) but have private URLs, i.e. you can’t find them on YouTube. It’s the only MOOC I’ve taken that has no deadlines for the automatically graded quizzes, other than the end of course date. You can therefore still enrol and cram the 9 quizzes within the remaining two weeks, if you have the time for it. The workload is given as 3-6h/week but I found it to be less than 3h if you just watch the lectures, which are well-paced and very clear, and answer the quizzes, which were rather straightforward and sometimes dull. For example, one quiz’s questions were mostly converting binary to decimal to get the Wolfram number of a cellular automaton. On the other hand, whereas the Coursera MOOCs allow multiple attempts per quiz, this course only allowed one.
More interesting was the weekly optional homework, which consisted mostly in modifying or extending the NetLogo models used in the lectures. Each homework had 3 problems, a simple one for beginners, a more interesting intermediate level problem, and an advanced, possibly rather open-ended problem. Solutions for some of the problems were provided. I think this course would have benefitted from putting the advanced homework problems into a distinction track, to give some credit for doing them. I didn’t attempt the homework but I guess the advanced problems would have taken the workload to over 6h/week.
Another good feature of the course were the recorded Skype chats between the lecturer and various world experts in the topics covered by the course. It’s a nice way of getting multiple perspectives.
The last unit was a guided tour of the SFI, and the lecturer talked to its president, some faculty and post-docs, the education and business programme directors, etc. Given that the SFI is dedicated to complexity studies, the guided tour felt appropriate and not as a marketing ploy (although there’s obviously a bit of it).
Two last tidbits: the students were asked for voluntary donations to help fund the further development and presentation of this course, and for voluntary translations of the subtitles into their native languages. A nice way to crowd-source part of the MOOC production.
Overall, a very good and gentle introduction to complexity across social, economic, biological systems. Another course I’m happy to recommend.
Write Like Mozart
Last week I started my second music MOOC, An Introduction to Classical Music Composition. The first lecture spells out the musical pre-requisites for this course. I don’t tick all the boxes, so I won’t be surprised if I find myself unable to finish the course. But so far, so good. The first week’s lectures were very clear, well paced and prepared directly for the assignment, which I had no trouble doing.
The way we were prepared for the assignment was quite simple and effective. The lecturer first presented the harmonisation problem, then the video was automatically paused for us to work out the solution ourselves. When ready we would click the ‘continue’ button and see the instructor’s solution. A single video would have several of these steps towards the complete solution.
The instructor’s welcome e-mail message spells out in detail his choice of assessment. Basically, each weekly assignment is a composition, but it is not assessed, as automatic grading is largely unfeasible, and peer-assessing each assignment would be a high workload. Hence, the instructor simply provides a possible ‘solution’ to each composition assignment, and the assessment is a peer-graded final project, with a passing mark of 65%.
Conclusions
After enrolling in 6 MOOCs, there are various lessons learnt, not only for learning but also for teaching via a MOOC. I won’t repeat the tips to MOOC developers I wrote in the previous post.
With regard to learning, just doing the barely necessary to pass is acceptable and is certainly enough to learn a lot of interesting stuff. However, the best option is to do one MOOC at a time and fully engage with its various learning opportunities, like doing advanced homework and projects, participating in discussion forums, and reading the additional materials. I was impatient to wait 6 or 12 months for the next offering of these MOOCs and in the end I didn’t take as much out of each as I could.
As for the pedagogy, clear, well-paced lectures are of course key, as they’re the main teaching vehicle. Aligning well the theory in the lectures with the practice in the assignments not only reinforces the learning, it also motivates students whilst reducing their workload. Automatically graded quizzes tend to be too simplistic, e.g. focussing mostly on recall, although several courses include multiple choice questions that require students to do computations or simulations. Another way to make assessment more engaging is to have 1 or 2 peer-assessed, larger projects, although this tends to introduce workload peaks. Not all the students have the same abilities, interest or time available. Differentiated activities make therefore sense, and two courses catered for the keener students, with more time on their hands, to be challenged and dive deeper into the course topic. Two of the courses provide model solutions to example assignments, which is particularly helpful in a setting where students don’t have individual support by a tutor.
While there are four common core mechanisms (video lectures, discussion forums, mix of short and longer assessment pieces and of automatic and peer marking), there is quite some variety in the actual use of those mechanisms, as the examples in this and the previous post show. Overall, my impression of MOOCs so far is quite positive, although they’re obviously not suitable for all kinds of learners and topics.
