The Great MOOC Slander? Realities and Narratives of Education and Learning [UPDATED]

education online (Photo credit: Sean MacEntee)

Note on the title: I was pretty miffed when I started writing this post. And the title was the first thing that came to my mind. But as I was writing, much of the initial anger dissipated but I still like the title. So I added a questionmark to the main title and left it as it was. The subtitle is really what the post is about.

Why do so many progressive educators trash talk MOOCs?

I just don’t get it. What do all of these educational progressives have against MOOCs? They all critique conventional education and try to innovate all the time. But when it comes to MOOCs, they behave as if the status quo was the best thing in the world.

The best explanation I can think of is the clash of ideology with personal privilege. I have noticed this in many areas where otherwise progressive people start saying “enough is enough” once the change they are espousing impinges on their privileges and requires some personal inconvenience. Black empowerment is great, but black power is going too far. Women rights are fine, but having to think about whether I can open the door for a woman or not is going too far. Protecting religious freedoms is fine, but not allowing anyone to insult anyone else’s religion is going too far.

The MOOC backlash seems to me to follow the same scenario: Reforming instruction is fine, but removing the need for an instructor altogether is going too far. Reforming assessment is fine but removing rigor is going too far. Open collaboration is fine, but copying whole paragraphs is going too far. Opening up education to more people is fine, but having 100,000 people in a class is going too far. Drop out of 10-30% is fine and expected but having only 10% of students finish a free no-commitment course is going too far.

And in many of these scenarios, there are cases where some reform is going “too far” or is not “faithful to the original intentions”. But I would ask any of these “too far” critics to consider how that “too far” affects them personally. This personal impact could be in future job prospects, change of practice, change of how things look, change of how we talk about things. And all of these will cause some personal discomfort if only in the form of cognitive dissonance. And then consider how their criticism is colored by the personal impact. Maybe it isn’t, or maybe it doesn’t matter. But I would recommend spending a few minutes on this.

This post came to me while I was listening to the great fortnightly podcast Digital Campus with Digital Humanities luminaries Amanda French, Dan Cohen, Mills Kelly, Tom Scheinfeldt and guests Audrey Waters and Bryan Alexander. http://digitalcampus.tv/2012/09/10/episode-90-back-to-school-special/

I have long puzzled over why this podcast has been snide and largely inaccurate about MOOCs. I would have expected these people to embrace MOOCs and work as hard as possible to make them better, experiment with the format, open them up to new subjects and new audiences. Sure, it’s annoying when something you support becomes a fad but the solution of “nobody goes there any more because it’s too crowded” just doesn’t seem to be the right one.

Show me the learning and the “education = learning” illusion

After some collective MOOC bashing (about which more below) Mills Kelly struck a conciliatory note when he advocated the motto “show me the learning” in the form of let’s look at the participants a year from now. And then we can compare the results with traditional education. I’m all for that. But I think we can pretty confidently predict what the results of both will be. Awful. And make it 5 years later, too. What have the participants learned. On average: nothing. Many university lecturers have had that encounter with a star student years later in which they discover that all the student has retained is vague impressions and inaccurate snippets of lore. Unless they stayed in the field and continued their engagement with it, they won’t have any meaningful foundation upon which to build. Some skills, some attitudes and some info, to be sure, but nowhere enough to pass the exam. In fact, I am convinced many professors could not pass some exams in their field set by their peers.

So I say let’s do that. Let’s do mass testing of student learning 1 year, 5 years and 10 years later and expose the great swindle that is higher education. Those students who took the MOOCs to learn actual skills like programming in Python will do great if they continued programming in Python or some other language and those who just took it out of vague interest or to fulfill some requirement will do really badly. Those students who took literature or history classes (MOOC or otherwise) and continued in that field will do great, those who just did them as part of some requirement will do badly. Will do the students who took the same classes in the traditional format do better? I doubt it. It will vary across classes and subjects but any difference you might see a year later will be negligible 5 years on and erased 10 years later. Nothing is nothing.

But of course, there’s a fundamental flaw in this approach. It assumes that we can meaningfully measure learning across cohorts and that cohort results tell a good story about the individuals and vice versa. Academics are well aware of this flaw which is why they engage in the narratives of “learning to think”, “acquiring study skills”, “getting foundations for further learning” or even the odious “becoming a rounded individual” which mostly means “becoming like me and mine”. But none of these can be measured in a consistent way (though I have proposed a Turing test of education) so we have to rely on anecdotes and vague impressions that are irretrievably polluted by the prevailing narratives.

But the truth is the actual learning is largely irrelevant. Higher education has never been primarily about learning actual things or actual skills. It has always been (and I mean all the way to the first universities) about peer acceptance for the graduates and meeting the requirements of the institution for the students. That’s not to say that a lot of people don’t learn lots of useful things while attending university. But if that was enough lawyers wouldn’t need the bar, doctors wouldn’t need their residency and university teachers would not have to learn everything all over again when they start teaching a new subject.

Mills continued his conscilience by saying that he is skeptical about a lot of university instruction. But I think that is the wrong approach to take. University instruction has always been just abominable. The vast majority of classes most university students have attended throughout history were taught by drones more or less competent in their subject sometimes reading out of a textbook sometimes cracking a joke. If that really mattered how would have we ever gotten to where we are now? Massive innovation and erudition as far as the eye can see. Even those we disagree with (like the neocons and creationists for me) cannot really be accused of a lack of intelligence or erudition. We talk about the need for better historical education but some of the worst political decisions have been taken by people who studied history meticulously (and it’s no good saying “if they had only read that one paper I wrote on that issue”). We talk about the need for better science education but some of the best innovations have come out of school drop outs who flunked the foundational STEM subjects. Why on earth would we think tinkering around with instruction would make a dent in any of that?

Access over instructional purity: The case for MOOCs

So why do I care about MOOCs? Well, first, even though I think instructional reform is of marginal importance in the overall scheme of things, I still prefer an engaging lecturer over a boring one, and a motivating activity over a dull exercise. Both as a teacher and a student. And it’s absolutely true that in individual cases, there are ways to help people learn particular things more easily (they just don’t make as much of a difference in aggregate as we pretend). And MOOCs offer a lot of opportunities for instructional innovation and even where they are traditional like presenting videos, they are likely to present the better of the traditional instructional approaches out there.

But most importantly MOOCs expand access to knowledge and hopefully with time also to certification. They expand it socially as well as globally. They still need to do a better job in expanding the social networks that are such an important part of attending university but they do a good enough job in some formats and are looking to expand this some more.

Why do good researchers ignore good research practice when talking about policy affecting their own practice?

For some reasons, there is a lot of bad information about MOOCs out there. People who are taking them to write about them generally do it with agendas a mile long and I have yet to see a write up that wasn’t colored by the writer’s own professional wishes. I am not aware of any non-anecdotal research on MOOCs that would investigate them by talking to students and teachers following the best ethnographic research practices.

[UPDATE: The day after, I posted this, I was watching this video on teaching by the Harvard physicist Eric Mazur who said [about 18 minutes in to the video]: “It’s very important to have data in education. I often go to faculty meetings where my colleagues and I talk about teaching and there are quite a Nobel laureates around me. And it is always surprising to me how whenever the discussion shifts from teaching to education, people, even the most reputable scientists, completely abandon the scientific method. All of a sudden the discussion is about anecdotes: ‘My students learn better whenever…’, or ‘My students like it when I do this…’, as if liking equates learning. Data are important.” Now, I think I would strongly disagree with Mazur on the kind of data that is important or the scope of what constitutes the scientific method, but we would have no quarrel in that whatever underpins policy debates about education is not it.]

Whose controversy?

The podcast that got me so riled up started with saying that there were a lot of MOOCs over the summer and a lot of controversy around them. And perhaps if we look at the blogs about them, we could say that a lot of them were critical about the MOOCs. But the controversy was entirely generated by the same people who now refer to it as “the controversy”. A much more accurate statement would have been, “there were a lot of MOOCs over the summer and a lot of people like us were writing about how we all don’t like them”.

Plagiarism

Another reference that gave me pause was an offhand comment about the “rampant plagiarism” in MOOCs as if that was in any way a proven thing. There was some evidence of plagiarism but how “rampant” it actually is would require careful and cconscientiousr research rather than skimming a few biased and alarmist blog posts. I was struck that nobody on the podcast (full of excellent humanities scholars) asked what is the evidence of the plagiarism? No one raised the question of the “discourse of plagiarism”, or even suggested that “plagiarism” is equally “rampant” in traditional schooling. But even the article referred to only talks about “dozens” of reported cases of plagiarism in a course taken by tens of thousands. Surely, this is pretty good going percentage wise.

Non-native speakers

Another question raised was that of “non-native speakers” in one of the literature MOOCs. But nobody mentioned the fact (which I believe was discussed on the same podcast about a year ago) that lots of students in traditional US universities are exposed to non-native speaker TAs and lecturers. Nobody suggested that new disciplines often constitute a foreign language to “native speakers”. The one suggestion was that people could write and grade papers in their “native” languages but nobody suggested that the “native” English speakers try to get over it and learn how to interpret the English of non-native speakers and give good feedback. Surely all the humanities scholars on that podcast must be aware of the difficulties surrounding language skill and language politics. Nobody mentioned that often “native” speakers write appallingly and “non-native” speakers are quite good writers (if plagued by difficulties with idioms). I once graded 10 papers from a small class with one Chinese students whose spoken English was very hard to follow but I couldn’t pick out her essay out of the lot. This was in part because her written English was not marked by accent and the pressure of conversation and in part because the some of the “natives” couldn’t string together a coherent written sentence in English. We should get used to the fact that written English has elements of a foreign language to it.

Peer assessment

There was also this vague notion that “peer assessment” was somehow suspect. Luckily there were some dissenting voices on this. But nobody mentioned how variable the level of feedback students receive from their instructors and TAs in traditional universities is. I’ve had to moderate a lot of feedback by experts and I have no illusions about its quality.

I took the first 2 weeks of the same Coursera course mentioned as beyond redemption on the podcast and the feedback I received was of entirely sufficient quality. There were a few duff ones but since every assignment received at least 5 pieces of feedback, in aggregate this would lead to improvement. Sure, there would be ways to improve this system, like tracking the quality of feedback, rating “helpfulness” or requesting moderation in cases of disagreement but those are just tweaks not fundamental issues.

“Training is not education”: The spaces in between

Towards the end of the MOOC discussion one of the hosts (I believe to was Tom Scheinfeldt) summarized his unease about the MOOCs as the difference between education and training. MOOCs, he claimed, lack the spaces in between. Even if the acquisition of knowledge is the same, the students don’t get to chat to the professor during the break afterwards and get random snippets of wisdom. It was curious that he chose as an example “being told that it is not OK to copy Wikipedia” which seems to me to be so trivial that a it could be delivered over Twitter and also something that should probably not be left to a random hallway conversation.

But the bigger problem is that this description of higher education (or any education) is completely illusory. There is masses of ethnographic research replicated pretty much every decade since the 1960s that has shown (beyond doubt, I would say) that nothing like that happens in the “spaces in between”. Most of the conversations students have with each other about their classes is about how to pass them, not what they about. And almost no students ever talk to the professor personally or in a way that can justify $20,000 a year. (See References section below.)

But even if we were to accept that such serendipitous encounters are important and they are certainly narratively salient (see below), there would be numerous ways of replicating them in a MOOC. We would just have to ask the critics to promise not to take the few inevitable bad examples and present them as evidence of MOOCs being broken.

This wouldn’t work in a MOOC or what do we do when we teach?

There seems to be a small subgenre of MOOC criticism that has to do with their online nature. These critics are accepting that MOOCs are what they are but keep listing subjects (typically their own) of which they are convinced that they could not be taught via an online course, let alone a MOOC. I used to think the same about my subjects but every time I have I was shown the error of my ways. Now I think that essentially, if somebody can perform a surgery on themselves with online instructions, you can pretty much learn anything online. That doesn’t mean that personal contact is often not a useful shortcut or that a particular kind of feedback is not more effective in person. But is the lab, seminar, or the office hour the only alternative? I was thinking about some of these when I was thinking about the flipped school year.

I was struck by the reference to “people teaching on a MOOC” by one of the podcast discussants. I have long been puzzled by the references to “teaching” by educators. It first came when I was attending a seminar over 20 years ago, the phone rang and the professor said, “sorry, can’t talk now, I’m teaching”. But it didn’t feel to me like he was teaching. He was sitting there and telling us things. Was that it? Was that what teaching was? What do educators mean when they look at their schedules and say “I’m teaching on Monday from 10 to 11”? And isn’t saying “I’m teaching on a MOOC” referring to something completely different? Perhaps MOOCs are so disturbing to many “teachERs” because they don’t get to “teach” any more, or at least not in any recognizable form?

I have noticed that a lot of people (including me) who are very skeptical about lectures as a medium of instructions actually like lecturing. The ritual of standing up in front of a group of people and telling them things is just exhilarating for some teachers. The challenge of formulating the content in a way that will lead people to learning, posing the right questions, coming up with the right analogy, seeing the “whites of their eyes”. But lectures are also a performance of teaching in the ethnographic sense. Both the lecturer and the student play a role they feel is required for them to describe their activities as teaching. Giving and attending lectures is fullfiling a ritual duty that is hard to overcome. I personally also like preparing lectures because they let me think about the subject in more organized ways. And I think others do too. Feynman famously told his colleagues in pure research that they should do some “teaching”. I suspect this is what he had in mind.

The same goes for looking over students’ shoulders and giving them expert advice. One feels so useful when one can point out a little error, share one’s accumulated wisdom. Isn’t that what being a teacher is after all?

But even that is just so much voodoo. How much do we actually learn from this feedback? I have observed hundreds of lessons and dozens of lesson observers. I have seen people give feedback and receive feedback. And I am highly skeptical about its effectiveness. There’s just too much going on. Too many things to focus on. Repetition and incremental improvement are the norm. Not comprehensive and exhaustive feedback on every minutiae of someone’s performance. Now, there’s lots of research on the importance of targeted coaching for skill acquisition. But that’s hardly what goes on in a typical section seminar.

We can all think of stories where just the right kind of feedback at the right time made all the difference. Just the right analogy or illustration chosen in a lecture or presentation made things previously muddy crystal clear. But if we look at the complex thing that is our current expertise, can we really see a clear an unequivocal pattern of instruction? I suspect not. So why are we so hell bent on recreating it for others?

Improving MOOCs

Do MOOCs need improvement? Absolutely, I can think of a dozen ways in which to improve MOOCs. Some of these improvements are along the lines of improving higher education in general and some of them are to do with the format. But I would argue that we do not actually yet know what the real problems with MOOCs are. Which is why I am so puzzled by the backlash (see also here).

Will MOOCs be the salvation of higher education? Probably not. Will they cause the downfall of traditional higher education institutions? One can always hope but it is highly unlikely.

The most likely future of MOOCs is opening some new avenues of access and making available some content. They are likely to be co-opted by traditional institutions that will use them as advertising or loss leaders. And with that the MOOCs will become more and more traditional. But at the same time, they will leave their mark and higher education will not be quite the same as it was before. Kind of like with all “revolutions”.

So there’s a lot of hype about MOOCs, but there’s just as much hype about universities. The massive fee inflation and variable quality should give us just as much pause as any deficiencies (real or imagined) in MOOCs that are still only on the periphery of this oligopolistic industry.

Instead I would recommend to the progressive educators (among whose numbers I count myself) to examine how much of their opposition is driven by the challenge MOOCs provide to their status as innovators and their mass awareness raising success when compared to their own decades-long and admirable records. I am not for a moment accusing anybody of small mindedness or petty jealousy, just the normal human biases that we often ignore in academia. But maybe it’s just me and their epistemological consciences are as pure as the driven snow. In which case, I apologize.

References: Research on the student experiences of higher education

These are the references for research about what actually happens in higher education. There have been pretty consistent results in this field since the 1960s but seem to have made pretty small impact on how academics (even those inclined to consider this kind of evidence) think about higher education.

Becker, Howard Saul, Blanche Geer, and Everett Cherrington Hughes. 1968. Making the Grade: The Academic Side of College Life. New York,: Wiley.

Becker, Howard Saul, Blanche Geer, Everett C. Hughes, and Anselm L. Strauss. 1977 [1961]. Boys in White : Student Culture in Medical School. New Brunswick, N.J.: Transaction Books.

Holland, Dorothy C., and Margaret A. Eisenhart. 1990. Educated in Romance: Women, Achievement, and College Culture. Chicago: University of Chicago Press.

Horowitz, Helen Lefkowitz. 1987. Campus Life: Undergraduate Cultures from the End of the Eighteenth Century to the Present. New York: Knopf.

Moffatt, Michael. 1989. Coming of Age in New Jersey: College and American Culture. New Brunswick: Rutgers University Press.

Nathan, Rebekah. 2005. My Freshman Year: What a Professor Learned by Becoming a Student. Ithaca: Cornell University Press.

Nespor, Jan. 1994. Knowledge in Motion: Space, Time, and Curriculum in Undergraduate Physics and Management. London: Falmer Press.