Do MOOCs Help?

Unlike at colleges and universities, where students finish their studies and leave with a diploma, most learners who complete one of Coursera’s massive open online courses report benefits that help them in less measurable ways.

But results from a new report, billed as “the first longitudinal study of open online learning outcomes,” also suggests many learners credit MOOCs directly for pay raises, promotions, academic progress and more.

Specifically, the report corroborates previous findings that more learners are using MOOCs to further their careers than their education, and also that those from less-advantaged backgrounds are most likely to benefit from the courses. The full report, titled “Impact Revealed: Learner Outcomes in Open Online Courses,” appears in Harvard Business Review.

“The impetus for doing this was we’ve been offering access to amazing education for years now, but people have been asking us, ‘It’s great that you’re doing this, but are you helping anybody?’” said Daphne Koller, co-founder of Coursera. The MOOC provider has always believed it is helping people, she said, “but belief and data are two different things.”

Brandon Alcorn, Ezekiel J. Emanuel and Gayle Christensen, three prominent MOOC researchers from the University of Pennsylvania and the University of Washington, are behind the study. It builds on preliminary internal research from Coursera, which also provided some logistical support but did not fund the study.

Those names should sound familiar to anyone following MOOC research; the team’s previous research has helped shape public understanding of online courses, the people who enroll in them and their effect on traditional higher education.

Using data from MOOCs offered by the University of Pennsylvania, Alcorn, Christensen and Emanuel were some of the first to suggest that MOOC learners were more likely to be employed men in developed countries who had previously earned a degree — countering the early narrative that MOOCs would democratize higher education around the world. Another study by the researchers reassured business schools that MOOCs wouldn’t cannibalize their enrollments.

This study examined what — if anything — learners have gained from completing MOOCs, and whether they serve as a “luxury good for rich people” or as a boon to learners of low socioeconomic status, said Emanuel, who is vice provost for global initiatives at Penn.

Christensen, assistant vice provost for global affairs at the University of Washington, said, “Is there really any benefit to taking massive open online courses? Is it something that people are doing for edification, or are people actually seeing benefits of doing that?”

The researchers received survey responses in December 2014 from 51,954 learners who completed a Coursera MOOC before Sept. 1 of that year. The sample reflects MOOC enrollments as a whole. More than half, or 58 percent, were male, 69 percent held either a bachelor’s or a master’s degree, and 58 percent said they were employed full time. About one-third of the responses came from learners in the U.S., and about one-quarter from countries outside the Organisation for Economic Co-operation and Development (OECD).

The results suggest a majority of learners feel they benefit either professionally or educationally from completing a MOOC. When asked to name tangible benefits, such as securing a new job or receiving academic credit, however, those rates dropped.

About half of the surveyed learners, or 52 percent, said they initially enrolled in a MOOC to advance their careers, compared to only 28 percent who did so for educational purposes. In both groups, nearly 90 percent said they benefited from the MOOCs, though more learners reported tangible career benefits (33 percent) than tangible educational benefits (18 percent).

Among the “career builders,” as the report labels them, 62 percent said they felt better prepared for their jobs after completing a MOOC, and 43 percent said the MOOC helped them become more competitive applicants for a job. Finding a new job was the highest-scoring tangible benefit, with 26 percent reporting that outcome. Starting a business, earning a raise or receiving a promotion all ranked in the single digits.

Similarly, among the “education seekers,” more respondents reported perceived benefits such as gaining knowledge, deciding on a major or reviewing previously learned material. About one in every five learners, or 18 percent, said they received academic credit or had prerequisites waived after completing a MOOC.

“This is flexible technology,” said Alcorn, project manager for global initiatives at Penn. “If [learners are] trying to get into college, they’re able to use MOOCs to determine their major or prepare for standardized tests. … If they’re trying to get a new job, they’re able to do that too.”

Moreover, low-income, under- or unemployed learners without bachelor’s degrees were more likely to say they benefited from the MOOCs. Among education seekers, 91 percent of learners of low socioeconomic status reported benefits, compared to 86 percent among learners of high socioeconomic status.

“The fact that people who don’t have those backgrounds are reporting benefits is an important point,” Christensen said. “We might think that the opposite would be the case, because as studies of massive open online courses have shown, they’re largely taken by people who are more advantaged.”

Although the gap is in the single percentage points, Christensen said it and the other results in the report are statistically significant. She acknowledged that the low survey response rate could limit the significance of the findings. Although tens of thousands of learners responded, about 700,000 did not — a single-digit response rate.

The researchers said future studies could look at learners who take part of a MOOC but don’t complete it, a group not captured in this report. If those learners also report career and education benefits, the findings could be used to counter the fact that MOOCs often have single-digit completion rates, a common argument among MOOC critics.