SAN FRANCISCO – To keep up with the breakneck pace of developments in online education, higher education researchers must be nimble and sometimes make do with “dirty” and quickly gathered data. Otherwise weighty discussions about student learning might get lost in all the hype around massive open online courses and other digital innovations.

That was a takeaway Tuesday during a panel discussion here at the annual meeting of the American Educational Research Association. Participants in the session tried to define a meaningful research agenda around emerging forms of course delivery.

Research on digital learning should probably take at most six months to develop, said Kendall Guthrie, a senior program officer and evaluation lead for the higher education program at the Bill & Melinda Gates Foundation.

“Dirty data at the time a decision is made is way more influential than perfect data after a decision,” said Guthrie. “Respect the pace of innovation.”

Even so, she said researchers must be careful about drawing overly broad conclusions about a field that, in many cases, is in its nascent form. For example, she is skeptical about early efforts to prove whether brand-new online offerings are working yet.

“Don’t rush to judgment with premature efficacy studies,” said Guthrie.

One promising area for researchers to plumb, she said, is when digital innovation might lead to unintended consequences.

That possibility is real as policy makers look to online education to help serve more students efficiently. State lawmakers in particular are increasingly optimistic about the promise of MOOCs or competency-based education.

“People are desperate for solutions,” said Christopher J. Rasmussen, a panelist and vice president for research and policy analysis at the Midwestern Higher Education Compact.

But with new approaches often come new challenges. Guthrie said the Gates Foundation is interested in research that looks at whether, and how, digital innovation could potentially exacerbate racial or socioeconomic achievement gaps, or contribute to dips in grading standards.

There are plenty of areas of research on digital learning that have not been well explored, panelists said. One is whether social media can be harnessed to help boost retention and graduation rates among community college students, said Cynthia D. Wilson, vice president of learning and research at the League for Innovation in the Community College.

College marketers are way ahead of researchers in looking at social media, said Wilson. That’s because recruiters know “you fish where the fish are.”

‘New Science of Learning’

Public fascination with massive open online courses has upped the stakes for education researchers.

Journalists are playing a big role in driving the conversation around online learning, said Mario Martinez, a professor of educational psychology and higher education at the University of Nevada at Las Vegas. Martinez, who led the session, recently wrote a blog entry about how Tom Friedman, a columnist for The New York Times, and others in the news media are helping to shape public understanding of MOOCs.

Also having their say are investors, panelists said, particularly those based in Silicon Valley.

“Higher education researchers will never move as fast as private capital,” said Mitchell L. Stevens, ­ director of digital research and planning at Stanford University’s graduate school of education and an associate professor there.

Of course, that’s not always a bad thing, as tech money has made bad bets, like with the Internet bubble of the late ‘90s. Even online learning has gone through previous iterations that went belly-up.

“We’ve been here before, sort of,” said Rasmussen. He specifically mentioned the relative lack of success of ventures from a decade or so ago, including Fathom, Universitas 21 and Barnes and Noble University. “These efforts represent something akin to version 1.0.”

This time around, Rasmussen predicted that major players among massive open online course providers would drift toward credentialing. Currently they are the latest iteration of open educational resources (OER), he said, but will eventually morph into a form of competency-based education. “Ultimately, that’s what MOOCs will become.”

Regardless of how digital learning develops, higher education researchers will have many vexing questions to probe along the way. Rasmussen said those include whether the technology can be used effectively at scale, if it will exacerbate inequalities and how best to develop new assessment techniques.

Stevens agreed that academics have a big role to play. He called the expansion of online learning a “moment of extraordinary intellectual opportunity.”

At play is a fundamental shift in focus from education to learning, he said. That’s because of increasing acceptance that learning can occur outside of the traditional classroom and throughout a lifetime.

Educational researchers have an opportunity to participate in the creation of this emerging “new science of learning,” Stevens said. “Fasten your seatbelts. It’s only going to go faster.”