Graduate student members of Stanford’s interdisciplinary Lytics Lab offer updates on a diverse set of education research projects in progress
Just three years old, the Stanford Lytics Lab is an open, interdisciplinary research community of graduate students who share a passion for advancing our understanding of learning through the use of digital data and technology.
Lab members have already generated an impressive body of research, and recently held an open house at their home base, the Barnum Hub for Digital Learning Research in the middle of campus, to share updates on their current works in progress.
The event featured nine projects that collectively showcased the range and depth of questions being asked in the lab – from how to improve student performance in Massive Open Online Courses (MOOCs), to what virtual reality can contribute to our understanding of how we learn and whether it’s possible to measure and influence stereotype threat and growth mindsets among learners and instructors.
The Lytics Lab had its genesis in the fall of 2012, when a number of Stanford faculty began offering MOOCs that quickly signed up hundreds of thousands of learners and garnered worldwide attention. Nearer to home, five Stanford doctoral students interested in technology and learning – Emily Schneider (Education), René Kizilcec (Communication), Sherif Halawa (Electrical Engineering), Chris Piech and Chinmay Kulkarni (Computer Science) – were also paying attention and started meeting informally to talk about how they might use the data streams generated by these online courses to further their own research.
“I sat in on a couple of these meetings and what was so thrilling about them was that education researchers were thinking about data and computational technologies that they’d never had experience with, and computer scientists were thinking about educational questions that they didn’t have experience with,” recalled Lytics Lab faculty co-director Mitchell Stevens and professor of Education, in welcoming guests to the open house.
The group quickly expanded and began working with education professor Roy Pea and now-Vice Provost of Teaching and Learning John Mitchell to facilitate ongoing access to Stanford’s MOOC data for research. Within a year, under the direction of John Mitchell and Mitchell Stevens, Lytics -- a playful contraction of “learning analytics”-- became a formal credit-bearing seminar with the aim of supporting students’ ongoing research interests in areas that extended well beyond online teaching and learning.
Today Lytics hosts researchers with backgrounds in economics, education, engineering, communication, computer science, neuroscience, and psychology, noted current co-director and professor of education Candace Thille in introducing representatives from each of the featured projects. “What we try and do at Lytics,” she said, “is create a shared vocabulary so that everyone can talk with each other.”
The result has been a rare kind of academic alchemy, added Stevens. It’s exceptional, he said, “to have these very different kinds of scientists who are thinking through a whole new kind of empirical information jointly and co-equally.”
Building better MOOCs
As evidenced by three of the featured projects, MOOCs remain both a rich source of research data and a valuable tool for unearthing insights into how learning occurs and how it might happen more reliably.
René Kizilcec is interested in making online learning a more successful experience for more users, especially those who sign up from less developed countries. At present, even though learners in poorer nations are likely to benefit more from completing a MOOC than students from OECD countries, they significantly underperform OECD participants. Together with colleagues Andy Saltarelli and Geoffrey Cohen, Kizilcec has been looking at one particularly promising avenue for addressing this disparity: online exercises in self-affirmation and social belonging. These have been shown to impact achievement gaps between white and minority US college students and between men and women in US engineering programs.
“We’re adapting the interventions and scaling them to the massive scale of MOOCs,” Kizilcec explained to a group of open house guests, “and seeing if we can close the achievement gaps between students from the more and less developed countries. So far the results are looking positive.”
Another approach to improving student success online has intrigued education PhD candidate Betsy Williams. Her research explores the impact of team dynamics on achievement in MOOCs that require learners to work in groups. “In general, the more socially connected you are in a class, the more likely you are to finish,” she said, noting that this has implications for future course design. “We think this is a way that we can scale up MOOCs, so that even if people get less time with their instructor, they can get lots of quality feedback from their peers.”
Learners’ online class experiences are also affected by the degree to which they both seek help and take part in discussions, reported Iris Howley, who joined the Lytics Lab from Carnegie Mellon University just this month. “What we’ve found is that using up and down voting in discussion forums had a negative impact on help seeking, but using ‘help giver’ badges alleviated this negative effect,” she recounted. The results are part of Howley’s Ph.D. thesis and suggest other productive avenues to pursue at Stanford. For example, she said, “We actually have a social recommendation algorithm that pairs learners with potential helpers, and we’ve never looked at the system in the longer term to see how initial interactions affect later interactions.”
Enhancing the hybrid experience
Educational technologies have the potential to impact more than online classes. Emily Schneider has been investigating how software designed to support hybrid classroom instruction can improve the learning experience and teaching opportunities for instructors.
“I’ve had the opportunity to explore that question really deeply with a platform called Lacuna Stories, which is a shared reading platform used in humanities classes here on campus and with a few other university collaborators,” she explained. The platform allows students to discuss texts with each other and offers instructors data about those conversations so they can understand more about where students are in their reading process. Digging into the data with colleague Stacy Hartman and other collaborators in Stanford’s Poetic Media Lab (which created Lacuna Stories), Schneider found that the platform’s use varied between classes, which itself had an measurable impact. “When the platform was really deeply integrated, the social experience of reading online was very different,” she reported. “Faculty were able to understand where students were in their reading process texts significantly better, and really bring that knowledge into the classroom and build on those conversations.”
Another technology with great potential for aiding learning is virtual reality. Simply by wearing a VR headset, users generate information about what they have paid attention to as they’ve navigated the imaginary space. “We’re interested in using that to look at what students have actually learned,” said communications Ph.D. student Andrea Stevenson Won. Working with Brian Perone, Jeremy Bailenson, and Roy Pea, Won has designed a virtual reality experience that takes learners in Stanford professor Fiorenza Micheli’s ocean studies class on a “field trip” through a coral reef to learn about the effects of ocean acidification. “We’re finding that we can use the non-verbal behavior that we capture in these environments to help assess different kinds of engagement and how it related to learning,” Won said.
Petr Johanes, meanwhile, is looking to digital technologies to help validate the potential of a long popular but little studied form of instruction, the graphic novel. Over the last year and a half, working with colleagues Alberto Salleo, Colin Reeves-Fortney, Andy Saltarelli, M. F. Kiyono, Johannes has been creating an online graphic novel that uses science and storytelling to convey the principals behind thermodynamics. The topic, he explained, “is foundational to most science and engineering. But there’s very little research on how best to teach it.” Johanes’ hypothesis is that a graphic novel can do it well, especially if it’s created and presented in digital form, where reader use can be tracked and updates added to sections that students persistently have trouble understanding.
Widely applicable insights about teaching and learning
Although all Lytics Labs studies use digital data as evidence from which to draw conclusions about teaching and learning, many focus on questions that, if answered, will have implications for any kind of instruction.
Take the bane of many a parent and child’s life: endless-seeming sets of math homework problems. At some point, even children who find math easy can view these long sets as a frustrating burden that robs time from other valuable activities. “Wouldn’t it be great if we could measure how much repetition is necessary to reach a sufficient level of understanding?” asked Olle Bälter, a visiting fellow from Sweden’s Royal Institute of Technology. Together with Ross Strader, Candace Thille, and Dawn Zimmaro, Bälter is mining learning data from online math courses, where it’s possible to correlate the number of problems solved to levels of comprehension. The project has just started, but Bälter hopes to be able to predict how many problems are required for mastery of different kinds of questions and then test those hypotheses against student results. “I suspect,” he said, “that at present we are overdoing it, just to be sure – and because the negative consequences of doing more than necessary is not a burden on the person requiring it.”
In a similar vein, Daniel Greene showcased a study that asks how much teachers know about the notion of growth mindset – the understanding that your intelligence is malleable and open to improvement - and how they convey that idea to their students. “I’m developing an assessment of practical teacher mindset knowledge,” he reported, “and testing a tool that could help them convey that knowledge to students.” While the study and tool are housed online, the impact is felt in the classroom. The research platform and methodology might also have applicability beyond growth mindset research to better understand and impact other beliefs held by teachers about learning that, if changed, could improve student performance.
Sometimes it takes a meeting at the Lab to spur a particular line of research. Michelle Friend studies computer science education and gender equity and had collected two sets of words that girls enrolled in a summer program in computer science used to describe both themselves and computer scientists, one set before and one after they had completed the program.
“I had no idea what to do with these,” Friend recalled. “So I presented at Lytics, and Chris Piech, a computer scientist, and Rob Semmens, who is a statistics guy, said let’s work together.” They then compared the overlap between those paired sets and used sentiment analysis to determine how positive or negative the words were. Piech extended this to create a ‘stereotype analysis’ algorithm to show how much the girls' descriptions of computer scientists deviated from the common (typically male and nerdy) stereotype of a computer scientist by the end of the program. Gaining direct experience with computing, they found, led to girls being more positive towards computer science and to describe computer scientists less stereotypically.
A call for more collaborators
Lytics Labs graduate students are far from the only researchers on the Stanford campus interested in how digitally-derived data can help inform our ideas about teaching and learning, a point reinforced by co-director Mitchell Stevens when he invited other researchers to collaborate and share their work at the next open house.
His appeal was echoed by colleague, Candace Thille. Thille left Carnegie Mellon to come to Stanford in 2013. “A big part of that decision was Lytics,” she suggested. “When I came to visit, I had the opportunity to sit in on a Lytics Lab meeting and my reaction was, “oh my, this is where it’s happening.””
“Lytics has continued to be an inter-disciplinary, open research community,” Thille added, “but we need and want more researchers from different perspectives and different disciplines to join us.”