Objectives: The traditional, instructor-led form of live coding has been extensively studied, with findings showing that this form of live coding imparts similar learning to static-code examples. However, a concern with Traditional Live Coding is that it can turn into a passive learning activity for students as they simply observe the instructor program. Therefore, this study compares Active Live Coding—a form of live coding that leverages in-class coding activities and peer discussion—to Traditional Live Coding on three outcomes: (1) students’ adherence to effective programming processes, (2) students’ performance on exams and in-lecture questions, and (3) students’ lecture experience.
Participants: Roughly 530 students were enrolled in an advanced, CS1 course taught in Java at a large, public university in North America. The students were primarily first- and second-year undergraduate students with some prior programming experience. The student population was spread across two lecture sections—348 students in the Active Live Coding (ALC) lecture and 185 students in the Traditional Live Coding (TLC) lecture.
Study Methods: We used a mixed-methods approach to answer our‘ research questions. To compare students’ programming processes, we applied process-oriented metrics related to incremental development and error frequencies. To measure students’ learning outcomes, we compared students’ performance on major course components and used pre- and post-lecture questionnaires to compare students’ learning gain during lectures. Finally, to understand students’ lecture experience, we used a classroom observation protocol to measure and compare students’ behavioral engagement during the two lectures. We also inductively coded open-ended survey questions to understand students’ perceptions of live coding.
Findings: We did not find a statistically significant effect of ALC on students’ programming processes or learning outcomes. It seems that both ALC and TLC impart similar programming processes and result in similar student learning. However, our findings related to students’ lecture experience shows a persistent engagement effect of ALC, where students’ behavioral engagement peaks and remains elevated after the in-class coding activity and peer discussion. Finally, we discuss the unique affordances and drawbacks of the lecture technique as well as students’ perceptions of ALC.
Conclusions: Despite being motivated by well-established learning theories, Active Live Coding did not result in improved student learning or programming processes. This study is preceded by several prior works that showed that Traditional Live Coding imparts similar student learning and programming skills as static-code examples. Though potential reasons for the lack of observed learning benefits are discussed in this work, multiple future analyses to further investigate Active Live Coding may help the community understand the impacts (or lack thereof) of the instructional technique.
