This study combines a large language model (LLM) with expert qualitative coding to investigate how variations in computer-supported collaborative learning (CSCL) instructional designs shape collaboration dynamics in data-centric programming. Using an automated LLM-powered labeling pipeline, we classified and quantified indicators of collaborative competency across 73 team transcripts collected from two CSCL activities designed for data science courses. This analysis revealed that instructional design variations yield statistically significant differences in how teams co-construct shared knowledge—particularly in developing shared understanding of problems and solutions. Subsequent expert qualitative coding of these collaborative exchanges confirmed and enriched the LLM findings by revealing significant variations in both frequency and depth of collaboration attributable to design differences. Drawing on established CSCL frameworks and prior research, we derive actionable instructional insights for calibrating scripted collaboration activities in the context of data-centric programming, with the potential to inform similar designs in broader collaborative programming tasks. In addition, this work demonstrates a replicable methodology whereby learning scientists can effectively combine LLM-based automation with human qualitative analysis to evaluate and enhance CSCL in computer science education.

Fri 20 Feb

Displayed time zone: Central Time (US & Canada) change

13:40 - 15:00
From Unplugged Activities to LLM Insights: Rethinking How We Teach Intro Computing CoursesPapers at Meeting Room 275
Chair(s): Miranda Parker
13:40
20m
Talk
CS Unplugged in Gateway Computing Courses: A Collaborative, Active Learning Approach in Introductory Computing
Papers
Hyesung Park Georgia Gwinnett College, Evelyn Brannock Georgia Gwinnett College, Tacksoo Im Georgia Gwinnett College, Wei Jin Georgia Gwinnett College, Sunae Shin Georgia Gwinnett College, Xin Xu Georgia Gwinnett College, David Kerven Georgia Gwinnett College
14:00
20m
Talk
Deriving Instructional Insights from Human–LLM Co-Evaluation of Student Collaboration in Data-Centric ProgrammingGlobal
Papers
Marshall An Carnegie Mellon University, Christine Kwon Carnegie Mellon University, Yoonjae Lee Seoul National University, Ji-Hyeon Hur Seoul National University, Dongho LEE Dalhousie University, Vincent Huai Carnegie Mellon University, Barry Zheng Carnegie Mellon University, Matthew Yu Carnegie Mellon University, Joana Liu Carnegie Mellon University, Jenny Pugh Carnegie Mellon University, Gahgene Gweon Graduate School of Convergence Science and Technology, Seoul National University, John Stamper Carnegie Mellon University
14:20
20m
Talk
Repetition Meets Context: Teaching CS1 Through Two Scientific DomainsGlobal
Papers
Meiying Qin York University, Jade Atallah York University, Hovig Kouyoumdjian York University, Jonatan Schroeder York University, Larry Yueli Zhang York University, May Haidar York University
14:40
20m
Talk
Systems for Scaling Accessibility Efforts in Large Computing Courses
Papers
Ritesh Kanchi Harvard University, Miya Natsuhara University of Washington, Seattle, Matt Wang University of Washington
Media Attached