Exploring Student Choice and the Use of Multimodal Generative AI in Programming Learning
This program is tentative and subject to change.
The broad adoption of Generative AI (GenAI) is impacting Computer Science education, and recent studies found its benefits and potential concerns when students use it for programming learning. However, most existing explorations focus on GenAI tools that primarily support text-to-text interaction. With recent developments, GenAI applications have begun supporting multiple modes of communication, known as multimodality. In this work, we explored how undergraduate programming novices choose and work with multimodal GenAI tools, and their criteria for choices. We selected a commercially available multimodal GenAI platform for interaction, as it supports multiple input and output modalities, including text, audio, image upload, and real-time screen-sharing. Through 16 think-aloud sessions that combined participant observation with follow-up semi-structured interviews, we investigated student modality choices for GenAI tools when completing programming problems and the underlying criteria for modality selections. With multimodal communication emerging as the future of AI in education, this work aims to spark continued exploration on understanding student interaction with multimodal GenAI in the context of CS education.
This program is tentative and subject to change.
Thu 19 FebDisplayed time zone: Central Time (US & Canada) change
10:40 - 12:00 | |||
10:40 20mTalk | Capturing Student Reasoning with Low-Cost AI: An Early Experience in a Data-Structures Course Papers Kwabena Bamfo Ashesi University, Olaf Hall-Holt St. Olaf College, Oluwakemi Ola University of British Columbia, Govindha Yeluripati Ashesi University, Dennis Owusu Ashesi University | ||
11:00 20mTalk | Exploring Student Choice and the Use of Multimodal Generative AI in Programming Learning Papers Xinying Hou University of Michigan, Ruiwei Xiao Carnegie Mellon University, Runlong Ye University of Toronto, Michael Liut University of Toronto Mississauga, John Stamper Carnegie Mellon University | ||
11:20 20mTalk | If You Can’t Beat ‘Em, Conscript ‘Em: Experiences Requiring the Use of AI in a Capstone Course Papers David Levine University of Southern Maine | ||