Examining Discourse in a Large Online Education Program: A Machine-in-the-Loop ApproachOnline
This paper explores the discourse surrounding large online education programs by focusing on a Computer Science Masters program offered by an R1 research university in the United States. Leveraging a machine-in-the-loop (MITL) approach, the authors combine traditional qualitative methods with computational techniques, including semantic embedding, natural language processing, and unsupervised learning, to extract and analyze discourse from 14 social media platforms. The research identifies various categories of discourse from Program Administration, Structure, and Outcomes, Reputation and Rigor, Interactions and Peer Learning, to Emerging and Niche Discussions. The paper reveals insights into public perceptions, motivations, and concerns related to online education, such as the importance of career outcomes, program flexibility, academic rigor, and community building. The study also demonstrates the value of MITL approaches by integrating large language models (LLMs) into qualitative research to efficiently analyze large datasets and by using semantic differentiation to uncover nuanced discourse for further analysis. The methods and results have implications for educators, curriculum designers, administrators, and future research in online education, highlighting the need for institutions to engage with online communities and monitor public discourse to enhance program perception, quality and impact.