What Happens When Students Leave Office Hours? Measuring Post-Interaction Code Progress in CS2 Projects
When students leave office hours, what do they do next? This study introduces preliminary work on a behavior-based measure of help-seeking success by exploring the relationship between office hour (OH) interactions and subsequent coding progress. We ask: Can students’ code changes following help-seeking interactions serve as evidence of forward progress?
To evaluate this, we examined the first GitHub commit made within a post-interaction window, defined as the earlier of the next OH interaction or twelve hours. Each commit was compared with the prior commit across three indicators: reduction of static analysis errors, increase in teaching staff test correctness, and increase in statement coverage. These indicators capture forward or negative progress, and we aggregated them per interaction to estimate if the help-seeking interaction contributed to progress.
Analyzing 1806 OH interactions over six semesters, we found only 501 (27.7%) contained project changes within the post-interaction window. Of those interactions, 170 (33.9%) led to forward progress on the first commit, 88 (17.6%) led to regression, and 243 (48.5%) resulted in no change in the three progress indicators. Of the 501 first commits, 27.7% were made between 30 min - 2hr, and observed the most forward progress and the least negative progress.
This poster presents the first step of a broader study. Future work will extend the approach to asynchronous help-seeking (e.g., online forums) to evaluate its impact on project progress. By linking instructional help with observable programming outcomes, this study provides a practical measure of help-seeking success for evaluating future interventions.