Data Science Problem Solving (DSPS) competency refers to the ability to make key decisions when tackling real-world data challenges. As generative AI increasingly capable of automating low-level routine tasks, it becomes critical to focus on developing students’ higher-order reasoning and problem-solving skills. Despite the high demand for such competencies, training them effectively within data science courses remains a challenge. To develop students’ adaptive expertise, which enables students to apply problem-solving strategies and tactics in novel contexts, it is essential to expose them to a variety of problem-solving scenarios. Meanwhile, it is desirable to let students receive timely feedback to enhance their reflections and learning. In this paper, we present our experience piloting caselets—bite-sized case studies designed to scaffold problem-solving—in graduate-level data science courses. We describe the rationale, design and implementation of the caselets tool, analyze student performance and experience using the tool as part of their course, and reflect on the instructional design implications. Drawing from instructors’ observations and reflections, we discuss lessons learned and offer recommendations for improving and scaling caselet-based practices to better support the needs of both students and instructors.