We have developed a novel Automated Assessment Tool (AAT) facilitating formative assessments for novice programming students in a Higher Education (HE) Computer Science (CS) setting. This AAT forms an analogue of traditional in-person HE CS computer lab sessions. Traditionally teaching staff would engage with students in-person offering feedback to, and receiving feedback from, students about learning materials. Nowadays meaningful two-way communication can be difficult to achieve with individuals in larger HE CS cohorts, which may be taught in a blended way. Our novel AAT offers automatically generated feedback to students, and captures feedback from students - collated per question and per assessment to inform teaching practice. The novel AAT has been used in four semesters with student volunteers from the authors HE CS setting. A mixed-methods approach has captured qualitative and quantitative data. Key findings are:

1. Parsons problem type questions were associated with no negative feedback from students in initial or repeat experiments, unique amongst the four AAT question-types offered.

2. Students report concerns about providing free text feedback per question, but we observe this is a valuable source of information for teaching staff - if automatically collated for presentation in conjunction with other feedback types and student learning analytics.

This research informs on best practices supporting communication between novice programming students and teaching staff where an AAT is a pedagogical necessity, and promotes consideration for automatically collating student feedback, as part of AATs.
Future work should replicate and repeat the experiments conducted in other HE CS settings, globally.