Code comprehension is an essential ability for Computer Science students, providing a solid foundation for learning programming. An effective approach to evaluating students’ proficiency in this skill is through Explain-in-Plain-English (EiPE) problems, which require students to articulate the behavior of code snippets. Recent advances in Large Language models (LLMs) have made promising strides toward making autograding EiPE questions feasible. However, prior research has primarily focused on using proprietary LLMs, raising concerns over data privacy. In an effort to return autonomy over educational data to instructors and institutions, we investigate the viability of open-source Medium-Sized Language Models (MLMs), defined as having parameter counts ranging from six billion to 100 billion, for EiPE autograding. Our work evaluated several state-of-the-art open-source MLMs on a test set consisting of 620 historical student responses split across 17 EiPE question categories, employing few-shot prompting with three correct and incorrect examples per question. We find several models, such as Llama 3.1 70B Instruct and Qwen 2.5 72B Instruct, that achieve grading accuracy comparable to leading proprietary models like GPT-4o. These results demonstrate that larger open-source MLMs are promising alternatives for EiPE autograding capable of deployment on local or institution-owned cloud infrastructure. Additionally, we observe that smaller open-source MLMs offer a trade-off between significantly reduced deployment costs and only slightly decreased grading accuracy, making them well-suited for institutions with limited computational resources.