Evaluating LLaMA LLMs and Prompt Engineering for Educational Applications
This poster discusses the evaluation of prompt engineering strategies in large language models (LLMs), through an analysis of response accuracy, reproducibility, and latency across multiple model variants. The study included 174 prompt–response trials across ten categories, including logic, mathematics, and language interpretation, administered to three versions of Meta’s Llama models (1B, 3B, and 8B parameters). Each trial measured response accuracy, reproducibility, and latency (using Python code to systematically quantify the latter two) to assess how model scale influences both output quality and computational efficiency. Results reveal distinct scaling patterns: the Llama 8B model achieved the highest accuracy (75.86%) and reproducibility (0.38) but demonstrated longer latency (24.27 seconds on average), whereas smaller models responded faster but with lower accuracy (24.14% for 1B and 44.83% for 3B). Across all models, the average accuracy was 48.28%, reproducibility 0.35, and mean latency 14.78 seconds. These findings highlight a measurable trade-off between reasoning reliability and computational speed. The significance of this work extends to computing education, where LLMs are increasingly integrated as teaching, tutoring, and assessment tools. Understanding how prompt design and model scale affect the consistency of generated responses is critical for developing trustworthy and transparent educational AI systems. By analyzing LLM behavior across structured prompts, this study provides insight into how educators and developers can optimize model interactions for classroom learning, automated feedback, and equitable evaluation.