Prompting Analysis for Programming Exercises in Computer Science Courses

This study explores how different prompting strategies influence student evaluations of programming exercises in computer science education. We analyze six prompting types—Zero-Shot, Few-Shot, Chain-of-Thought, Role, Instruction, and Multi-Turn—comparing student-generated assessments with expert judgments under optimistic, neutral, and pessimistic approaches. Our findings reveal that structured prompts, particularly Chain-of-Thought and Role, lead to higher alignment with expert evaluations, especially under stricter assessment conditions. However, all strategies exhibit performance variability and low-end outliers, suggesting that prompting alone does not guarantee expert-level reasoning. These results underscore the need for adaptive prompt design that accounts for diverse student profiles and cognitive patterns.