Investigating the Impact of Codio Coach on Student Engagement

This research article investigates the impact of Codio Coach, a specialized AI learning assistant, on student engagement and performance in asynchronous computer science courses. The study utilizes Large Language Models (LLMs) to enhance learning by providing immediate, personalized feedback without giving direct answers. Codio Coach consists of three modules: Summarizer, Error Explanation, and Hints, aimed at facilitating independent problem-solving. The analysis indicates a significant increase in assistant usage and improved student performance, with a 12% rise in Mean Grade and a 15% increase in Median Grade for assignments where the assistant was utilized. Additionally, a low error event rate of 0.12% suggests that AI tools may contribute to more effective programming environments. The findings support the efficacy of tailored AI learning assistants in enhancing engagement and performance in computing education, recommending that educators encourage students to leverage such tools to improve learning outcomes.