Persona Reinforcement for Secure Programming AI Tutors: Adaptive Assistance in Action

Abstract

The world needs safe software, but producing it requires secure programming skills. Unfortunately, effectively teaching students secure coding skills remains a critical open problem. Pedagogy suggests that students acquire skills best through a combination of conceptual reasoning and practical experience. In computing, students gain this practical experience through hands-on programming exercises and projects. In the age of generative AI, modern tools, such as large language models (LLMs), often hinder effective learning by providing solutions to coding problems without engaging students in the learning process. Instead of internalizing the key concepts, students can simply copy answers without understanding, undermining the learning objectives. Unexpectedly, generative AI offers a promising opportunity to address the problem it has created, providing appropriate constraints and design. To that end, we present Secure Programming with Adaptive Reasoning Companion (SPARC), an AI-powered tutor designed to guide students through secure programming exercises, rather than directly provide solutions. Our design reinforces SPARC's tutor persona through a confluence of three techniques: (1) tailored prompt engineering, (2) a novel combination of AI techniques---coined as a learning safeguard proxy ---designed to prevent the tutor from directly providing solutions, and (3) a responsive algorithm that adapts responses to student proficiencies. We have integrated SPARC with SecureCoder, a drill-and-practice platform for secure coding skills, and evaluated its effectiveness via a pilot study. Across 120 study sessions (80 with SPARC and 40 with GPT-4o-mini), SPARC facilitated a 95% exercise completion rate compared to 80% for GPT-4o-mini, and pilot study participants demonstrated statistically higher satisfaction with SPARC's adaptability than GPT-4o-mini. Further, unlike GPT-4o-mini, all interactions with SPARC avoided providing participants with complete solutions. Finally, our study demonstrated that more than 85% of participants found SPARC's guidance to be clear, adaptive, and helpful, with 80% reporting improved understanding of secure programming concepts. Our evaluation suggests that SPARC's novel design achieves its goal of serving as a secure programming tutor. SPARC provides helpful guidance that most students found to enhance their learning experiences. As secure programming skills are vitally important, this work contributes to secure computing education by employing generative AI as an educator's ally, rather than its adversary.