Abstract
Accurately modeling student cognition is crucial for developing effectiveAI-driven educational technologies. A key challenge is creating realisticstudent models that satisfy two essential properties: (1) accuratelyreplicating specific misconceptions, and (2) correctly solving problems wherethese misconceptions are not applicable. This dual requirement reflects thecomplex nature of student understanding, where misconceptions coexist withcorrect knowledge. This paper investigates whether Large Language Models (LLMs)can be instruction-tuned to meet this dual requirement and effectively simulatestudent thinking in algebra. We introduce MalAlgoPy, a novel Python librarythat generates datasets reflecting authentic student solution patterns througha graph-based representation of algebraic problem-solving. Utilizing MalAlgoPy,we define and examine Cognitive Student Models (CSMs) - LLMs instruction tunedto faithfully emulate realistic student behavior. Our findings reveal that LLMstrained on misconception examples can efficiently learn to replicate errors.However, the training diminishes the model's ability to solve problemscorrectly, particularly for problem types where the misconceptions are notapplicable, thus failing to satisfy second property of CSMs. We demonstratethat by carefully calibrating the ratio of correct to misconception examples inthe training data - sometimes as low as 0.25 - it is possible to develop CSMsthat satisfy both properties. Our insights enhance our understanding ofAI-based student models and pave the way for effective adaptive learningsystems.