Abstract
Developing narrative and comprehension skills in early childhood is criticalfor later literacy. However, teachers in large preschool classrooms struggle toaccurately identify students who require intervention. We present a system forautomatically assessing oral narratives of preschool children in Afrikaans andisiXhosa. The system uses automatic speech recognition followed by a machinelearning scoring model to predict narrative and comprehension scores. Forscoring predicted transcripts, we compare a linear model to a large languagemodel (LLM). The LLM-based system outperforms the linear model in most cases,but the linear system is competitive despite its simplicity. The LLM-basedsystem is comparable to a human expert in flagging children who requireintervention. We lay the foundation for automatic oral assessments inclassrooms, giving teachers extra capacity to focus on personalised support forchildren's learning.