Memorising vocabulary is an important aspect of formal foreign language learning. Advances in cognitive psychology have led to the development of adaptive learning systems that make vocabulary learning more efficient. These computer-based systems measure learning performance in real time to create optimal repetition schedules for individual learners. While such adaptive learning systems have been successfully applied to written word learning, they have thus far seen little application in spoken word learning. Here we present a system for adaptive, speech-based word learning using a modified adaptive model that was originally developed for typing-based word learning. We show that typing- and speech-based learning result in similar behavioral patterns that can be used to reliably estimate individual memory processes and we show that it is possible to improve the efficiency of speech-based learning using the adaptive model. Our work provides a basis for the development of language learning applications that use real-time pronunciation assessment software to score the accuracy of the learner's pronunciations. The development of speech-based learning applications is important for two reasons. First, by focusing on speech, the model can be applied for individuals who lack the opportunity or ability to type - as is demonstrated by the successful application of the model in an elderly participant population. Second, speech-based learning models are educationally relevant because they focus on what may be the most important aspect of language learning: to practice speech. In this OSF-repository, the experiment data, analyses scripts and supplementary materials can be found.