A large number of Internet of Things (IoT) devices today are powered bybatteries, which are often expensive to maintain and may cause seriousenvironmental pollution. To avoid these problems, researchers have begun toconsider the use of energy systems based on energy-harvesting units for suchdevices. However, the power harvested from an ambient source is fundamentallysmall and unstable, resulting in frequent power failures during the operationof IoT applications involving, for example, intermittent speech signals and thestreaming of videos. This paper presents a deep-learning-based speech recoverysystem that reconstructs intermittent speech signals from self-powered IoTdevices. Our intermittent speech recovery system (ISR) consists of threestages: interpolation, recovery, and combination. The experimental results showthat our recovery system increases speech quality by up to 707.1%, whileincreasing speech intelligibility by up to 92.1%. Most importantly, our ISRsystem also enhances the WER scores by up to 65.6%. To the best of ourknowledge, this study is one of the first to reconstruct intermittent speechsignals from self-powered-sensing IoT devices. These promising results suggestthat even though self powered microphone devices function with weak energysources, our ISR system can still maintain the performance of mostspeech-signal-based applications.