Abstract
Multivariate time series forecasting (MTSF) endeavors to predict futureobservations given historical data, playing a crucial role in time series datamanagement systems. With advancements in large language models (LLMs), recentstudies employ textual prompt tuning to infuse the knowledge of LLMs into MTSF.However, the deployment of LLMs often suffers from low efficiency during theinference phase. To address this problem, we introduce TimeKD, an efficientMTSF framework that leverages the calibrated language models and privilegedknowledge distillation. TimeKD aims to generate high-quality futurerepresentations from the proposed cross-modality teacher model and cultivate aneffective student model. The cross-modality teacher model adopts calibratedlanguage models (CLMs) with ground truth prompts, motivated by the paradigm ofLearning Under Privileged Information (LUPI). In addition, we design asubtractive cross attention (SCA) mechanism to refine these representations. Tocultivate an effective student model, we propose an innovative privilegedknowledge distillation (PKD) mechanism including correlation and featuredistillation. PKD enables the student to replicate the teacher's behavior whileminimizing their output discrepancy. Extensive experiments on real data offerinsight into the effectiveness, efficiency, and scalability of the proposedTimeKD.