Time series observations can be seen as realizations of an underlyingdynamical system governed by rules that we typically do not know. For timeseries learning tasks, we need to understand that we fit our model on availabledata, which is a unique realized history. Training on a single realizationoften induces severe overfitting lacking generalization. To address this issue,we introduce a general recursive framework for time series augmentation, whichwe call Recursive Interpolation Method, denoted as RIM. New samples aregenerated using a recursive interpolation function of all previous values insuch a way that the enhanced samples preserve the original inherent time seriesdynamics. We perform theoretical analysis to characterize the proposed RIM andto guarantee its test performance. We apply RIM to diverse real world timeseries cases to achieve strong performance over non-augmented data onregression, classification, and reinforcement learning tasks.