Accurate and fast extraction of foreground object is a key prerequisite for awide range of computer vision applications such as object tracking andrecognition. Thus, enormous background subtraction methods for foregroundobject detection have been proposed in recent decades. However, it is stillregarded as a tough problem due to a variety of challenges such as illuminationvariations, camera jitter, dynamic backgrounds, shadows, and so on. Currently,there is no single method that can handle all the challenges in a robust way.In this letter, we try to solve this problem from a new perspective bycombining different state-of-the-art background subtraction algorithms tocreate a more robust and more advanced foreground detection algorithm. Moreconcretely, a encoder-decoder fully convolutional neural network architectureis trained to automatically learn how to leverage the characteristics ofdifferent algorithms to fuse the results produced by different backgroundsubtraction algorithms and output a more precise result. Comprehensiveexperiments evaluated on the CDnet 2014 dataset demonstrate that the proposedmethod outperforms all the considered single background subtraction algorithm.And we show that our solution is more efficient than other combinationstrategies.