Moving object detection (MOD) is a significant problem in computer visionthat has many real world applications. Different categories of methods havebeen proposed to solve MOD. One of the challenges is to separate moving objectsfrom illumination changes and shadows that are present in most real worldvideos. State-of-the-art methods that can handle illumination changes andshadows work in a batch mode; thus, these methods are not suitable for longvideo sequences or real-time applications. In this paper, we propose anextension of a state-of-the-art batch MOD method (ILISD) to anonline/incremental MOD using unsupervised and generative neural networks, whichuse illumination invariant image representations. For each image in a sequence,we use a low-dimensional representation of a background image by a neuralnetwork and then based on the illumination invariant representation, decomposethe foreground image into: illumination change and moving objects. Optimizationis performed by stochastic gradient descent in an end-to-end and unsupervisedfashion. Our algorithm can work in both batch and online modes. In the batchmode, like other batch methods, optimizer uses all the images. In online mode,images can be incrementally fed into the optimizer. Based on our experimentalevaluation on benchmark image sequences, both the online and the batch modes ofour algorithm achieve state-of-the-art accuracy on most data sets.