The Generative Adversarial Networks (GANs) have demonstrated impressiveperformance for data synthesis, and are now used in a wide range of computervision tasks. In spite of this success, they gained a reputation for beingdifficult to train, what results in a time-consuming and human-involveddevelopment process to use them. We consider an alternative training process, named SGAN, in which severaladversarial "local" pairs of networks are trained independently so that a"global" supervising pair of networks can be trained against them. The goal isto train the global pair with the corresponding ensemble opponent for improvedperformances in terms of mode coverage. This approach aims at increasing thechances that learning will not stop for the global pair, preventing both to betrapped in an unsatisfactory local minimum, or to face oscillations oftenobserved in practice. To guarantee the latter, the global pair never affectsthe local ones. The rules of SGAN training are thus as follows: the global generator anddiscriminator are trained using the local discriminators and generators,respectively, whereas the local networks are trained with their fixed localopponent. Experimental results on both toy and real-world problems demonstrate thatthis approach outperforms standard training in terms of better mitigating modecollapse, stability while converging and that it surprisingly, increases theconvergence speed as well.