The paper presents a Traffic Sign Recognition (TSR) system, which can fastand accurately recognize traffic signs of different sizes in images. The systemconsists of two well-designed Convolutional Neural Networks (CNNs), one forregion proposals of traffic signs and one for classification of each region. Inthe proposal CNN, a Fully Convolutional Network (FCN) with a dual multi-scalearchitecture is proposed to achieve scale invariant detection. In training theproposal network, a modified "Online Hard Example Mining" (OHEM) scheme isadopted to suppress false positives. The classification network fusesmulti-scale features as representation and adopts an "Inception" module forefficiency. We evaluate the proposed TSR system and its components withextensive experiments. Our method obtains $99.88\%$ precision and $96.61\%$recall on the Swedish Traffic Signs Dataset (STSD), higher thanstate-of-the-art methods. Besides, our system is faster and more lightweightthan state-of-the-art deep learning networks for traffic sign recognition.