The recognition of texts existing in camera-captured images has become animportant issue for a great deal of research during the past few decades. Thisgive birth to Scene Character Recognition (SCR) which is an important step inscene text recognition pipeline. In this paper, we extended the Bag of Features(BoF)-based model using sparse stacked auto-encoder technique for representinglocal features for accurate SCR of different languages. In the features codingstep, a Sparse Auto-encoder (SAE)-based strategy was applied to enhance therepresentative and discriminative abilities of image features. This techniqueprovides more efficient features representation and therefore a betterrecognition accuracy, compared to other feature coding techniques. This codingstep was followed by a Spatial Pyramid Matching (SPM) and max-pooling to keepthe spatial information and form the global image signature. Our system wasevaluated extensively on six scene character datasets of five differentlanguages. The experimental results proved the efficiency of our system for amultilingual SCR.