In this paper, a simple and efficient Hybrid Classifier is presented which isbased on deep learning and reinforcement learning. Q-Learning has been usedwith two states and 'two or three' actions. Other techniques found in theliterature use feature map extracted from Convolutional Neural Networks and usethese in the Q-states along with past history. This leads to technicaldifficulties in these approaches because the number of states is high due tolarge dimensions of the feature map. Because our technique uses only twoQ-states it is straightforward and consequently has much lesser number ofoptimization parameters, and thus also has a simple reward function. Also, theproposed technique uses novel actions for processing images as compared toother techniques found in literature. The performance of the proposed techniqueis compared with other recent algorithms like ResNet50, InceptionV3, etc. onpopular databases including ImageNet, Cats and Dogs Dataset, and Caltech-101Dataset. Our approach outperforms others techniques on all the datasets used.