Light plays a vital role in vision either human or machine vision, theperceived color is always based on the lighting conditions of the surroundings.Researchers are working to enhance the color detection techniques for theapplication of computer vision. They have implemented proposed several methodsusing different color detection approaches but still, there is a gap that canbe filled. To address this issue, a color detection method, which is based on aConvolutional Neural Network (CNN), is proposed. Firstly, image segmentation isperformed using the edge detection segmentation technique to specify the objectand then the segmented object is fed to the Convolutional Neural Networktrained to detect the color of an object in different lighting conditions. Itis experimentally verified that our method can substantially enhance therobustness of color detection in different lighting conditions, and our methodperformed better results than existing methods.