Vision and language tasks such as Visual Relation Detection and VisualQuestion Answering benefit from semantic features that afford proper groundingof language. The 3D depth of objects depicted in 2D images is one such feature.However it is very difficult to obtain accurate depth information withoutlearning the appropriate features, which are scene dependent. The state of theart in this area are complex Neural Network models trained on stereo image datato predict depth per pixel. Fortunately, in some tasks, its only the relativedepth between objects that is required. In this paper the extent to whichsemantic features can predict course relative depth is investigated. Theproblem is casted as a classification one and geometrical features based onobject bounding boxes, object labels and scene attributes are computed and usedas inputs to pattern recognition models to predict relative depth. i.e behind,in-front and neutral. The results are compared to those obtained from averagingthe output of the monodepth neural network model, which represents thestate-of-the art. An overall increase of 14% in relative depth accuracy overrelative depth computed from the monodepth model derived results is achieved.