Artificial neural networks (ANNs) have become an important tool for imageclassification with many applications in research and industry. However, itremains largely unknown how relevant image features are selected and how dataproperties affect this process. In particular, we are interested whether theabstraction level of image cues correlating with class membership influencesfeature selection. We perform experiments with binary images that contain acombination of cues, representing two different levels of abstractions: one isa pattern drawn from a random distribution where class membership correlateswith the statistics of the pattern, the other a combination of symbol-likeentities, where the symbolic code correlates with class membership. When thenetwork is trained with data in which both cues are equally significant, weobserve that the cues at the lower abstraction level, i.e., the pattern, islearned, while the symbolic information is largely ignored, even in networkswith many layers. Symbol-like entities are only learned if the importance oflow-level cues is reduced compared to the high-level ones. These findings raiseimportant questions about the relevance of features that are learned by deepANNs and how learning could be shifted towards symbolic features.