The majority of descriptor-based methods for geometric processing ofnon-rigid shape rely on hand-crafted descriptors. Recently, learning-basedtechniques have been shown effective, achieving state-of-the-art results in avariety of tasks. Yet, even though these methods can in principle work directlyon raw data, most methods still rely on hand-crafted descriptors at the inputlayer. In this work, we wish to challenge this practice and use a neuralnetwork to learn descriptors directly from the raw mesh. To this end, weintroduce two modules into our neural architecture. The first is a localreference frame (LRF) used to explicitly make the features invariant to rigidtransformations. The second is continuous convolution kernels that providerobustness to sampling. We show the efficacy of our proposed network inlearning on raw meshes using two cornerstone tasks: shape matching, and humanbody parts segmentation. Our results show superior results over baselinemethods that use hand-crafted descriptors.