Visual complexity identifies the level of intricacy and details in an imageor the level of difficulty to describe the image. It is an important concept ina variety of areas such as cognitive psychology, computer vision andvisualization, and advertisement. Yet, efforts to create large, downloadableimage datasets with diverse content and unbiased groundtruthing are lacking. Inthis work, we introduce Savoias, a visual complexity dataset that compromisesof more than 1,400 images from seven image categories relevant to the aboveresearch areas, namely Scenes, Advertisements, Visualization and infographics,Objects, Interior design, Art, and Suprematism. The images in each categoryportray diverse characteristics including various low-level and high-levelfeatures, objects, backgrounds, textures and patterns, text, and graphics. Theground truth for Savoias is obtained by crowdsourcing more than 37,000 pairwisecomparisons of images using the forced-choice methodology and with more than1,600 contributors. The resulting relative scores are then converted toabsolute visual complexity scores using the Bradley-Terry method and matrixcompletion. When applying five state-of-the-art algorithms to analyze thevisual complexity of the images in the Savoias dataset, we found that thescores obtained from these baseline tools only correlate well with crowdsourcedlabels for abstract patterns in the Suprematism category (Pearson correlationr=0.84). For the other categories, in particular, the objects and advertisementcategories, low correlation coefficients were revealed (r=0.3 and 0.56,respectively). These findings suggest that (1) state-of-the-art approaches aremostly insufficient and (2) Savoias enables category-specific methoddevelopment, which is likely to improve the impact of visual complexityanalysis on specific application areas, including computer vision.