Abstract

Image classification models built into visual support systems and otherassistive devices need to provide accurate predictions about their environment.We focus on an application of assistive technology for people with visualimpairments, for daily activities such as shopping or cooking. In this paper,we provide a new benchmark dataset for a challenging task in this application -classification of fruits, vegetables, and refrigerated products, e.g. milkpackages and juice cartons, in grocery stores. To enable the learning processto utilize multiple sources of structured information, this dataset not onlycontains a large volume of natural images but also includes the correspondinginformation of the product from an online shopping website. Such informationencompasses the hierarchical structure of the object classes, as well as aniconic image of each type of object. This dataset can be used to train andevaluate image classification models for helping visually impaired people innatural environments. Additionally, we provide benchmark results evaluated onpretrained convolutional neural networks often used for image understandingpurposes, and also a multi-view variational autoencoder, which is capable ofutilizing the rich product information in the dataset.

 

Introduction (beta)

None

 

Conclusion (beta)

None