Abstract
We focus on a class of real-world domains, where gathering hierarchicalknowledge is required to accomplish a task. Many problems can be represented inthis manner, such as network penetration testing, targeted advertising ormedical diagnosis. In our formalization, the task is to sequentially requestpieces of information about a sample to build the knowledge hierarchy andterminate when suitable. Any of the learned pieces of information can befurther analyzed, resulting in a complex and variable action space. We presenta combination of techniques in which the knowledge hierarchy is explicitlyrepresented and given to a deep reinforcement learning algorithm as its input.To process the hierarchical input, we employ Hierarchical Multiple-InstanceLearning and to cope with the complex action space, we factor it withhierarchical softmax. Our end-to-end differentiable model is trained with A2C,a standard deep reinforcement learning algorithm. We demonstrate the method ina set of seven classification domains, where the task is to achieve the bestaccuracy with a set budget on the amount of information retrieved. Compared tobaseline algorithms, our method achieves not only better results, but alsobetter generalization.