Abstract
Unstructured information in electronic health records provide an invaluableresource for medical research. To protect the confidentiality of patients andto conform to privacy regulations, de-identification methods automaticallyremove personally identifying information from these medical records. However,due to the unavailability of labeled data, most existing research isconstrained to English medical text and little is known about thegeneralizability of de-identification methods across languages and domains. Inthis study, we construct a varied dataset consisting of the medical records of1260 patients by sampling data from 9 institutes and three domains of Dutchhealthcare. We test the generalizability of three de-identification methodsacross languages and domains. Our experiments show that an existing rule-basedmethod specifically developed for the Dutch language fails to generalize tothis new data. Furthermore, a state-of-the-art neural architecture performsstrongly across languages and domains, even with limited training data.Compared to feature-based and rule-based methods the neural method requiressignificantly less configuration effort and domain-knowledge. We make all codeand pre-trained de-identification models available to the research community,allowing practitioners to apply them to their datasets and to enable futurebenchmarks.