Abstract
Aiding humans with scientific designs is one of the most exciting ofartificial intelligence (AI) and machine learning (ML), due to their potentialfor the discovery of new drugs, design of new materials and chemical compounds,etc. However, scientific design typically requires complex domain knowledgethat is not familiar to AI researchers. Further, scientific studies involveprofessional skills to perform experiments and evaluations. These obstaclesprevent AI researchers from developing specialized methods for scientificdesigns. To take a step towards easy-to-understand and reproducible research ofscientific design, we propose a benchmark for the inverse design ofnanophotonic devices, which can be verified computationally and accurately.Specifically, we implemented three different nanophotonic design problems,namely a radiative cooler, a selective emitter for thermophotovoltaics, andstructural color filters, all of which are different in design parameterspaces, complexity, and design targets. The benchmark environments areimplemented with an open-source simulator. We further implemented 10 differentinverse design algorithms and compared them in a reproducible and fairframework. The results revealed the strengths and weaknesses of existingmethods, which shed light on several future directions for developing moreefficient inverse design algorithms. Our benchmark can also serve as thestarting point for more challenging scientific design problems. The code ofIDToolkit is available at https://github.com/ThyrixYang/IDToolkit.