Abstract
Integrating expert knowledge, e.g. from large language models, into causaldiscovery algorithms can be challenging when the knowledge is not guaranteed tobe correct. Expert recommendations may contradict data-driven results, andtheir reliability can vary significantly depending on the domain or specificquery. Existing methods based on soft constraints or inconsistencies inpredicted causal relationships fail to account for these variations inexpertise. To remedy this, we propose L2D-CD, a method for gauging thecorrectness of expert recommendations and optimally combining them withdata-driven causal discovery results. By adapting learning-to-defer (L2D)algorithms for pairwise causal discovery (CD), we learn a deferral functionthat selects whether to rely on classical causal discovery methods usingnumerical data or expert recommendations based on textual meta-data. Weevaluate L2D-CD on the canonical T\"ubingen pairs dataset and demonstrate itssuperior performance compared to both the causal discovery method and theexpert used in isolation. Moreover, our approach identifies domains where theexpert's performance is strong or weak. Finally, we outline a strategy forgeneralizing this approach to causal discovery on graphs with more than twovariables, paving the way for further research in this area.