Abstract
Federated learning is gaining increasing popularity, with data heterogeneityand privacy being two prominent challenges. In this paper, we address bothissues within a federated transfer learning framework, aiming to enhancelearning on a target data set by leveraging information from multipleheterogeneous source data sets while adhering to privacy constraints. Werigorously formulate the notion of \textit{federated differential privacy},which offers privacy guarantees for each data set without assuming a trustedcentral server. Under this privacy constraint, we study three classicalstatistical problems, namely univariate mean estimation, low-dimensional linearregression, and high-dimensional linear regression. By investigating theminimax rates and identifying the costs of privacy for these problems, we showthat federated differential privacy is an intermediate privacy model betweenthe well-established local and central models of differential privacy. Ouranalyses incorporate data heterogeneity and privacy, highlighting thefundamental costs of both in federated learning and underscoring the benefit ofknowledge transfer across data sets.