Abstract
This paper investigates federated learning (FL) in a multi-hop communicationsetup, such as in constellations with inter-satellite links. In this setup,part of the FL clients are responsible for forwarding other client's results tothe parameter server. Instead of using conventional routing, the communicationefficiency can be improved significantly by using in-network model aggregationat each intermediate hop, known as incremental aggregation (IA). Prior works[1] have indicated diminishing gains for IA under gradient sparsification. Herewe study this issue and propose several novel correlated sparsification methodsfor IA. Numerical results show that, for some of these algorithms, the fullpotential of IA is still available under sparsification without impairingconvergence. We demonstrate a 15x improvement in communication efficiency overconventional routing and a 11x improvement over state-of-the-art (SoA) sparseIA.