FedDance: Efficient Participant Selection for Federated Learning in Highly Dynamic Environments

Abstract

This paper introduces FedDance, an efficient and intelligent participant selection framework tailored to tackle the dynamic nature in FL. FedDance employs a lightweight stochastic prediction method to anticipate the dynamic availability of each device. To address the diminishing marginal returns from frequently selected devices due to training dynamics, especially in later training stages, FedDance explicitly quantifies the marginal return of each local training process over time. Additionally, FedDance incorporates a lightweight model to characterize data heterogeneity, enhancing the advantages of managing device and training dynamics. Extensive experiments conducted on four A100 GPUs demonstrate that FedDance can substantially outperform the state-of-the-art systems in final model accuracy and convergence speed, while also providing a considerable advantage in computational overhead.

Yuanhang Chen

2023 - Current

My research interests include federated learning systems.

Xiaosong Chen

2022 - Current

My research interest includes online algorithm and its application to cloud-edge systems.

Wenyan Chen

2021 - 2025 PhD student

My research interests include resource management and task scheduling in GPU clusters.

Huanle Xu

2021 - Current

I am currently an assistant professor from the Department of Computer and Information Scicence, Univeristy of Macau.