References & Citations
Computer Science > Machine Learning
Title: Decentralized Personalized Federated Learning based on a Conditional Sparse-to-Sparser Scheme
(Submitted on 24 Apr 2024 (v1), last revised 25 Apr 2024 (this version, v2))
Abstract: Decentralized Federated Learning (DFL) has become popular due to its robustness and avoidance of centralized coordination. In this paradigm, clients actively engage in training by exchanging models with their networked neighbors. However, DFL introduces increased costs in terms of training and communication. Existing methods focus on minimizing communication often overlooking training efficiency and data heterogeneity. To address this gap, we propose a novel \textit{sparse-to-sparser} training scheme: DA-DPFL. DA-DPFL initializes with a subset of model parameters, which progressively reduces during training via \textit{dynamic aggregation} and leads to substantial energy savings while retaining adequate information during critical learning periods.
Our experiments showcase that DA-DPFL substantially outperforms DFL baselines in test accuracy, while achieving up to $5$ times reduction in energy costs. We provide a theoretical analysis of DA-DPFL's convergence by solidifying its applicability in decentralized and personalized learning. The code is available at:this https URL
Submission history
From: Qianyu Long [view email][v1] Wed, 24 Apr 2024 16:03:34 GMT (1487kb,D)
[v2] Thu, 25 Apr 2024 08:53:57 GMT (1487kb,D)
Link back to: arXiv, form interface, contact.