Title:Efficient Communication in Federated Learning Using Floating-Point Lossy Compression

Abstract:In the expanding realm of machine learning (ML) within edge computing, the efficient exchange of information in federated learning (FL) environments is paramount. FL's decentralized nature often leads to significant communication bottlenecks, particularly in settings where resources are limited. Traditional data compression techniques, such as quantization and pruning, provide partial solutions but can compromise model performance or necessitate costly retraining. Our paper addresses this issue through \textit{FedSZ}, a novel lossy compression-based FL framework. \textit{FedSZ} is designed to minimize the size of local model updates without impacting model performance. Our framework features a compression pipeline integrating data partitioning, lossy and lossless model parameters, metadata compression, and efficient serialization. We conduct a thorough evaluation of \textit{FedSZ} utilizing a variety of lossy compressors, among which SZ2 emerged as the most effective, consistently performing well across diverse neural network architectures, including AlexNet, MobileNetV2, and ResNet50, and datasets such as CIFAR-10, Caltech101, and FMNIST. A relative error bound of 1E-2 balances compression and data integrity, achieving compression ratios ranging from $5.55\mbox{--}12.61\times$. Furthermore, we observed that the runtime overhead introduced by \textit{FedSZ} is minimal, at less than $4.7\%$, compared to a significant reduction in network transfer times, which we noted to exceed $13.3\times$ reduction or saving of over $100$s in edge networks operating at 10Mbps. Our findings firmly establish the efficacy of \textit{FedSZ}, offering valuable insights for achieving an optimal balance between communication efficiency and model performance in FL settings, particularly in edge computing environments.