Class imbalance in Federated Learning

Xiao, Chenguang (2025). Class imbalance in Federated Learning. University of Birmingham. Ph.D.

Xiao2025PhD.pdf
Text - Accepted Version
Available under License All rights reserved.
Download (18MB)

Abstract

Federated Learning (FL) is a distributed and privacy protected machine learning paradigm that trains a global model across multiple clients without gathering their local data. Class imbalance is a common issue in machine learning, where the number of samples in each class is not balanced. In the field of federated learning, class imbalance is one of the key challenges that undermine the performance of the global model. Concentrating on the class imbalance problems in FL, this thesis investigates the scenarios and corresponding impacts, explores the mechanisms behind the issues, and proposes effective solutions from different perspectives. At the beginning, two metrics are proposed to quantify the issue from the global and local perspectives. After that, a novel resampling methods utilizing both minority and majority classes are proposed to address the class imbalance in distributed settings. The proposed methods are robust and effective in severe class imbalance scenarios and small datasets. Worse case in FL is the inter-client class imbalance, which refers to the class distribution mismatch between clients, sometimes also known as data heterogeneity. This mismatch results in catastrophic forgetting in local training on the minority class, slowing down the convergence of the global model. By analysing back-propagation with class imbalance, Error Asymmetry is identified as the root cause of catastrophic forgetting. By connecting the Error Asymmetry with gradient, a novel gradient alignment method is proposed to mitigate the issue. Furthermore, advanced optimization techniques show promising results in addressing the class imbalance in federated learning. A new momentum based optimization algorithm is proposed to improve the speed-up the training by applying a reverse exponential moving average on the gradient. Experiments show the effectiveness of the proposed methods in addressing the data heterogeneity.

Type of Work:

Thesis (Doctorates > Ph.D.)

Award Type:

Doctorates > Ph.D.

Supervisor(s):