Improving federated learning in heterogeneous wireless networks

UNCG Author/Contributor (non-UNCG co-authors, if there are any, appear on document)
Shipra Shanu (Creator)
Institution
The University of North Carolina at Greensboro (UNCG )
Web Site: http://library.uncg.edu/
Advisor
Jing Deng

Abstract: The next era of privacy preserving machine learning is built upon the basic principle centered around data privacy. Due to data privacy being the primary concern among data owners, there is an increasing interest in a fast-growing machine learning technique called Federated Learning (FL) [1], in which the model training is distributed over mobile user equipment, exploiting user equipment local computation and training data. Federated learning has emerged rapidly and has been used widely for creating privacy-preserving models by building local models privately. This new sought-out technology does not require uploading one’s own private data to a central server to train the models. Rather, computations are moved closer to the data, i.e., a globally shared model is brought to where the data resides. Models are moved towards the device and this allows the models to be collectively trained as a whole. FL has become an answer for working out all the conflicts among data sharing requisite and data privacy concerns, as it sends the models to the data not the other way around. The concept of FL is beneficial in wireless networks where it plays an effective role in training FL models on devices like mobile phones and IoT (Internet Of Things). To describe FL in simpler words, it is an approach that allows users to train a neural network model without holding client data physically at their locations. It is a type of machine learning where we do not centralize all the data on a single server. The concept of FL has made possible to collaborate data from heterogeneous sources, train the model local at the data location. There are multiple advantages of having the data reside in silos and still various types of analysis can be performed securely. Despite its many advantages, FL comes with an ample amount of challenges that need to be addressed for effective training and testing such as Statistical and System heterogeneity across all the user equipment’s data and physical resources. FL relies on multiple sources of data and each of the clients is unevenly distributed. They are quite likely generating non-identical independent distributed (non-IID) data. Researchers have worked on multiple algorithms trying to resolve/increase the accuracy of model. However, there are many fields and gaps in the experimental study on understanding their advantages and disadvantages. Most of the previous studies have rather unreal data partitioning strategies among the clients, which are hardly representative of the real-world scenarios [2]. One major issue with FL is the data discrepancy among different locations and users moving among different locations. In this work, we investigate comprehensive data partitioning strategies to cover the typical non-independent identical distribution (i.i.d.) data cases. Moreover, we conduct extensive experiments to evaluate algorithms with various data partitioning strategies and how to recover the loss of accuracy. Our multiple ways of dividing data in non-IID settings brings significant challenges in learning accuracy of FL algorithms. We explore Simple CNN (Convolutions Neural Networks) models for training on data locally to the data zone. CNN models make use of Stochastic gradient descent to minimize the cost function. We alter the different parameters of SGD to learn its effects on the model training.

Additional Information

Publication
Thesis
Language: English
Date: 2022
Keywords
Fed ML, Federated Machine learning, Hyperparameters, IID, Non-IID, Stochastic Gradient Descent
Subjects
Machine learning
Random variables
Data privacy

Email this document to