Federated Learning (FL) was introduced by Google in 2016 as a decentralized approach for training machine learning models. Instead of aggregating data in one place, FL keeps data on local devices and uses locally computed updates to train a shared model. This involves multiple client nodes, each with a portion of data. The global model is trained through several communication rounds, where updates from local data are sent to a central server, which aggregates these updates. This process continues until specific goals, like performance benchmarks or a set number of rounds, are met.
Key Federated Learning Architectures
In centralized FL, a central server communicates with multiple client nodes, handling significant computational and storage tasks, which makes it easier to manage and protect data. Decentralized FL, on the other hand, omits the central server, with client nodes communicating directly with each other, reducing communication bottlenecks but increasing management complexity.
Data Partitioning in Federated Learning
Horizontal FL involves different data samples with similar features on each client node, increasing training samples from diverse sources. Vertical FL features overlapping data samples but different features on each node, expanding the feature space. Federated Transfer Learning combines data from different but related domains with minimal overlap, transferring knowledge from one domain to another.
Federated Learning Settings
Cross-device FL involves numerous mobile or IoT devices, facing hardware heterogeneity and higher communication costs. Cross-silo FL involves fewer participants, like organizations or data centers, with more reliable communication and less risk of malicious nodes.
Challenges in Federated Learning
FL faces several challenges, including high communication costs that can create bottlenecks, especially in cross-device settings. System heterogeneity, where devices vary in hardware and connectivity, can lead to slowdowns. Data heterogeneity, with local datasets potentially being biased or non-IID, complicates training. Ensuring privacy and security is also a challenge despite decentralized data storage. Finally, encouraging participation from data owners requires meaningful incentives.
EFRA Consortium’s Implementation
The EFRA consortium will implement a cross-silo FL system on MOY Park’s data, using separate virtual machines for different facilities. These machines will support machine learning training and prediction, with real-time data processing accessible via an API.
Federated Learning offers a promising approach to collaborative model training while maintaining data privacy and security. As the field evolves, it will address current challenges and expand its applications across various domains.
Article originally posted on Medium.
