Federated learning

Deep learning and artificial intelligence (AI) are becoming a part of everyday life, with organizations harnessing these technologies to improve, expedite and enhance processes. However, for many industries, such as healthcare and finance, securing training data can prove difficult.

With data privacy at the forefront of many people’s minds, organizations may struggle to attain enough of the right data to adequately train their AI models. Federated learning is a solution to this problem, offering a central platform for training models across multiple devices without sharing data.

What is federated learning?

Federated learning definition: Federated learning is a subset of machine learning that allows a model to be trained on datasets from multiple devices without the need to share raw data. This approach not only preserves data privacy but also enhances data security and complies with regulations. Also, federated learning allows for more accurate and robust models by using diverse data sources while maintaining data security.

Beyond data privacy and security, federated learning also encourages diversity by using data from multiple sources. With multiple datasets contributing to the overall learning, model and eventual output, federated learning ensures a model that is rooted in a wider range of data and data types, which boosts the accuracy and adaptability of a model.

Federated learning can use collaborative training techniques. By allowing multiple contributing bodies to train a single model on their varied data without having to share this raw data, a larger organization can benefit from collaborative group practices.

How does federated learning work?

Federated learning works by processing the data at its source, ensuring that sensitive information remains on local devices. Instead of sharing raw data, multiple parties collaborate by downloading a shared foundation model, training it on their private data and then encrypting and sharing the updated model.

Once the model is trained on one private dataset, it can be downloaded and trained by the next party. After all parties have trained and encrypted the model, it is ready to analyze new datasets from a range of sources.

Several different methods for federated learning exist, each differing in terms of the kind of data used. For example, some models are trained on large quantities of similar data types from disparate sources, while others are trained on a wide range of data types.

What is the history of federated learning?

Federated learning is a subset of machine learning that has emerged in recent years, driven by the evolution and rapid adoption of artificial intelligence systems.

The term “federated learning” was first introduced in 2016, in a paper by Google, which addressed the concept of decentralized training for machine learning models. In response to the growing issue of privacy breaches, Google developed new techniques for secure data training without the need to share raw data.

What are key types of federated learning?

There are several types of federated learning that vary mostly in the ways in which the model is trained on data and what kinds of data the models are trained on:

• Horizontal federated learning trains models on multiple datasets containing similar kinds of data. These datasets can be processed together easily.
• Vertical federated learning trains a model on multiple disparate but complementary datasets, enabling the model to produce a unified output.
• Federated transfer learning introduces pretrained foundation models to new datasets, allowing the original model to transfer its capabilities to perform new functions.
• Centralized federated learning coordinates the training process by collecting and aggregating model updates from all clients on a single server. This server acts as a central point for model aggregation and distribution.
• Decentralized federated learning lacks a central server. Instead, clients share their model updates directly with each other in a peer-to-peer manner. This approach can enhance security and privacy since there is no single point of failure or control.
• Heterogenous federated learning processes different data types in incompatible forms to train the model consistently. An adaptive step is required before the model’s training can be complete, and the contributing clients adapt their data and learning rate to achieve consistency within the training.

How is federated learning used?

Federated learning is a technology and method that can be applied to a range of machine learning applications. It offers a practical way to gather datasets without merging them at the source.

The key benefits to federated learning compared to other types of machine learning are security and data privacy. Therefore, the primary use cases for federated learning in AI are industries where data is inherently private and secure, such as healthcare.

Businesses within the healthcare industry use federated learning for gathering dispersed datasets about patients without compromising the privacy of those patients. One example would be a health insurance business compiling data about its clients from individual medical records, enabling hospitals and clinics to share private data securely.

Similarly, businesses within the finance industry may use federated learning to collaboratively benefit from more data to push fraud prevention and secure finance. This is an example of how interinstitutional data sharing can keep an industry secure.

Federated learning is also useful for developing new technologies. To facilitate innovation, researchers can securely share their data to benefit from each other’s data. An example of this is research to develop autonomous vehicles. Self-driving cars and other autonomous vehicles rely on machine learning technologies, models and algorithms that work with each other in real time, and there are a range of contributing models to consider. To develop autonomous vehicles and the technologies that power them, organizations responsible for individual elements of this technology can collaborate through federated learning to enable autonomous vehicles to respond more effectively to real-time stimuli.

Innovation is ongoing to develop more usable iterations of the technology, and different automotive brands are working within this space and have different test data. By sharing this data using federated learning, artificial intelligence models can offer insights, predictions and analysis that can propel AV development further.