Federated Learning: Training Models Without Collecting the Data

The usual way to train a model is to gather all the data in one place. For a lot of valuable data, that's impossible — patient records can't leave the hospital, keyboard input can't leave the phone, transaction data can't cross a bank's border. Federated learning inverts the process: instead of moving the data to the model, it moves the model to the data, trains locally, and combines only the learned updates. It has been running on billions of phones for years, and it's now moving into healthcare, finance and industry.

How it works

The basic loop, known as federated averaging:

1. A server sends the current global model to a set of participants (phones, hospitals, factories).
2. Each participant trains the model for a few steps on its own local data.
3. Each sends back only the resulting update — a change in the model's weights — not the data.
4. The server averages the updates, weighted by how much data each participant had, into a new global model.
5. Repeat for many rounds.

Two flavours dominate. Cross-device federation involves millions of unreliable, low-power clients each holding a little data — Google's Gboard next-word prediction and Apple's on-device personalisation are the canonical deployments. Cross-silo federation involves a handful of reliable institutions each holding a lot — hospitals training a tumour-segmentation model together, or banks building a fraud model without pooling transactions.

The privacy is not automatic

Keeping raw data local is a large improvement, but model updates are not harmless. Researchers have shown that gradients can be inverted to reconstruct training examples — including recognisable images — and that a model can memorise and leak rare records. Federated learning is therefore combined with additional protections:

• Secure aggregation. A cryptographic protocol that lets the server compute the sum of updates without ever seeing any individual one. Each participant masks its update with random values that cancel out only when all are combined.
• Differential privacy. Calibrated noise added to updates (or to the aggregate) so that the final model's dependence on any single participant's data is mathematically bounded. This provides a formal guarantee at the cost of some accuracy; the privacy budget (ε) makes the trade explicit.
• Trusted execution environments or homomorphic encryption for the aggregation step, in settings where a hardware or cryptographic guarantee is required.

A system that federates without these is better than centralised collection, but shouldn't be described as privacy-preserving in the formal sense.

Engineering realities

Federated training is harder than centralised training in ways that don't show up in the diagram.

• Non-IID data. Each participant's data is different — one user types in Spanish, one hospital sees older patients. Naive averaging can converge slowly or to a worse model. Algorithms like FedProx and SCAFFOLD, and personalisation layers, address this.
• Heterogeneous and unreliable clients. Phones drop out mid-round, have different compute and battery, and must only train when charging and on Wi-Fi. Cross-device systems assume most clients won't finish.
• Communication cost. Sending a full model update every round is expensive; compression, sparsification and training more local steps per round reduce it.
• Poisoning and Byzantine participants. A malicious client can send updates designed to backdoor the model. Robust aggregation (trimmed mean, median, anomaly detection on updates) mitigates but doesn't eliminate this, and secure aggregation makes it harder to spot bad actors because you can't inspect individual updates.
• Evaluation without data. You can't build a central test set from data you never collected. Evaluation itself has to be federated.

Where it's used

Beyond keyboards and voice assistants, the cross-silo applications are the ones growing fastest. Multi-hospital consortia have trained imaging and prognosis models across dozens of institutions without sharing scans; the results generalise better than single-site models precisely because they saw more diverse data. Banks use federation for anti-money-laundering models across borders. Manufacturers with plants in different countries pool predictive-maintenance learning. And the technique is central to improving on-device AI: the model that runs on your phone gets better from your usage without your usage being uploaded. Frameworks such as Flower, TensorFlow Federated, NVIDIA FLARE and OpenFL have made the infrastructure reasonably standard.

When to use it

Federated learning is the right tool when data cannot be centralised for legal, contractual or trust reasons and the parties nonetheless benefit from a shared model. It is not a way to avoid data governance — participants still need consent, purpose limitation and security — and it adds real engineering cost. If you can legally and safely centralise the data, do that. If you can't, federation with secure aggregation and differential privacy is how you train anyway.