Central Learning(CL) vs Federated Learning(FL)

[pygabc1]

Hi, can the performance of FL be better than the one of CL (with imbalanced-clients)? Central Learning (CL) is trained with datasets combined by all clients and tested by test data each client. I used two clients, and the data size of one client is three times larger than the one of other client. In larger client, the performance of FL is better than the one of CL. So far, I couldn't find any publication about this case. I used linux OS and 3.8 python, and 2.3 NVFlare version. Thank you.

[YuanTingHsieh]

I think it is possible. It depends on how you setup your CL and FL training pipelines right?
Like how you sample the mini-batches and how many epochs do you run for FL and CL?

When you say CL, you say you "trained with datasets combined by all clients and tested by test data each client.".

Let's assume client 1 has large dataset and client 2 has small dataset.

What if you try scenarios where:

• "trained with datasets on the client 1 and tested by test data of client 1"
• "trained with datasets on the client 1 and tested by test data of client 2"
• "trained with datasets on the client 2 and tested by test data of client 2"
• "trained with datasets on the client 2 and tested by test data of client 1"

And for FL you can also try FedAvg and other workflows such as Cyclic.

You might observe better results in Cyclic as well.

@holgerroth and @ZiyueXu77 please comment more in details, thanks!

[pygabc1]

Thank you, Yuan Ting. Then, is any way to apply Federated K-means, https://github.com/NVIDIA/NVFlare/tree/2.3/examples/advanced/sklearn-kmeans, to reduce imbalanced client issue to my own dataset? Or is sklearn-kmeans only for iris data?

[holgerroth]

kmeans can be adapted to different datasets. It uses a csv file reader at the moment but even that can be changed to others if needed.

Regarding your question about reducing the gap between FL and CL, take a look at our research on that topic https://github.com/NVIDIA/NVFlare/tree/main/research/fed-sm