Running inference on server

[lyscho]

Hi NVFlare-Team,

I'm wondering, whether it is possible to have a (global) test set on server and run inference with the current global model on the server dataset to evaluate its performance.
And is it generally possible to have the server execute and not just control/ distribute tasks?

Thank you already!

[YuanTingHsieh]

Hi @lyscho

Thanks for raising this discussion.

Yes, it is possible.

There are several places you can do this, to integrate into NVFlare system:

1. Add the logic into a "workflow", for example, if you are using the scatter and gather workflow, you could add the evaluation code at the end of each round: https://github.com/NVIDIA/NVFlare/blob/2.1.4/nvflare/app_common/workflows/scatter_and_gather.py#L237, you can copy these codes into the custom folder in your app, then add codes to them, just need to change your config_fed_server.json to use it.

2. Write a "widget/FLComponent" that listen to the AppEventType.AFTER_LEARNABLE_PERSIST event ((https://github.com/NVIDIA/NVFlare/blob/2.1.4/nvflare/app_common/workflows/scatter_and_gather.py#L232)) and do your evaluation logic there, an example to add a widget/FLComponent can refer to: https://github.com/NVIDIA/NVFlare/blob/2.1.4/examples/hello-pt-tb/config/config_fed_client.json#L34-L38.

Another way is that, you can write the "model persistor" and the final model will be saved on the server side.
You can always use any programs/library to do the inference/evaluation on server.
To write a custom "model persistor" you can refer to this example: https://github.com/NVIDIA/NVFlare/tree/2.1.4/examples/hello-tf2
https://github.com/NVIDIA/NVFlare/blob/2.1.4/examples/hello-tf2/custom/tf2_model_persistor.py

@holgerroth if you have other suggestions please also provide, thanks.

Hope this help.

[lyscho]

Thank you @YuanTingHsieh that was super helpful!
I ended up implementing a custom FLComponent that is triggered by the AppEventType.ROUND_DONE event.