Is there any way of evaluating clients' validation loss before FedAvg?

[Arthur Lee]

As title, say I am performing FedPub.

Is there anyway I can calculate and record the clients' validation loss after it tain locally and before the model is sent back for aggregation.

Thank you.

[Arthur Lee]

I already set
eval.split = ['validation']
eval.metrics = ['acc']
, but the output is still like this :

So is it correct that the code will only evaluate the metrics on train set no matter the eval.split setting?
And eval.split setting is for the evaluation of the server side?

Please give me any correction, I am confused, thanks!

[Weirui Kuang]

eval.split = ['test', 'val'] should be right.

For more details about usage of config, please refer to https://github.com/alibaba/FederatedScope/blob/llm/federatedscope/core/configs/README.md